Automated detection of cells from immunohistochemically-stained tissues: Application to Ki-67 nuclei staining

Abstract

An automated cell nuclei detection algorithm is described to be used for the quantification of immunohistochemically-stained tissues. Detection and segmentation of positively stained cells and their separation from the background and negatively-stained cells is crucial for fast, accurate, consistent and objective analysis of pathology images. One of the major challenges is the identification, hence accurate counting of individual cells, when these cells form clusters. To identify individual cell nuclei within clusters, we propose a new cell nuclei detection method based on the well-known watershed segmentation, which can lead to under- or over-segmentation for this problem. Our algorithm handles over-segmentation by combining H-minima transformed watershed algorithm with a novel region merging technique. To handle under-segmentation problem, we develop a Laplacian-of-Gaussian (LoG) filtering based blob detection algorithm, which estimates the range of the scales from the image adaptively. An SVM classifier was trained in order to separate non-touching single cells and touching cell clusters with five features representing connected region properties such as eccentricity, area, perimeter, convex area and perimeter-to-area ratio. Classified touching cell clusters are segmented with the H-minima based watershed algorithm. The resulting over-segmented regions are improved with the merging algorithm. The remaining under-segmented cell clusters are convolved with LoG filters to detect the cells within them. Cell-by-cell nucleus detection performance is evaluated by comparing computer detections with cell locations manually marked by eight pathology residents. The sensitivity is 89% when the cells are marked as positive at least by one resident and it increases to 99% when the evaluated cells are marked by all eight residents. In comparison, the average reader sensitivity varies between 70% +/- 18% and 95% +/- 11%.