Complex Cell Segmentation and Its Significance in Spatial Biology

In research, scientists are continually pushing boundaries to gain deeper insights into biology, medicine, and drug discovery. Single-cell spatial imaging has emerged as a powerful technology in the field of genomics, enabling researchers to study the spatial organization of cells and analyze their molecular characteristics at a singular level. One critical aspect of this pursuit is the ability to accurately segment complex-shaped cells, a capability that is rapidly evolving in the field of spatial molecular imaging. This article delves into the importance of cell segmentation in spatial biology, the challenges researchers face, and how cutting-edge technology, such as the CosMx™ Spatial Molecular Imager (SMI), is revolutionizing cell segmentation.

Why is Cell Segmentation Important?

Understanding Cellular Interactions: Cell segmentation allows scientists to explore cellular interactions in normal and pathological conditions, enhancing our understanding of fundamental biological processes.

Analyzing Biological Features: Cell segmentation empowers scientists to analyze essential biological features like cell count, type, , shape, and more. This analysis provides insights into how these features change over time and in response to different conditions.

Capturing Biologically Relevant Information: The morphology of cells reflects their physiological state, and effective cell segmentation captures biologically relevant morphological information.

Accelerating Drug Discovery: Insights gained from cell segmentation can drive drug discovery, diagnostics, and various other critical areas in biology, pharmacology, and personalized medicine.

In summary, cell segmentation is a cornerstone of single-cell spatial research, playing a fundamental role in understanding health and disease.

Why is Accurate Cell Segmentation Challenging?

Accurate cell segmentation can be a daunting task due to several factors:

  • Heterogeneous Shapes: Cells come in diverse and dynamic shapes, making it nearly impossible to define mathematical shape models.
  • Variation in Size and Shape: Unlike nuclei, which are typically similar-sized structures, the cytoplasm exhibits significant variations in shape and size.
  • Weak Boundary Gradients: Cells that are in close proximity can have weak boundary gradients, making it challenging to separate them.
  • Makeshift Nature of Segmentation Approaches: The applicability of segmentation methods can be limited by dataset constraints, including differences in staining or imaging modality, artifacts in image capture, or morphological differences.

These challenges underscore the complexity of cell segmentation and the need for advanced techniques to achieve accurate results in single-cell spatial research.

Why the CosMx™ SMI Excels in Cell Segmentation

CosMx SMI utilizes a sophisticated cell segmentation algorithm based on the enhanced CellPose algorithm.1Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nature Methods 18,100-106 (2021). This algorithm combines information from cell membrane proteins, nuclei, and RNA. It has been rigorously trained and tested on a wide range of tissue-type specimens, representing over 50 million single cells. The results are impressive:

  • CosMx SMI effectively segments complex-shaped cells, surpassing platforms using nuclear expansion segmentation strategies, especially for dispersed or complex cells.
  • For RNA assays, CosMx SMI natively provides 5-channel multiplexed immunofluorescence staining accurately capture cell shape and assign transcript to the correct cell.
  • For protein or multiomics assays, a robust pipeline combines high-plex protein images with machine learning cell segmentation models, ensuring accurate cell boundary detection as well as downstream analytics for cell typing, neighbor analysis and more.

The CosMx SMI technology continues to advance complex-shaped cell segmentation algorithms minimizing imaging-based segmentation errors. Nanostring scientists explained the CosMx SMI cell segmentation technology in more detail in a recent white paper titled “Evaluating the Technical Performance of Single-Cell Spatial Molecular Imaging Technologies.”  Below are a few of the figures from this white paper:

Illustration showing how mRNA can be mis-assigned with incorrect cell boundaries.
The impact of minor segmentation errors, leading to the assignment of incorrect genes to a cell, subsequently giving rise to artifacts in downstream data analysis such as clustering, differential gene expression, and pathway analysis.
Mouse Neuro tissues images from CosMx
A & B: The CosMx assay uses antibody-based protein detection morphology markers (cell membrane and nuclei) for cell segmentation, whereas (C) other spatial imagers often utilize simple nuclear expansion (~15 µm) for cell segmentation as shown here with the same image using simplified polygons for ease of visualization.

In conclusion, accurate cell segmentation is a vital factor in selecting a Spatial Imager, and the CosMx SMI technology addresses this challenge comprehensively. As technology evolves, the CosMx SMI will further enhance cell segmentation data, enabling ultra-high-plex protein and RNA data, setting new standards for cell segmentation quality.


  • 1
    Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nature Methods 18,100-106 (2021).
Relevant Products
By Lisa Kessler
For research use only. Not for use in diagnostic procedures.