Cellpose API Guide¶
Cellpose class¶

class
cellpose.models.
Cellpose
(gpu=False, model_type='cyto', net_avg=True, batch_size=8, device=None)[source]¶ main model which combines SizeModel and CellposeModel
 Parameters
gpu (bool (optional, default False)) – whether or not to save model to GPU, will check if GPU available
model_type (str (optional, default 'cyto')) – ‘cyto’=cytoplasm model; ‘nuclei’=nucleus model
net_avg (bool (optional, default True)) – loads the 4 builtin networks and averages them if True, loads one network if False
batch_size (int (optional, default 8)) – number of 224x224 patches to run simultaneously on the GPU (can make smaller or bigger depending on GPU memory usage)
device (mxnet device (optional, default None)) – where model is saved (mx.gpu() or mx.cpu()), overrides gpu input, recommended if you want to use a specific GPU (e.g. mx.gpu(4))

eval
(x, channels=None, diameter=30.0, invert=False, do_3D=False, net_avg=True, tile=True, flow_threshold=0.4, cellprob_threshold=0.0, rescale=None, progress=None)[source]¶ run cellpose and get masks
 Parameters
x (list or array of images) – can be list of 2D/3D images, or array of 2D/3D images, or 4D image array
channels (list (optional, default None)) – list of channels, either of length 2 or of length number of images by 2. First element of list is the channel to segment (0=grayscale, 1=red, 2=blue, 3=green). Second element of list is the optional nuclear channel (0=none, 1=red, 2=blue, 3=green). For instance, to segment grayscale images, input [0,0]. To segment images with cells in green and nuclei in blue, input [2,3]. To segment one grayscale image and one image with cells in green and nuclei in blue, input [[0,0], [2,3]].
diameter (float (optional, default 30.)) – if set to None, then diameter is automatically estimated if size model is loaded
invert (bool (optional, default False)) – invert image pixel intensity before running network
do_3D (bool (optional, default False)) – set to True to run 3D segmentation on 4D image input
net_avg (bool (optional, default True)) – runs the 4 builtin networks and averages them if True, runs one network if False
tile (bool (optional, default True)) – tiles image for test time augmentation and to ensure GPU memory usage limited (recommended)
flow_threshold (float (optional, default 0.4)) – flow error threshold (all cells with errors below threshold are kept) (not used for 3D)
cellprob_threshold (float (optional, default 0.0)) – cell probability threshold (all pixels with prob above threshold kept for masks)
rescale (float (optional, default None)) – if diameter is set to None, and rescale is not None, then rescale is used instead of diameter for resizing image
progress (pyqt progress bar (optional, default None)) – to return progress bar status to GUI
 Returns
masks (list of 2D arrays, or single 3D array (if do_3D=True)) – labelled image, where 0=no masks; 1,2,…=mask labels
flows (list of lists 2D arrays, or list of 3D arrays (if do_3D=True)) – flows[k][0] = XY flow in HSV 0255 flows[k][1] = flows at each pixel flows[k][2] = the cell probability centered at 0.0
styles (list of 1D arrays of length 64, or single 1D array (if do_3D=True)) – style vector summarizing each image, also used to estimate size of objects in image
diams (list of diameters, or float (if do_3D=True))
CellposeModel¶

class
cellpose.models.
CellposeModel
(gpu=False, pretrained_model=False, batch_size=8, diam_mean=27.0, net_avg=True, device=None, unet=False)[source]¶  Parameters
gpu (bool (optional, default False)) – whether or not to save model to GPU, will check if GPU available
pretrained_model (str or list of strings (optional, default False)) – path to pretrained cellpose model(s), if False, no model loaded; if None, builtin ‘cyto’ model loaded
net_avg (bool (optional, default True)) – loads the 4 builtin networks and averages them if True, loads one network if False
batch_size (int (optional, default 8)) – number of 224x224 patches to run simultaneously on the GPU (can make smaller or bigger depending on GPU memory usage)
diam_mean (float (optional, default 27.)) – mean ‘diameter’, 27. is built in value for ‘cyto’ model
device (mxnet device (optional, default None)) – where model is saved (mx.gpu() or mx.cpu()), overrides gpu input, recommended if you want to use a specific GPU (e.g. mx.gpu(4))

eval
(x, channels=None, invert=False, rescale=None, do_3D=False, net_avg=True, tile=True, flow_threshold=0.4, cellprob_threshold=0.0, compute_masks=True, progress=None)[source]¶ segment list of images x, or 4D array  Z x nchan x Y x X
 Parameters
x (list or array of images) – can be list of 2D/3D images, or array of 2D/3D images, or 4D image array
channels (list (optional, default None)) – list of channels, either of length 2 or of length number of images by 2. First element of list is the channel to segment (0=grayscale, 1=red, 2=blue, 3=green). Second element of list is the optional nuclear channel (0=none, 1=red, 2=blue, 3=green). For instance, to segment grayscale images, input [0,0]. To segment images with cells in green and nuclei in blue, input [2,3]. To segment one grayscale image and one image with cells in green and nuclei in blue, input [[0,0], [2,3]].
invert (bool (optional, default False)) – invert image pixel intensity before running network
rescale (float (optional, default None)) – resize factor for each image, if None, set to 1.0
do_3D (bool (optional, default False)) – set to True to run 3D segmentation on 4D image input
net_avg (bool (optional, default True)) – runs the 4 builtin networks and averages them if True, runs one network if False
tile (bool (optional, default True)) – tiles image for test time augmentation and to ensure GPU memory usage limited (recommended)
flow_threshold (float (optional, default 0.4)) – flow error threshold (all cells with errors below threshold are kept) (not used for 3D)
cellprob_threshold (float (optional, default 0.0)) – cell probability threshold (all pixels with prob above threshold kept for masks)
compute_masks (bool (optional, default True)) – Whether or not to compute dynamics and return masks. This is set to False when retrieving the styles for the size model.
progress (pyqt progress bar (optional, default None)) – to return progress bar status to GUI
 Returns
masks (list of 2D arrays, or single 3D array (if do_3D=True)) – labelled image, where 0=no masks; 1,2,…=mask labels
flows (list of lists 2D arrays, or list of 3D arrays (if do_3D=True)) – flows[k][0] = XY flow in HSV 0255 flows[k][1] = flows at each pixel flows[k][2] = the cell probability centered at 0.0
styles (list of 1D arrays of length 64, or single 1D array (if do_3D=True)) – style vector summarizing each image, also used to estimate size of objects in image
SizeModel¶

class
cellpose.models.
SizeModel
(cp_model, device=cpu(0), pretrained_size=None, **kwargs)[source]¶ linear regression model for determining the size of objects in image used to rescale before input to CellposeModel uses styles from CellposeModel
 Parameters
cp_model (CellposeModel) – cellpose model from which to get styles
device (mxnet device (optional, default mx.cpu())) – where cellpose model is saved (mx.gpu() or mx.cpu())
pretrained_size (str) – path to pretrained size model

eval
(x=None, style=None, channels=None, invert=False, tile=True, batch_size=8, progress=None)[source]¶ use images x to produce style or use style input to predict size of objects in image
Object size estimation is done in two steps: 1. use a linear regression model to predict size from style in image 2. resize image to predicted size and run CellposeModel to get output masks.
Take the median object size of the predicted masks as the final predicted size.
 Parameters
cp_model (CellposeModel) – cellpose model from which to get styles
device (mxnet device (optional, default mx.cpu())) – where cellpose model is saved (mx.gpu() or mx.cpu())
pretrained_size (str) – path to pretrained size model
Metrics¶

cellpose.metrics.
average_precision
(masks_true, masks_pred, threshold=[0.5, 0.75, 0.9])[source]¶ average precision estimation: AP = TP / (TP + FP + FN)
This function is based heavily on the fast stardist matching functions (https://github.com/mpicbgcsbd/stardist/blob/master/stardist/matching.py)
 Parameters
masks_true (list of NDarrays (int) or NDarray (int)) – where 0=NO masks; 1,2… are mask labels
masks_pred (list of NDarrays (int) or NDarray (int)) – NDarray (int) where 0=NO masks; 1,2… are mask labels
 Returns
ap (array [len(masks_true) x len(threshold)]) – average precision at thresholds
tp (array [len(masks_true) x len(threshold)]) – number of true positives at thresholds
fp (array [len(masks_true) x len(threshold)]) – number of false positives at thresholds
fn (array [len(masks_true) x len(threshold)]) – number of false negatives at thresholds

cellpose.metrics.
flow_error
(maski, dP_net)[source]¶ error in flows from predicted masks vs flows predicted by network run on image
This function serves to benchmark the quality of masks, it works as follows 1. The predicted masks are used to create a flow diagram 2. The maskflows are compared to the flows that the network predicted
If there is a discrepancy between the flows, it suggests that the mask is incorrect. Masks with flow_errors greater than 0.4 are discarded by default. Setting can be changed in Cellpose.eval or CellposeModel.eval.
 Parameters
maski (NDarray (int)) – masks produced from running dynamics on dP_net, where 0=NO masks; 1,2… are mask labels
dP_net (NDarray (float)) – ND flows where dP_net.shape[1:] = maski.shape
 Returns
flow_errors (float array with length maski.max()) – mean squared error between predicted flows and flows from masks
dP_masks (NDarray (float)) – ND flows produced from the predicted masks
Flows to masks¶

cellpose.dynamics.
fill_holes
(masks, min_size=15)[source]¶ fill holes in masks (2D) and discard masks smaller than min_size
fill holes in each mask using scipy.ndimage.morphology.binary_fill_holes
 Parameters
masks (int, 2D array) – labelled masks, 0=NO masks; 1,2,…=mask labels, size [Ly x Lx]
min_size (int (optional, default 15)) – minimum number of pixels per mask
 Returns
masks – masks with holes filled and masks smaller than min_size removed, 0=NO masks; 1,2,…=mask labels, size [Ly x Lx]
 Return type
int, 2D array

cellpose.dynamics.
follow_flows
(dP, niter=200)[source]¶ define pixels and run dynamics to recover masks in 2D
Pixels are meshgrid. Only pixels with nonzero cellprobability are used (as defined by inds)
 Parameters
dP (float32, 3D or 4D array) – flows [axis x Ly x Lx] or [axis x Lz x Ly x Lx]
niter (int (optional, default 200)) – number of iterations of dynamics to run
 Returns
p – final locations of each pixel after dynamics
 Return type
float32, 3D array

cellpose.dynamics.
get_masks
(p, iscell=None, rpad=20, flows=None, threshold=0.4)[source]¶ create masks using pixel convergence after running dynamics
Makes a histogram of final pixel locations p, initializes masks at peaks of histogram and extends the masks from the peaks so that they include all pixels with more than 2 final pixels p. Discards masks with flow errors greater than the threshold.
 Parameters
p (float32, 3D or 4D array) – final locations of each pixel after dynamics, size [axis x Ly x Lx] or [axis x Lz x Ly x Lx].
iscell (bool, 2D or 3D array) – if iscell is not None, set pixels that are iscell False to stay in their original location.
rpad (int (optional, default 20)) – histogram edge padding
threshold (float (optional, default 0.4)) – masks with flow error greater than threshold are discarded (if flows is not None)
flows (float, 3D or 4D array (optional, default None)) – flows [axis x Ly x Lx] or [axis x Lz x Ly x Lx]. If flows is not None, then masks with inconsistent flows are removed using remove_bad_flow_masks.
 Returns
M0 – masks with inconsistent flow masks removed, 0=NO masks; 1,2,…=mask labels, size [Ly x Lx] or [Lz x Ly x Lx]
 Return type
int, 2D or 3D array

cellpose.dynamics.
labels_to_flows
(labels)[source]¶ convert labels (list of masks or flows) to flows for training model
 Parameters
labels (list of NDarrays) – labels[k] can be 2D or 3D, if [3 x Ly x Lx] then it is assumed that flows were precomputed. Otherwise labels[k][0] or labels[k] (if 2D) is used to create flows and cell probabilities.
 Returns
flows – flows[k][0] is cell probability, flows[k][1] is Y flow, and flows[k][2] is X flow
 Return type
list of [3 x Ly x Lx] arrays

cellpose.dynamics.
masks_to_flows
(masks)[source]¶ convert masks to flows using diffusion from center pixel
Center of masks where diffusion starts is defined to be the closest pixel to the median of all pixels that is inside the mask. Result of diffusion is converted into flows by computing the gradients of the diffusion density map.
 Parameters
masks (int, 2D or 3D array) – labelled masks 0=NO masks; 1,2,…=mask labels
 Returns
mu (float, 3D or 4D array) – flows in Y = mu[2], flows in X = mu[1]. if masks are 3D, flows in Z = mu[0].
mu_c (float, 2D or 3D array) – for each pixel, the distance to the center of the mask in which it resides

cellpose.dynamics.
remove_bad_flow_masks
(masks, flows, threshold=0.4)[source]¶ remove masks which have inconsistent flows
Uses metrics.flow_error to compute flows from predicted masks and compare flows to predicted flows from network. Discards masks with flow errors greater than the threshold.
 Parameters
masks (int, 2D or 3D array) – labelled masks, 0=NO masks; 1,2,…=mask labels, size [Ly x Lx] or [Lz x Ly x Lx]
flows (float, 3D or 4D array) – flows [axis x Ly x Lx] or [axis x Lz x Ly x Lx]
threshold (float (optional, default 0.4)) – masks with flow error greater than threshold are discarded.
 Returns
masks – masks with inconsistent flow masks removed, 0=NO masks; 1,2,…=mask labels, size [Ly x Lx] or [Lz x Ly x Lx]
 Return type
int, 2D or 3D array

cellpose.dynamics.
steps2D
(p, dP, inds, niter)[source]¶ run dynamics of pixels to recover masks in 2D
Euler integration of dynamics dP for niter steps
 Parameters
p (float32, 3D array) – pixel locations [axis x Ly x Lx] (start at initial meshgrid)
dP (float32, 3D array) – flows [axis x Ly x Lx]
inds (int32, 2D array) – nonzero pixels to run dynamics on [npixels x 2]
niter (int32) – number of iterations of dynamics to run
 Returns
p – final locations of each pixel after dynamics
 Return type
float32, 3D array

cellpose.dynamics.
steps3D
(p, dP, inds, niter)[source]¶ run dynamics of pixels to recover masks in 3D
Euler integration of dynamics dP for niter steps
 Parameters
p (float32, 4D array) – pixel locations [axis x Lz x Ly x Lx] (start at initial meshgrid)
dP (float32, 4D array) – flows [axis x Lz x Ly x Lx]
inds (int32, 2D array) – nonzero pixels to run dynamics on [npixels x 3]
niter (int32) – number of iterations of dynamics to run
 Returns
p – final locations of each pixel after dynamics
 Return type
float32, 4D array
Image transforms¶

cellpose.transforms.
average_tiles
(y, ysub, xsub, Ly, Lx)[source]¶ average results of network over tiles
 Parameters
y (float, [ntiles x 3 x bsize x bsize]) – output of cellpose network for each tile
ysub (list) – list of arrays with start and end of tiles in Y of length ntiles
xsub (list) – list of arrays with start and end of tiles in X of length ntiles
Ly (int) – size of pretiled image in Y (may be larger than original image if image size is less than bsize)
Lx (int) – size of pretiled image in X (may be larger than original image if image size is less than bsize)
 Returns
yf – network output averaged over tiles
 Return type
float32, [3 x Ly x Lx]

cellpose.transforms.
make_tiles
(imgi, bsize=224, augment=True)[source]¶ make tiles of image to run at testtime
 there are 4 versions of each tile
original
flipped vertically
flipped horizontally
flipped vertically and horizontally
 Parameters
imgi (float32) – array that’s nchan x Ly x Lx
 Returns
IMG (float32) – array that’s ntiles x nchan x bsize x bsize
ysub (list) – list of arrays with start and end of tiles in Y of length ntiles
xsub (list) – list of arrays with start and end of tiles in X of length ntiles
Ly (int) – size of total image pretiling in Y (may be larger than original image if image size is less than bsize)
Lx (int) – size of total image pretiling in X (may be larger than original image if image size is less than bsize)

cellpose.transforms.
normalize99
(img)[source]¶ normalize image so 0.0 is 1st percentile and 1.0 is 99th percentile

cellpose.transforms.
normalize_img
(img)[source]¶ normalize each channel of the image so that so that 0.0=1st percentile and 1.0=99th percentile of image intensities
 Parameters
img (NDarray) – image of size [nchan x Ly x Lx]
 Returns
img – normalized image of size [nchan x Ly x Lx]
 Return type
NDarray, float32

cellpose.transforms.
pad_image_ND
(img0, div=16, extra=1)[source]¶ pad image for testtime so that its dimensions are a multiple of 16 (2D or 3D)
 Parameters
img0 (NDarray) – image of size [nchan (x Lz) x Ly x Lx]
div (int (optional, default 16)) –
 Returns
I (NDarray) – padded image
ysub (array, int) – yrange of pixels in I corresponding to img0
xsub (array, int) – xrange of pixels in I corresponding to img0

cellpose.transforms.
random_rotate_and_resize
(X, Y=None, scale_range=1.0, xy=224, 224, do_flip=True, rescale=None)[source]¶ augmentation by random rotation and resizing
X and Y are lists or arrays of length nimg, with dims channels x Ly x Lx (channels optional)
 Parameters
X (list of NDarrays, float) – list of image arrays of size [nchan x Ly x Lx] or [Ly x Lx]
Y (list of NDarrays, float (optional, default None)) – list of image labels of size [nlabels x Ly x Lx] or [Ly x Lx]. The 1st channel of Y is always nearestneighbor interpolated (assumed to be masks or 01 representation). If Y.shape[0]==3, then the labels are assumed to be [cell probability, Y flow, X flow].
scale_range (float (optional, default 1.0)) – Range of resizing of images for augmentation. Images are resized by (1scale_range/2) + scale_range * np.random.rand()
xy (tuple, int (optional, default (224,224))) – size of transformed images to return
do_flip (bool) – whether or not to flip images horizontally
rescale (array, float (optional, default None)) – how much to resize images by before performing augmentations
 Returns
imgi (NDarray, float) – transformed images in array [nimg x nchan x xy[0] x xy[1]]
lbl (NDarray, float) – transformed labels in array [nimg x nchan x xy[0] x xy[1]]
scale (array, float) – amount each image was resized by

cellpose.transforms.
reshape
(data, channels=[0, 0], invert=False)[source]¶ reshape data using channels and normalize intensities (w/ optional inversion)
 Parameters
data (numpy array that's (Z x ) Ly x Lx x nchan) –
channels (list of int of length 2 (optional, default [0,0])) – First element of list is the channel to segment (0=grayscale, 1=red, 2=blue, 3=green). Second element of list is the optional nuclear channel (0=none, 1=red, 2=blue, 3=green). For instance, to train on grayscale images, input [0,0]. To train on images with cells in green and nuclei in blue, input [2,3].
invert (bool) – invert intensities
 Returns
data
 Return type
numpy array that’s nchan x (Z x ) Ly x Lx

cellpose.transforms.
reshape_data
(train_data, test_data=None, channels=None)[source]¶ inputs converted to correct shapes for training and rescaled so that 0.0=1st percentile and 1.0=99th percentile of image intensities in each channel.
 Parameters
train_data (list of NDarrays, float) – list of training images of size [Ly x Lx], [nchan x Ly x Lx], or [Ly x Lx x nchan]
test_data (list of NDarrays, float (optional, default None)) – list of testing images of size [Ly x Lx], [nchan x Ly x Lx], or [Ly x Lx x nchan]
channels (list of int of length 2 (optional, default None)) – First element of list is the channel to segment (0=grayscale, 1=red, 2=blue, 3=green). Second element of list is the optional nuclear channel (0=none, 1=red, 2=blue, 3=green). For instance, to train on grayscale images, input [0,0]. To train on images with cells in green and nuclei in blue, input [2,3].
 Returns
train_data (list of NDarrays, float) – list of training images of size [2 x Ly x Lx]
test_data (list of NDarrays, float (optional, default None)) – list of testing images of size [2 x Ly x Lx]
run_test (bool) – whether or not test_data was correct size and is useable during training
Plot functions¶

cellpose.plot.
image_to_rgb
(img0, channels=[0, 0])[source]¶ image is 2 x Ly x Lx or Ly x Lx x 2  change to RGB Ly x Lx x 3

cellpose.plot.
interesting_patch
(mask, bsize=130)[source]¶ get patch of size bsize x bsize with most masks

cellpose.plot.
mask_overlay
(img, masks, colors=None)[source]¶ overlay masks on image (set image to grayscale)
 Parameters
img (int or float, 2D or 3D array) – img is of size [Ly x Lx (x nchan)]
masks (int, 2D array) – masks where 0=NO masks; 1,2,…=mask labels
colors (int, 2D array (optional, default None)) – size [nmasks x 3], each entry is a color in 0255 range
 Returns
RGB – array of masks overlaid on grayscale image
 Return type
uint8, 3D array

cellpose.plot.
masks_to_outlines
(masks)[source]¶ get outlines of masks as a 01 array
 Parameters
masks (int, 2D array) – size [Ly x Lx], 0=NO masks; 1,2,…=mask labels
 Returns
outlines – size [Ly x Lx], True pixels are outlines
 Return type
bool, 2D array

cellpose.plot.
outlines_list
(masks)[source]¶ get outlines of masks as a list to loop over for plotting

cellpose.plot.
show_segmentation
(fig, img, maski, flowi, channels=[0, 0], file_name=None)[source]¶ plot segmentation results (like on website)
Can save each panel of figure with file_name option. Use channels option if img input is not an RGB image with 3 channels.
 Parameters
fig (matplotlib.pyplot.figure) – figure in which to make plot
img (2D or 3D array) – image input into cellpose
maski (int, 2D array) – for image k, masks[k] output from Cellpose.eval, where 0=NO masks; 1,2,…=mask labels
flowi (int, 2D array) – for image k, flows[k][0] output from Cellpose.eval (RGB of flows)
channels (list of int (optional, default [0,0])) – channels used to run Cellpose, no need to use if image is RGB
file_name (str (optional, default None)) – file name of image, if file_name is not None, figure panels are saved