Downloading Data

Here, we will use the ImageryClient to get some data from the Kasthuri et al. 2014 dataset hosted by Google. In its simplest form, we just intialize an ImageryClient object with an image cloudpath and a segmentation cloudpath. Values are taken from the layers in the linked neuroglancer state.

import imageryclient as ic

img_src = 'precomputed://gs://neuroglancer-public-data/kasthuri2011/image_color_corrected'
seg_src = 'precomputed://gs://neuroglancer-public-data/kasthuri2011/ground_truth'

img_client = ic.ImageryClient(image_source=img_src, segmentation_source=seg_src)

Imagery cutouts

Cutouts are defined by their bounds, which can be specified by providing bounds. The most direct form is a pair of points representing the upper and lower corners of the bounding box. By default, coordinates are in the default resolution of the imagery as you would see in neuroglancer.

bounds = [
    [5119, 8477, 1201],
    [5519, 8877, 1202]
]

image = img_client.image_cutout(bounds)

# Use PIL to visualize
from PIL import Image
Image.fromarray(image.T)

Figure 1: A 2d image cutout at full resolution specified by bounds

Since often we are using analysis points to center an image on, we can alternatively define a center point and the width/height/depth of the bounding box (in voxels). The same image could be achived from this specification.

ctr = [5319, 8677, 1201]
img_width = 400
image = img_client.image_cutout(ctr, bbox_size=(img_width, img_width))

You can also generate bounds from a center and size.

bounds = ic.bounds_from_center(ctr, width=img_width, height=img_width, depth=1)

Resolution

A very important element in ImageryClient is the resolution, which specifies the units that you are using when providing bounds. You can check the resolution that the client is expecting with img_client.resolution. The resolution will default to the highest resolution available for the imagery, but you can specify another resolution manually. For example, to say that you are going to provide bounds in [8,8,30] voxel units, you would add the resolution argument:

img_client = ic.ImageryClient(..., resolution=[8,8,30])

You can also explicitly set the resolution to "image" or "segmentation" to use the highest available resolution available for either. Resolution can also be specified in each of the functions for image or segmentation cutouts, but will default to the client values.

Note that the volumetric data itself is not necessarily at the resolution specified, but rather this parameter determines how to interpret the coordinates. The resolution is set by the image and segmentation mip levels, which can be added either when creating the ImageryClient instance or when doing any cutout download. By default, ImageryClient will use the highest resolution mip level that is not labeled as a "placeholder" in CloudVolume.

Specifying image size instead of field of view

When upper and lower bounds are specified or a bbox_size is used, the resolution will change with mip level but the field of view that is downloaded will remain the same. Alternatively, one might want to download an image with a specific size in pixels and a specific mip level without having to calculate what bounding box would get you that.. This can be done in image_cutout by specifying the center point in the place of bounds and also specify image_size as a 2- or 3-element array. In this case, the center point will be adjusted according to the resolutions specified, while the field of view will change with image size.

In practice, this only is needed for non-default mip levels. If you specify mip level, this approach will always yield an image with the same size while a bounds-based approach will get smaller with increasing mips as the effective resolution gets coarser.

For example, using bounds:

image = img_client.image_cutout(bounds, mip=3)
Image.fromarray(image.T)

Figure 2: Imagery at MIP 3 resolution within specific bounds

And using specified pixel dimensions:

img_size=(400, 400)
image = img_client.image_cutout(ctr, mip=3, image_size=img_size)
Image.fromarray(image.T)

Figure 3: Imagery at MIP 3 with the specified image size

You can also use the scale_to_bounds=True argument to upscale an image to the size specified in the bounding box, equivalent to having one pixel for each voxel as measured by the resolution parameter.

Segmentations

An aligned segmentation cutout is retrieved similarly. Note that segmentations show segment ids, and are not directly visualizable. However, in this case we can convert to a uint8 greyscale and see the gist, although there are many better approaches to coloring segmentations that will be shown later. Note that for dynamic segmentations, you can use the timestamp parameter to (optionally) set the time at which segmentation will e looked up.

seg = img_client.segmentation_cutout(bounds)

import numpy as np
Image.fromarray( (seg.T / np.max(seg) * 255).astype('uint8') )

Figure 4: Segmentation of all objects within bounds

Specific root ids can also be specified. All pixels outside those root ids have a value of 0.

root_ids = [2282, 4845]
seg = img_client.segmentation_cutout(bounds, root_ids=root_ids)
Image.fromarray( (seg.T / np.max(seg) * 255).astype('uint8') )

Figure 5: segmentation of specific objects within bounds

Split segmentations

It's often convenient to split out the segmentation for each root id as a distinct mask. These "split segmentations" come back as a dictionary with root id as key and binary mask as value.

split_seg = img_client.split_segmentation_cutout(bounds, root_ids=root_ids)

Image.fromarray((split_seg[ root_ids[0] ].T * 255).astype('uint8'))

Figure 6: A single object's segmentation mask from the dictionary of segmentation masks.

Aligned cutouts

Aligned image and segmentations can be downloaded in one call, as well. If the lowest mip data in each differs in resolution, the lower resolution data will be optionally upsampled to the higher resolution in order to produce aligned overlays. Root ids and split segmentations can be optionally specified. This is the best option if your primary goal is overlay images.

image, segs = img_client.image_and_segmentation_cutout(bounds,
                                                       split_segmentations=True,
                                                       root_ids=root_ids)

Note that image_size is not an option for joint image and segmentation calls, because it's not clear which bounds to use. If this is needed, you can use the img_client.segmentation_bbox_size_from_dimensions or img_client.image_bbox_size_from_dimensions to get the appropriate bbox_size argument for a segmentation-based image dimension (or image-based, respectively).

Using with CAVEclient

While the ImageryClient was designed to work specifically with the CAVEclient and its associated suite of services, it should work with any cloudvolume project.

However, if you are working within an CAVEclient-compatible project, a CAVEclient can be used to help configure the ImageryClient, filling in the imagery source, the segmentation source, authentication information, and the default resolution used in Neuroglancer.

For example, we can download the data around this view of the MICRoNs mouse visual cortex data from Neuroglancer.

from caveclient import CAVEclient
client = CAVEclient('minnie65_public') .  # Note that you have to set up a token for this to work, see below.

img_client = ic.ImageryClient(client=client)

ctr = [240640, 207872, 21360]

image, segs = img_client.image_and_segmentation_cutout(ctr,
                                                       split_segmentations=True,
                                                       bbox_size=(1024, 1024),
                                                       scale_to_bounds=True,
)

ic.composite_overlay(segs, imagery=image, palette='husl')

Figure 7: Example segmentation cutout from the MICrONs dataset

Note that the following code requires setting up a CAVE token to access the server. See here for details.