chunkedgraph

PyChunkedgraph service python interface

ChunkedGraphClientV1(server_address, auth_header, api_version, endpoints, server_key=SERVER_KEY, timestamp=None, table_name=None, verify=True, max_retries=None, pool_maxsize=None, pool_block=None, over_client=None)

Bases: ClientBase

ChunkedGraph Client for the v1 API

base_resolution property

MIP 0 resolution for voxels assumed by the ChunkedGraph

Returns:

Type	Description
list	3-long list of x/y/z voxel dimensions in nm

segmentation_info property

Complete segmentation metadata

server_version: Optional[Version] property

The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.

timestamp: Optional[datetime.datetime] property writable

The default timestamp for queries which expect a timestamp. If None, uses the current time.

do_merge(supervoxels, coords, resolution=(4, 4, 40))

Perform a merge on the chunked graph.

Parameters:

Name	Type	Description	Default
supervoxels	iterable	An N-long list of supervoxels to merge.	required
coords	array	An Nx3 array of coordinates of the supervoxels in units of resolution.	required
resolution	tuple	What to multiply coords by to get nanometers. Defaults to (4,4,40).	(4, 4, 40)

execute_split(source_points, sink_points, root_id, source_supervoxels=None, sink_supervoxels=None)

Execute a multicut split based on points or supervoxels.

Parameters:

Name	Type	Description	Default
source_points	array or list	Nx3 list or array of 3d points in nm coordinates for source points (red).	required
sink_points	array or list	Mx3 list or array of 3d points in nm coordinates for sink points (blue).	required
root_id	int	Root ID of object to do split preview.	required
source_supervoxels	(array, list or None)	If providing source supervoxels, an N-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.	None
sink_supervoxels	(array, list or None)	If providing sink supervoxels, an M-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.	None

Returns:

Name	Type	Description
operation_id	int	Unique ID of the split operation
new_root_ids	list of int	List of new root IDs resulting from the split operation.

find_path(root_id, src_pt, dst_pt, precision_mode=False)

Find a path between two locations on a root ID using the level 2 chunked graph.

Parameters:

Name	Type	Description	Default
root_id	int	Root ID to query.	required
src_pt	array	3-element array of xyz coordinates in nm for the source point.	required
dst_pt	array	3-element array of xyz coordinates in nm for the destination point.	required
precision_mode	bool	Whether to perform the search in precision mode. Defaults to False.	False

Returns:

Name	Type	Description
centroids_list	array	Array of centroids along the path.
l2_path	np.array of int	Array of level 2 chunk IDs along the path.
failed_l2_ids	np.array of int	Array of level 2 chunk IDs that failed to find a path.

get_change_log(root_id, filtered=True)

Get the change log (splits and merges) for an object.

Parameters:

Name	Type	Description	Default
root_id	int	Object root ID to look up.	required
filtered	bool	Whether to filter the change log to only include splits and merges which affect the final state of the object (filtered=True), as opposed to including edit history for objects which as some point were split from the query object root_id (filtered=False). Defaults to True.	True

Returns:

Type

Description

dict

Dictionary summarizing split and merge events in the object history, containing the following keys: "n_merges": int Number of merges "n_splits": int Number of splits "operations_ids": list of int Identifiers for each operation "past_ids": list of int Previous root ids for this object "user_info": dict of dict Dictionary keyed by user (string) to a dictionary specifying how many merges and splits that user performed on this object

get_children(node_id)

Get the children of a node in the chunked graph hierarchy.

Parameters:

Name	Type	Description	Default
node_id	int	Node ID to query.	required

Returns:

Type	Description
np.array of np.int64	IDs of child nodes.

get_contact_sites(root_id, bounds, calc_partners=False)

Get contacts for a root ID.

Parameters:

Name	Type	Description	Default
root_id	int	Root ID to query.	required
bounds		Bounds within a 3x2 numpy array of bounds [[minx,maxx],[miny,maxy],[minz,maxz]] for which to find contacts. Running this query without bounds is too slow.	required
calc_partners	bool	If True, get partner root IDs. By default, False.	False

Returns:

Type	Description
dict	Dict relating ids to contacts

get_delta_roots(timestamp_past, timestamp_future=datetime.datetime.now(datetime.timezone.utc))

Get the list of roots that have changed between timetamp_past and timestamp_future.

Parameters:

Name	Type	Description	Default
timestamp_past	datetime	Past timepoint to query	required
timestamp_future	datetime	Future timepoint to query. Defaults to datetime.datetime.now(datetime.timezone.utc).	now(utc)

Returns:

Name	Type	Description
old_roots	np.ndarray of np.int64	Roots that have expired in that interval.
new_roots	np.ndarray of np.int64	Roots that are new in that interval.

get_latest_roots(root_id, timestamp=None, timestamp_future=None)

Returns root IDs that are related to the given root_id at a given timestamp. Can be used to find the "latest" root IDs associated with an object.

Parameters:

Name	Type	Description	Default
root_id	int	Object root ID.	required
timestamp	datetime or None	Timestamp of where to query IDs from. If None, uses the timestamp property for this client, which defaults to the current time.	None
timestamp_future	datetime or None	DEPRECATED name, use timestamp instead. Timestamp to suggest IDs from (note can be in the past relative to the root). By default, None.	None

Returns:

Type	Description
ndarray	1d array with all latest successors.

get_leaves(root_id, bounds=None, stop_layer=None)

Get all supervoxels for a root ID.

Parameters:

Name	Type	Description	Default
root_id	int	Root ID to query.	required
bounds		If specified, returns supervoxels within a 3x2 numpy array of bounds [[minx,maxx],[miny,maxy],[minz,maxz]]. If None, finds all supervoxels.	None
stop_layer	int	If specified, returns chunkedgraph nodes at layer stop_layer default will be stop_layer=1 (supervoxels).	None

Returns:

Type	Description
np.array of np.int64	Array of supervoxel IDs (or node ids if stop_layer>1).

get_lineage_graph(root_id, timestamp_past=None, timestamp_future=None, as_nx_graph=False, exclude_links_to_future=False, exclude_links_to_past=False)

Returns the lineage graph for a root ID, optionally cut off in the past or the future.

Each change in the chunked graph creates a new root ID for the object after that change. This function returns a graph of all root IDs for a given object, tracing the history of the object in terms of merges and splits.

Parameters:

Name	Type	Description	Default
root_id	int	Object root ID.	required
timestamp_past	datetime or None	Cutoff for the lineage graph backwards in time. By default, None.	None
timestamp_future	datetime or None	Cutoff for the lineage graph going forwards in time. By default, uses the timestamp property for this client, which defaults to the current time.	None
as_nx_graph		If True, a NetworkX graph is returned.	False
exclude_links_to_future		If True, links from nodes before timestamp_future to after timestamp_future are removed. If False, the link(s) which has one node before timestamp and one node after timestamp is kept.	False
exclude_links_to_past		If True, links from nodes before timestamp_past to after timestamp_past are removed. If False, the link(s) which has one node before timestamp and one node after timestamp is kept.	False

Returns:

Type	Description
dict	Dictionary describing the lineage graph and operations for the root ID. Not returned if as_nx_graph is True. The dictionary contains the following keys: "directed" : bool Whether the graph is directed. "graph" : dict Dictionary of graph attributes. "links" : list of dict Each element of the list is a dictionary describing an edge in the lineage graph as "source" and "target" keys. "multigraph" : bool Whether the graph is a multigraph. "nodes" : list of dict Each element of the list is a dictionary describing a node in the lineage graph, usually with "id", "timestamp", and "operation_id" keys.
DiGraph	NetworkX directed graph of the lineage graph. Only returned if as_nx_graph is True.

get_merge_log(root_id)

Get the merge log (splits and merges) for an object.

Parameters:

Name	Type	Description	Default
root_id	int	Object root ID to look up.	required

Returns:

Type	Description
list	List of merge events in the history of the object.

get_oldest_timestamp()

Get the oldest timestamp in the database.

Returns:

Type	Description
datetime	Oldest timestamp in the database.

get_operation_details(operation_ids)

Get the details of a list of operations.

Parameters:

Name	Type	Description	Default
operation_ids	Iterable[int]	List/array of operation IDs.	required

Returns:

Type

Description

dict of str to dict

A dict of dicts of operation info, keys are operation IDs (as strings), values are a dictionary of operation info for the operation. These dictionaries contain the following keys: "added_edges"/"removed_edges": list of list of int List of edges added (if a merge) or removed (if a split) by this operation. Each edge is a list of two supervoxel IDs (source and target). "roots": list of int List of root IDs that were created by this operation. "sink_coords": list of list of int List of sink coordinates for this operation. The sink is one of the points placed by the user when specifying the operation. Each sink coordinate is a list of three integers (x, y, z), corresponding to spatial coordinates in segmentation voxel space. "source_coords": list of list of int List of source coordinates for this operation. The source is one of the points placed by the user when specifying the operation. Each source coordinate is a list of three integers (x, y, z), corresponding to spatial coordinates in segmentation voxel space. "timestamp": str Timestamp of the operation. "user": str User ID number who performed the operation (as a string).

get_original_roots(root_id, timestamp_past=None)

Returns root IDs that are the latest successors of a given root ID.

Parameters:

Name	Type	Description	Default
root_id	int	Object root ID.	required
timestamp_past	datetime or None	Cutoff for the search going backwards in time. By default, None.	None

Returns:

Type	Description
ndarray	1d array with all latest successors.

get_past_ids(root_ids, timestamp_past=None, timestamp_future=None)

For a set of root IDs, get the list of IDs at a past or future time point that could contain parts of the same object.

Parameters:

Name	Type	Description	Default
root_ids	Iterable of int	Iterable of root IDs to query.	required
timestamp_past	datetime or None	Time of a point in the past for which to look up root ids. Default is None.	None
timestamp_future	datetime or None	Time of a point in the future for which to look up root ids. Not implemented on the server currently. Default is None.	None

Returns:

Type	Description
dict	Dict with keys "future_id_map" and "past_id_map". Each is a dict whose keys are the supplied root_ids and whose values are the list of related root IDs at timestamp_past/timestamp_future.

get_root_id(supervoxel_id, timestamp=None, level2=False)

Get the root ID for a specified supervoxel.

Parameters:

Name	Type	Description	Default
supervoxel_id	int	Supervoxel id value	required
timestamp	datetime	UTC datetime to specify the state of the chunkedgraph at which to query, by default None. If None, uses the timestamp property for this client, which defaults to the current time.	None

Returns:

Type	Description
int64	Root ID containing the supervoxel.

get_root_timestamps(root_ids, latest=False, timestamp=None)

Retrieves timestamps when roots where created.

Parameters:

Name	Type	Description	Default
root_ids		Iterable of root IDs to query.	required
latest	bool	If False, returns the first timestamp that the root_id was valid for each root ID. If True, returns the newest/latest timestamp for each root ID. Note, this will return the timestamp at which the query was run when the root is currently valid. This means that you will get a different answer if you make this same query at a later time if you don't specify a timestamp parameter.	False
timestamp	datetime	Timestamp to query when using latest=True. Use this to provide consistent results for a particular timestamp. If an ID is still valid at a point in the future past this timestamp, the query will still return this timestamp as the latest moment in time. An error will occur if you provide a timestamp for which the root ID is not valid. If None, uses the timestamp property for this client, which defaults to the current time.	None

Returns:

Type	Description
np.array of datetime.datetime	Array of timestamps when root_ids were created.

get_roots(supervoxel_ids, timestamp=None, stop_layer=None)

Get the root ID for a list of supervoxels.

Parameters:

Name	Type	Description	Default
supervoxel_ids	list or np.array of int	Supervoxel IDs to look up.	required
timestamp	datetime	UTC datetime to specify the state of the chunkedgraph at which to query, by default None. If None, uses the timestamp property for this client, which defaults to the current time.	None
stop_layer	int or None	If True, looks up IDs only up to a given stop layer. Default is None.	None

Returns:

Type	Description
np.array of np.uint64	Root IDs containing each supervoxel.

get_subgraph(root_id, bounds)

Get subgraph of root id within a bounding box.

Parameters:

Name	Type	Description	Default
root_id	int	Root (or any node ID) of chunked graph to query.	required
bounds	array	3x2 bounding box (x,y,z) x (min,max) in chunked graph coordinates.	required

Returns:

Type	Description
np.array of np.int64	Node IDs in the subgraph.
np.array of np.double	Affinities of edges in the subgraph.
np.array of np.int32	Areas of nodes in the subgraph.

get_tabular_change_log(root_ids, filtered=True)

Get a detailed changelog for neurons.

Parameters:

Name	Type	Description	Default
root_ids	list of int	Object root IDs to look up.	required
filtered	bool	Whether to filter the change log to only include splits and merges which affect the final state of the object (filtered=True), as opposed to including edit history for objects which as some point were split from the query objects in root_ids (filtered=False). Defaults to True.	True

Returns:

Type

Description

dict of pd.DataFrame

The keys are the root IDs, and the values are DataFrames with the following columns and datatypes: "operation_id": int Identifier for the operation. "timestamp": int Timestamp of the operation, provided in milliseconds. To convert to datetime, use datetime.datetime.utcfromtimestamp(timestamp/1000). "user_id": int User who performed the operation. "before_root_ids: list of int Root IDs of objects that existed before the operation. "after_root_ids: list of int Root IDs of objects created by the operation. Note that this only records the root id that was kept as part of the query object, so there will only be one in this list. "is_merge": bool Whether the operation was a merge. "user_name": str Name of the user who performed the operation. "user_affiliation": str Affiliation of the user who performed the operation.

get_user_operations(user_id, timestamp_start, include_undo=True, timestamp_end=None)

Get operation details for a user ID. Currently, this is only available to admins.

Parameters:

Name	Type	Description	Default
user_id	int	User ID to query (use 0 for all users (admin only)).	required
timestamp_start	datetime	Timestamp to start filter (UTC).	required
include_undo	bool	Whether to include undos. Defaults to True.	True
timestamp_end	datetime	Timestamp to end filter (UTC). Defaults to now.	None

Returns:

Type	Description
DataFrame	DataFrame including the following columns: "operation_id": int Identifier for the operation. "timestamp": datetime.datetime Timestamp of the operation. "user_id": int User who performed the operation.

is_latest_roots(root_ids, timestamp=None)

Check whether these root IDs are still a root at this timestamp.

Parameters:

Name	Type	Description	Default
root_ids	array-like of int	Root IDs to check.	required
timestamp	datetime	Timestamp to check whether these IDs are valid root IDs in the chunked graph. If None, uses the timestamp property for this client, which defaults to the current time.	None

Returns:

Type	Description
np.array of bool	Array of whether these are valid root IDs.

is_valid_nodes(node_ids, start_timestamp=None, end_timestamp=None)

Check whether nodes are valid for given timestamp range.

Valid is defined as existing in the chunked graph. This makes no statement about these IDs being roots, supervoxel or anything in-between. It also does not take into account whether a root ID has since been edited.

Parameters:

Name	Type	Description	Default
node_ids	array-like of int	Node IDs to check.	required
start_timestamp	datetime	Timestamp to check whether these IDs were valid after this timestamp. Defaults to None (assumes now).	None
end_timestamp	datetime	Timestamp to check whether these IDs were valid before this timestamp. If None, uses the timestamp property for this client, which defaults to the current time.	None

Returns:

Type	Description
np.array of bool	Array of whether these are valid IDs.

level2_chunk_graph(root_id, bounds=None)

Get graph of level 2 chunks, the smallest agglomeration level above supervoxels.

Parameters:

Name	Type	Description	Default
root_id	int	Root id of object	required
bounds	array	3x2 bounding box (x,y,z) x (min,max) in chunked graph coordinates (use client.chunkedgraph.base_resolution to view this default resolution for your chunkedgraph client). Note that the result will include any level 2 nodes which have chunk boundaries within some part of this bounding box, meaning that the representative point for a given level 2 node could still be slightly outside of these bounds. If None, returns all level 2 chunks for the root ID.	None

Returns:

Type	Description
list of list	Edge list for level 2 chunked graph. Each element of the list is an edge, and each edge is a list of two node IDs (source and target).

preview_split(source_points, sink_points, root_id, source_supervoxels=None, sink_supervoxels=None, return_additional_ccs=False)

Get supervoxel connected components from a preview multicut split.

Parameters:

Name	Type	Description	Default
source_points	array or list	Nx3 list or array of 3d points in nm coordinates for source points (red).	required
sink_points	array or list	Mx3 list or array of 3d points in nm coordinates for sink points (blue).	required
root_id	int	Root ID of object to do split preview.	required
source_supervoxels	(array, list or None)	If providing source supervoxels, an N-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.	None
sink_supervoxels	(array, list or None)	If providing sink supervoxels, an M-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.	None
return_additional_ccs	bool	If True, returns any additional connected components beyond the ones with source and sink points. In most situations, this can be ignored. By default, False.	False

Returns:

Name	Type	Description
source_connected_component	list	Supervoxel IDs in the component with the most source points.
sink_connected_component	list	Supervoxel IDs in the component with the most sink points.
successful_split	bool	True if the split worked.
	other_connected_components (optional) : list of lists of int	List of lists of supervoxel IDs for any other resulting connected components. Only returned if return_additional_ccs is True.

raise_for_status(r, log_warning=True) staticmethod

Raises requests.HTTPError, if one occurred.

remesh_level2_chunks(chunk_ids)

Submit specific level 2 chunks to be remeshed in case of a problem.

Parameters:

Name	Type	Description	Default
chunk_ids	list	List of level 2 chunk IDs.	required

suggest_latest_roots(root_id, timestamp=None, stop_layer=None, return_all=False, return_fraction_overlap=False)

Suggest latest roots for a given root id, based on overlap of component chunk IDs. Note that edits change chunk IDs, and so this effectively measures the fraction of unchanged chunks at a given chunk layer, which sets the size scale of chunks. Higher layers are coarser.

Parameters:

Name	Type	Description	Default
root_id	int	Root ID of the potentially outdated object.	required
timestamp	datetime	Datetime at which "latest" roots are being computed, by default None. If None, uses the timestamp property for this client, which defaults to the current time. Note that this has to be a timestamp after the creation of the root_id.	None
stop_layer	int	Chunk level at which to compute overlap, by default None. No value will take the 4th from the top layer, which emphasizes speed and works well for larger objects. Lower values are slower but more fine-grained. Values under 2 (i.e. supervoxels) are not recommended except in extremely fine grained scenarios.	None
return_all	bool	If True, return all current IDs sorted from most overlap to least, by default False. If False, only the top is returned.	False
return_fraction_overlap	bool	If True, return all fractions sorted by most overlap to least, by default False. If False, only the top value is returned.	False

undo_operation(operation_id)

Undo an operation.

Parameters:

Name	Type	Description	Default
operation_id	int	Operation ID to undo.	required

Returns:

Type	Description
dict

package_split_data(root_id, source_points, sink_points, source_supervoxels, sink_supervoxels)

Create the data for preview or executed split operations