crantpy.queries.nested_connectivity_matrices module

crantpy.queries.nested_connectivity_matrices module#

Functionality for creating and visualizing nested connectivity matrices.

The NestedMatrix class enables the construction of hierarchical connectivity matrices that organize neurons by cell type, allowing for both neuron-level and type-level connectivity analysis. It supports visualization with customizable boundaries, filtering, and relative weight calculations. ROI-resolved workflows are available through NestedMatrix.from_synapses_by_neuropil(), which returns one nested matrix per neuropil ROI.

DirectedNestedMatrix provides the rectangular counterpart for workflows that need independent source and target axes, such as analyzing ER_input -> ER without also including ER -> ER.

Examples

>>> import pandas as pd
>>> from crantpy.queries.nested_connectivity_matrices import NestedMatrix
>>>
>>> # Create from synapse dataframe
>>> synapses_df = pd.DataFrame({
...     'pre_pt_root_id': [1, 1, 2, 2],
...     'post_pt_root_id': [3, 4, 3, 4],
...     'Weight': [10, 20, 15, 25]
... })
>>> annotations_df = pd.DataFrame({
...     'root_id': [1, 2, 3, 4],
...     'cell_type': ['KC', 'KC', 'MB', 'MB']
... })
>>> matrix = NestedMatrix.from_synapses(
...     synapses_df,
...     annotations_df,
...     weight_mode="column",
...     weight_column="Weight",
... )
>>>
>>> # Plot the matrix
>>> matrix.plot(level="neuron")
>>>
>>> relative = NestedMatrix.from_synapses(
...     synapses_df,
...     annotations_df,
...     weight_mode="relative_outgoing",
... )
>>> # Build ROI-specific matrices using neuropil meshes
>>> matrices = NestedMatrix.from_synapses_by_neuropil(
...     synapses_df,
...     annotations_df,
...     neuropil_names=["protocerebral_bridge", "ellipsoid_body"],
...     coordinates="nm",
... )
class crantpy.queries.nested_connectivity_matrices.DirectedNestedMatrix(matrix, source_neurons, target_neurons, source_type_boundaries=None, target_type_boundaries=None, source_neuron_to_type=None, target_neuron_to_type=None)[source]#

Bases: object

A rectangular nested connectivity matrix with independent axes.

Rows are presynaptic/source neurons and columns are postsynaptic/target neurons. Unlike NestedMatrix, the matrix may be rectangular and the source and target cell type sets may differ. Type and explicit neuron selectors on the same axis are unioned. If an axis selector is None, that axis includes all available neurons for that axis independently of the other axis selector.

Parameters:

  • matrix (pd.DataFrame)

  • source_neurons (Iterable[Any])

  • target_neurons (Iterable[Any])

  • source_type_boundaries (Mapping[Any, tuple[int, int]] | None)

  • target_type_boundaries (Mapping[Any, tuple[int, int]] | None)

  • source_neuron_to_type (Mapping[Any, Any] | None)

  • target_neuron_to_type (Mapping[Any, Any] | None)

classmethod from_connectivity(connections_df, neuron_annotations, source_types=None, target_types=None, source_neurons=None, target_neurons=None, cell_type_column='cell_type', neuron_id_column='root_id', type_order=None, source_type_order=None, target_type_order=None, annotation_scope='annotated_only')[source]#

Create a rectangular directed nested matrix from connectivity data.

source_* selectors apply to rows and target_* selectors apply to columns. Type and explicit neuron selectors on the same axis are unioned. None selects all available neurons on that axis.

Parameters:
Return type:

DirectedNestedMatrix

classmethod from_synapses(synapses_df, neuron_annotations, source_types=None, target_types=None, source_neurons=None, target_neurons=None, pre_col='pre_pt_root_id', post_col='post_pt_root_id', weight_mode='relative_outgoing', weight_column=None, normalization_scope='selected', cell_type_column='cell_type', neuron_id_column='root_id', type_order=None, source_type_order=None, target_type_order=None, annotation_scope='annotated_only')[source]#

Create a rectangular directed nested matrix from synapse rows.

The default normalization_scope="selected" normalizes relative weights after source/target filtering. Use "all" to normalize by all partners for the selected source or target neurons before the matrix is restricted to the selected axes.

Parameters:
Return type:

DirectedNestedMatrix

classmethod from_synapses_by_neuropil(synapses_df, neuron_annotations, neuropil_names=None, coordinates='nm', position_column='ctr_pt_position', source_types=None, target_types=None, source_neurons=None, target_neurons=None, pre_col='pre_pt_root_id', post_col='post_pt_root_id', weight_mode='relative_outgoing', weight_column=None, normalization_scope='selected', cell_type_column='cell_type', neuron_id_column='root_id', type_order=None, source_type_order=None, target_type_order=None, annotation_scope='annotated_only', include_other=True, voxel_offset=None)[source]#

Create directed nested matrices per neuropil using mesh containment.

Parameters:
Return type:

NeuropilCollection

get_relative_weights(by_type=False)[source]#

Calculate row-normalized source-to-target weights.

Parameters:

by_type (bool)

Return type:

pandas.DataFrame

property matrix: pandas.DataFrame#

Read-only source-to-target neuron connectivity matrix.

property mean_type_matrix: pandas.DataFrame#

Mean source-to-target connectivity by source and target cell type.

plot(output_path=None, level='neuron', figsize=(16, 14), show_neuron_labels=False, vmin_percentile=0.0, vmax_percentile=100.0, min_neurons_for_plot=1, linewidth_scale=1.0)[source]#

Plot the rectangular directed connectivity matrix as a heatmap.

Parameters:

  • output_path (str | None)

  • level (Literal['neuron', 'type_mean', 'type_sum'])

  • figsize (tuple[int, int])

  • show_neuron_labels (bool)

  • vmin_percentile (float)

  • vmax_percentile (float)

  • min_neurons_for_plot (int)

  • linewidth_scale (float)

Return type:

tuple[Figure, Axes]

property source_neuron_to_type: Mapping[str, Any]#

Read-only mapping from source neuron ID to cell type.

property source_neurons: tuple[str, ...]#

Read-only ordered source neuron IDs.

property source_type_boundaries: Mapping[str, tuple[int, int]]#

Read-only mapping from source cell type to its row slice.

property sum_type_matrix: pandas.DataFrame#

Sum source-to-target connectivity by source and target cell type.

property target_neuron_to_type: Mapping[str, Any]#

Read-only mapping from target neuron ID to cell type.

property target_neurons: tuple[str, ...]#

Read-only ordered target neuron IDs.

property target_type_boundaries: Mapping[str, tuple[int, int]]#

Read-only mapping from target cell type to its column slice.

class crantpy.queries.nested_connectivity_matrices.NestedMatrix(matrix, type_boundaries, ordered_neurons, neuron_to_type)[source]#

Bases: object

A nested connectivity matrix organized by cell type.

This class provides functionality to create, manipulate, and visualize connectivity matrices where neurons are grouped by their cell type annotations. The matrix maintains both neuron-level and type-level connectivity information, allowing for hierarchical visualization of connectivity patterns.

Note

Instances are immutable after construction. Public attributes expose read-only views; use .copy(), dict(...), or list(...) when mutable data is needed. Derived type matrices are cached against the immutable private state.

Parameters:

  • matrix (pd.DataFrame)

  • type_boundaries (Mapping[Any, tuple[int, int]])

  • ordered_neurons (Iterable[Any])

  • neuron_to_type (Mapping[Any, Any])

matrix#

Full neuron-to-neuron connectivity matrix with neurons ordered by type.

Type:

pd.DataFrame

type_boundaries#

Dictionary mapping cell type names to (start, end) index tuples indicating where each type appears in the matrix.

Type:

Mapping[str, tuple[int, int]]

ordered_neurons#

Tuple of neuron IDs in the order they appear in the matrix.

Type:

tuple[str, …]

neuron_to_type#

Dictionary mapping neuron IDs to their cell type annotations.

Type:

Mapping[str, Any]

Examples

>>> matrix = NestedMatrix.from_connectivity(
...     connections_df=adjacency_df,
...     neuron_annotations=annotations_df
... )
>>> type_matrix = matrix.sum_type_matrix
>>> matrix.plot(level="type_mean")
classmethod from_connectivity(connections_df, neuron_annotations, cell_type_column='cell_type', neuron_id_column='root_id', type_order=None, annotation_scope='annotated_only')[source]#

Create a NestedMatrix from a connectivity dataframe that you can get with cp.get_connectivity().

Constructs a nested connectivity matrix from an adjacency or edge-list dataframe, organizing neurons by their cell type annotations. Type blocks follow type_order when provided, otherwise a generic label-aware sort is used. Within each type block, neurons preserve the resolved annotation row order, except "EPG/PEG" rows which use the EB column order encoded in cell_instance or cell_subtype when available.

Parameters:

  • connections_df (pd.DataFrame) –

    Connectivity data in one of the following formats: - Adjacency matrix (index and columns are neuron IDs) - Edge list with columns [‘type.from’, ‘type.to’, ‘weight’] - Edge list with columns [‘pre’, ‘post’, ‘weight’] - Edge list with columns [‘source’, ‘target’, ‘weight’] or [‘source’, ‘target’, ‘n_syn’]

    where each row is already aggregated to a unique source-target pair, such as the output of cp.get_connectivity()

  • neuron_annotations (pd.DataFrame) – DataFrame containing neuron annotations with at least the neuron ID and cell type columns.

  • cell_type_column (str, default "cell_type") – Name of the column in neuron_annotations containing cell type labels.

  • neuron_id_column (str, default "root_id") – Name of the column in neuron_annotations containing neuron IDs.

  • type_order (list or tuple, optional) – Preferred order for cell types. Values are normalized like annotation labels. Types will be sorted alphabetically if not provided, with numeric suffixes handled intelligently.

  • annotation_scope ({"annotated_only", "all"}, default "annotated_only") – Controls which neurons are retained. "annotated_only" keeps only neurons present in neuron_annotations. "all" keeps all neurons from the connectivity input, appending neurons missing annotations after the typed blocks.

Returns:

A new NestedMatrix instance with neurons organized by type.

Return type:

NestedMatrix

Examples

>>> adjacency = pd.DataFrame({
...     1: [0, 10, 0], 2: [5, 0, 15], 3: [0, 20, 0]
... }, index=[1, 2, 3])
>>> annotations = pd.DataFrame({
...     'root_id': [1, 2, 3],
...     'cell_type': ['ER', 'ER', 'Pbt']
... })
>>> matrix = NestedMatrix.from_connectivity(adjacency, annotations)
classmethod from_synapses(synapses_df, neuron_annotations, pre_col='pre_pt_root_id', post_col='post_pt_root_id', weight_mode='relative_outgoing', weight_column=None, cell_type_column='cell_type', neuron_id_column='root_id', type_order=None, annotation_scope='annotated_only')[source]#

Create a NestedMatrix from a synapse dataframe.

Aggregates synapse-level data into a connectivity matrix and organizes neurons by cell type. This constructor returns a single matrix over the supplied synapses. For ROI-specific outputs, pre-filter synapses_df to the ROI of interest or use from_synapses_by_neuropil() to build one matrix per neuropil ROI. Ordering semantics match from_connectivity(): type_order controls type blocks, and neurons within each type preserve resolved annotation row order except for "EPG/PEG" rows, which use EB columns from cell_instance or cell_subtype when available.

Parameters:

  • synapses_df (pd.DataFrame) – DataFrame containing synapse data with pre- and post-synaptic neuron IDs. Must contain columns for presynaptic and postsynaptic IDs.

  • neuron_annotations (pd.DataFrame) – DataFrame containing neuron annotations with at least the neuron ID and cell type columns.

  • pre_col (str, default "pre_pt_root_id") – Name of the column in synapses_df containing presynaptic neuron IDs.

  • post_col (str, default "post_pt_root_id") – Name of the column in synapses_df containing postsynaptic neuron IDs.

  • weight_mode ({"relative_outgoing", "relative_incoming", "count", "column"}, default "relative_outgoing") – How to derive edge weights from synapse rows. "relative_outgoing" counts synapses per pair and normalizes each presynaptic neuron’s outgoing row to sum to 1. "relative_incoming" counts synapses per pair and normalizes each postsynaptic neuron’s incoming column to sum to 1. "count" uses raw synapse counts per pair. "column" sums the values from weight_column per pair.

  • weight_column (str | None, optional) – Column to sum when weight_mode="column". Must be provided for column-based weighting and omitted otherwise.

  • cell_type_column (str, default "cell_type") – Name of the column in neuron_annotations containing cell type labels.

  • neuron_id_column (str, default "root_id") – Name of the column in neuron_annotations containing neuron IDs.

  • type_order (list or tuple, optional) – Preferred order for cell types. Values are normalized like annotation labels. Types will be sorted alphabetically if not provided, with numeric suffixes handled intelligently.

  • annotation_scope ({"annotated_only", "all"}, default "annotated_only") – Controls which synapse rows contribute to the matrix. "annotated_only" keeps only rows whose pre/post neurons are both present in neuron_annotations. "all" keeps all rows and appends missing/untyped neurons after the typed blocks.

Returns:

A new NestedMatrix instance with neurons organized by type across the supplied synapses.

Return type:

NestedMatrix

Examples

>>> synapses = pd.DataFrame({
...     'pre_pt_root_id': [1, 1, 2],
...     'post_pt_root_id': [3, 4, 3],
...     'Weight': [10, 20, 15]
... })
>>> annotations = pd.DataFrame({
...     'root_id': [1, 2, 3, 4],
...     'cell_type': ['KC', 'KC', 'MB', 'MB']
... })
>>> matrix = NestedMatrix.from_synapses(
...     synapses,
...     annotations,
...     weight_mode="column",
...     weight_column="Weight",
... )
>>> relative = NestedMatrix.from_synapses(
...     synapses,
...     annotations,
...     weight_mode="relative_outgoing",
... )
>>> # For neuropil ROI-specific matrices, use:
>>> # NestedMatrix.from_synapses_by_neuropil(...)
classmethod from_synapses_by_neuropil(synapses_df, neuron_annotations, neuropil_names=None, coordinates='nm', position_column='ctr_pt_position', pre_col='pre_pt_root_id', post_col='post_pt_root_id', weight_mode='relative_outgoing', weight_column=None, cell_type_column='cell_type', neuron_id_column='root_id', type_order=None, annotation_scope='annotated_only', include_other=True, voxel_offset=None)[source]#

Create NestedMatrix instances per neuropil using mesh containment.

Assigns each synapse to one or more neuropil ROIs by testing its spatial coordinates against neuropil meshes, then builds a separate NestedMatrix for each neuropil ROI that contains synapses. Weighting, annotation scope, and ordering semantics match from_synapses() within each ROI-specific subset.

Parameters:

  • synapses_df (pd.DataFrame) – DataFrame containing synapse data. Must include the position column and pre/post neuron ID columns.

  • neuron_annotations (pd.DataFrame) – DataFrame with neuron ID and cell type columns.

  • neuropil_names (list[str] | None, optional) – Neuropil mesh names to test against (values from NEUROPIL_MESH_DICT). If None, all available neuropils are used.

  • coordinates (str, default "nm") – Coordinate system of the position column. "nm" for nanometers (meshes are in nm space), "pixels" to auto-convert using scale factors.

  • position_column (str, default "ctr_pt_position") – Column containing [x, y, z] coordinates for containment testing.

  • pre_col (str, default "pre_pt_root_id") – Column with presynaptic neuron IDs.

  • post_col (str, default "post_pt_root_id") – Column with postsynaptic neuron IDs.

  • weight_mode ({"relative_outgoing", "relative_incoming", "count", "column"}, default "relative_outgoing") – How to derive edge weights from synapse rows within each neuropil subset. Semantics match from_synapses().

  • weight_column (str | None, optional) – Column to sum when weight_mode="column".

  • cell_type_column (str, default "cell_type") – Column in neuron_annotations with cell type labels.

  • neuron_id_column (str, default "root_id") – Column in neuron_annotations with neuron IDs.

  • type_order (list or tuple, optional) – Preferred order for cell types in each matrix. Values are normalized like annotation labels.

  • annotation_scope ({"annotated_only", "all"}, default "annotated_only") – Controls which synapse rows contribute to the ROI-specific matrices. "annotated_only" keeps only rows whose pre/post neurons are both present in neuron_annotations before ROI assignment. "all" keeps all rows and appends missing/untyped neurons after the typed blocks within each ROI matrix.

  • include_other (bool, default True) – If True, synapses not inside any neuropil mesh are collected under the "other" key.

  • voxel_offset (tuple[float, float, float] | None, optional) – Offset to add to pixel coordinates before converting to nm, to align synapse positions with the neuropil mesh coordinate space. Only applied when coordinates="pixels".

Returns:

Dict-like collection mapping neuropil ROI name (and optionally "other") to a NestedMatrix built from the synapses in that region. Supports collection.plot(name, **kwargs) shortcut and attribute access (e.g., collection.ellipsoid_body).

Return type:

NeuropilCollection

Raises:

ValueError – If an invalid neuropil name or coordinates value is provided.

Examples

>>> matrices = NestedMatrix.from_synapses_by_neuropil(
...     synapses_df=synapses,
...     neuron_annotations=annotations,
...     neuropil_names=["antennal_lobe_left", "mushroom_body_pedunculus_and_lobes_left"],
...     coordinates="nm",
... )
>>> for name, mat in matrices.items():
...     print(name, mat.sum_type_matrix.shape)
get_relative_weights(by_type=False)[source]#

Calculate relative connection weights normalized by row sums.

Computes the proportion of each neuron’s (or type’s) total output that goes to each target. Values range from 0 to 1, where 1 indicates all output goes to that target.

Parameters:

by_type (bool, default False) – If True, calculate relative weights at the type level. If False, calculate at the neuron level.

Returns:

Matrix of relative weights where each row sums to 1.0 (except for rows with zero total output).

Return type:

pd.DataFrame

Examples

>>> matrix = NestedMatrix.from_connectivity(...)
>>> relative = matrix.get_relative_weights(by_type=True)
>>> # Shows what proportion of each type's output goes to each target type
property matrix: pandas.DataFrame#

Read-only neuron-to-neuron connectivity matrix.

property mean_type_matrix: pandas.DataFrame#

Calculate the mean cell type-level connectivity matrix.

Aggregates the neuron-level matrix into a type-level matrix by taking the arithmetic mean of all neuron-to-neuron weights within each type pair block. This includes zero-valued entries in the block, so larger cell types do not automatically dominate the visualization by virtue of having more neurons. The returned DataFrame is a read-only view; call .copy() before editing.

Returns:

A square matrix with cell types as both index and columns, where each value represents the mean connectivity weight across all neuron pairs in the corresponding type block.

Return type:

pd.DataFrame

property neuron_to_type: Mapping[str, Any]#

Read-only mapping from neuron ID to cell type.

property ordered_neurons: tuple[str, ...]#

Read-only neuron order used by both matrix axes.

plot(output_path=None, level='neuron', figsize=(16, 14), show_neuron_labels=False, vmin_percentile=0.0, vmax_percentile=100.0, min_neurons_for_plot=1, linewidth_scale=1.0)[source]#

Plot the connectivity matrix as a heatmap.

Creates a visualization of the connectivity matrix with optional type boundaries, customizable color scaling, and filtering options.

Parameters:

  • output_path (str | None, optional) – If provided, save the plot to this file path. Directory will be created if it doesn’t exist.

  • level ({"neuron", "type_mean", "type_sum"}, default "neuron") – Which matrix to visualize. "neuron" plots the neuron-level matrix with nested type boundaries. "type_mean" plots mean_type_matrix. "type_sum" plots sum_type_matrix.

  • figsize (tuple[int, int], default (16, 14)) – Figure size in inches (width, height).

  • show_neuron_labels (bool, default False) – If True, show individual neuron IDs on axes. Only applies when level="neuron". If False, shows type labels at boundaries.

  • vmin_percentile (float, default 0.0) – Percentile for minimum color scale value (0-100). Computed over nonzero values only, so zero-weight entries always map to the bottom of the colormap.

  • vmax_percentile (float, default 100.0) – Percentile for maximum color scale value (0-100). Computed over nonzero values only. Values below 100 clip the strongest connections to the max color, making mid-range weights more visible.

  • min_neurons_for_plot (int, default 1) – Minimum number of neurons required per type to include in plot. Types with fewer neurons will be filtered out.

  • linewidth_scale (float, default 1.0) – Multiplier applied to the default boundary line widths. Use values above 1.0 for thicker boundaries and below 1.0 for thinner ones.

Returns:

Matplotlib figure and axes objects for further customization.

Return type:

tuple[plt.Figure, plt.Axes]

Examples

>>> matrix = NestedMatrix.from_connectivity(...)
>>> fig, ax = matrix.plot(level="type_mean", output_path='connectivity.png')
>>> plt.show()
property sum_type_matrix: pandas.DataFrame#

Calculate the cell type-level connectivity matrix (sum).

Aggregates the neuron-level matrix into a type-level matrix by summing all connections between neurons of each type pair. The returned DataFrame is a read-only view; call .copy() before editing.

Returns:

A square matrix with cell types as both index and columns, where each value represents the total connectivity weight between those two cell types.

Return type:

pd.DataFrame

Examples

>>> matrix = NestedMatrix.from_connectivity(...)
>>> type_connectivity = matrix.sum_type_matrix
>>> print(type_connectivity.loc['KC', 'MB'])  # KC -> MB connections
property type_boundaries: Mapping[str, tuple[int, int]]#

Read-only mapping from cell type to its matrix slice.

class crantpy.queries.nested_connectivity_matrices.NeuropilCollection[source]#

Bases: dict

Dict subclass mapping neuropil names to nested matrix instances.

Provides convenience methods for accessing and plotting individual neuropil/ROI matrices with cleaner syntax.

Examples

>>> matrices = NestedMatrix.from_synapses_by_neuropil(...)
>>> matrices.plot("protocerebral_bridge", level="neuron")
>>> matrices.plot(All)
>>> matrices.plot(All.minus("fan_shaped_body"))
>>> matrices.protocerebral_bridge.sum_type_matrix
plot(name, **kwargs)[source]#

Plot connectivity matrix/matrices.

Parameters:

  • name (str or All selector) – A single neuropil name, or All / All.minus(...) to plot multiple neuropils at once.

  • **kwargs – Forwarded to the contained matrix object’s plot() method.

Returns:

  • tuple[Figure, Axes] – When name is a single neuropil string.

  • dict[str, tuple[Figure, Axes]] – When name is an All selector.

Return type:

tuple[Figure, Axes] | dict[str, tuple[Figure, Axes]]

Contents