API Reference¶
This section provides detailed API documentation for the ndx-hed extension.
Core Classes¶
HedLabMetaData¶
- class HedLabMetaData(*args: Any, **kwargs: Any)[source]¶
Bases:
- __init__(hed_schema_version, definitions=None)[source]¶
- Parameters:
hed_schema_version (
strorlistortuple) – The HED schema version(s) used by this data: one version string such as ‘8.5.0’, a comma-joined string of versions in one namespace such as ‘8.5.0,score_2.2.0’, a JSON array string such as ‘[“8.5.0”, “sc:score_2.2.0”]’, or a list of version strings. Each version is ‘[namespace:]X.Y.Z’ for the standard schema or ‘[namespace:]library_X.Y.Z’ for a library schema. A list is stored as the JSON array string.definitions (
str) – A string containing one or more HED definitions.
HedValueVector¶
- class HedValueVector(*args, **kwargs)[source]¶
Bases:
VectorDataColumn storing values and a single HED annotation that applies to all values in the column. A HED string is a comma-separated, and possibly parenthesized list of HED tags selected from a valid HED vocabulary as specified by the NWBFile field HedVersion.
A HedValueVector column may have any name. HED-aware code identifies these columns by their neurodata type (
isinstance(col, HedValueVector)), not by the column name.- __init__(name, description, hed, data=[])[source]¶
- Parameters:
name (
str) – the name of this VectorDatadescription (
str) – a description for this columnhed (
str) – HED annotation template for all values in the columndata (
ndarrayorlistortupleorDatasetorSeriesorExtensionArrayorStrDatasetorHDMFDatasetorAbstractDataChunkIteratororDataIO) – a dataset where the first dimension is a concatenation of multiple vectors
- add_row(val)¶
Append a data value to this VectorData column
- Parameters:
val (None) – the value to add to this column
- all_children()¶
Get a list of all child objects and their child objects recursively.
If the object has an object_id, the object will be added to “ret” to be returned. If that object has children, they will be added to the “stack” in order to be: 1) Checked to see if has an object_id, if so then add to “ret” 2) Have children that will also be checked
- property all_objects¶
Get a LabelledDict that indexed all child objects and their children by object ID.
- append(arg)¶
- property children¶
- property container_source¶
The source of this Container
- property data¶
- data_type = 'VectorData'¶
- property description¶
a description for this column
- extend(ar, **kwargs)¶
Add all elements of the iterable arg to the end of this VectorData.
Each subclass of VectorData should have its own extend method to ensure functionality and efficiency.
- Parameters:
arg – The iterable to add to the end of this VectorData
- property fields¶
Subclasses use this class attribute to add properties to autogenerate. fields allows for lists and for dicts with the keys {‘name’, ‘child’, ‘required_name’, ‘doc’, ‘settable’}. 1. name: The name of the field property 2. child: A boolean value to set the parent/child relationship between the field property and the container. 3. required_name: The name the field property must have such that name matches required_name. 4. doc: Documentation of the field property 5. settable: If true, a setter function is created so that the field can be changed after creation.
- generate_new_id(recurse=True)¶
Changes the object ID of this Container and all of its children to a new UUID string.
- Parameters:
recurse (
bool) – whether or not to change the object ID of this container’s children
- get(key, **kwargs)¶
Retrieve elements from this VectorData
- Parameters:
key – Selection of the elements
kwargs – Ignored
- get_ancestor(data_type=None)¶
Traverse parent hierarchy and return first instance of the specified data_type
- Parameters:
data_type (
str) – the data_type to search for
- get_ancestors()¶
- classmethod get_fields_conf()¶
- get_meanings()¶
Get the MeaningsTable associated with this VectorData column, if one exists.
- Returns:
- The MeaningsTable for this column, or None if this column
is not part of a DynamicTable or has no associated MeaningsTable.
- Return type:
MeaningsTable
- get_read_io()¶
Get the io object used to read this container.
If self.read_io is None, this function will iterate through the parents and return the first io object found on a parent container
- Returns:
The
HDMFIOobject used to read this container. Returns None in case no io object is found, e.g., in case this container has not been read from file.
- property hed¶
Return the HED annotation template for this column.
- property modified¶
- property name¶
The name of this Container
- namespace = 'ndx-hed'¶
- neurodata_type = 'HedValueVector'¶
- property object_id¶
- property parent¶
The parent Container of this Container
- property read_io¶
The
HDMFIOobject used for reading the container.This property will typically be None if this Container is not a root Container (i.e., if parent is not None). Use get_read_io instead if you want to retrieve the
HDMFIOobject used for reading from the parent container.
- reset_parent()¶
Reset the parent of this Container to None and remove the Container from the children of its parent.
Use with caution. This can result in orphaned containers and broken links.
- set_data_io(data_io_class: type[DataIO], data_io_kwargs: dict, data_chunk_iterator_class: type[AbstractDataChunkIterator] | None = None, data_chunk_iterator_kwargs: dict = None) None¶
Apply DataIO object to the data held by this Data object.
- Parameters:
data_io_class (type[DataIO]) – The DataIO to apply to the data held by this Data.
data_io_kwargs (dict) – The keyword arguments to pass to the DataIO.
data_chunk_iterator_class (type[AbstractDataChunkIterator]) – The DataChunkIterator to use for the DataIO. If None, no DataChunkIterator is used.
data_chunk_iterator_kwargs (dict) – The keyword arguments to pass to the DataChunkIterator.
Notes
If data_chunk_iterator_class is not None, the data is wrapped in the DataChunkIterator before being wrapped in the DataIO. This allows for rewriting the backend configuration of hdf5 datasets.
- set_modified(modified=True)¶
- Parameters:
modified (
bool) – whether or not this Container has been modified
- property shape¶
Get the shape of the data represented by this container :return: Shape tuple :rtype: tuple of ints
- transform(func)¶
Transform data from the current underlying state.
This function can be used to permanently load data from disk, or convert to a different representation, such as a torch.Tensor
- Parameters:
func (
function) – a function to transform data
- classmethod type_hierarchy()¶
HedDurationVector¶
- class HedDurationVector(*args, **kwargs)[source]¶
Bases:
DurationVectorDataThe duration column of an EventsTable (durations in seconds) with a single HED annotation template that applies to all values in the column, as a HedValueVector does. It is a DurationVectorData, so code that reads event durations sees an ordinary duration column.
HED-aware code treats it as a HedValueVector: the same rules apply (it may not appear in a MeaningsTable, and its values must satisfy the value class of the template’s
#tag).HED_TEMPLATE_TYPESholds both.- __init__(name, description, hed, data=[], resolution=None)[source]¶
- Parameters:
name (
str) – Name of this DurationVectorDatadescription (
str) – Description of this DurationVectorDatahed (
str) – HED annotation template for all values in the columndata (
ndarrayorlistortupleorDatasetorSeriesorExtensionArrayorStrDatasetorHDMFDatasetorAbstractDataChunkIteratororDataIO) – a dataset where the first dimension is a concatenation of multiple vectorsresolution (
float) – The temporal resolution of the durations, in seconds. This is typically the sampling period (1 / sampling_rate), also known as the clock period, of the data acquisition system from which the durations were recorded or derived.
- add_row(val)¶
Append a data value to this VectorData column
- Parameters:
val (None) – the value to add to this column
- all_children()¶
Get a list of all child objects and their child objects recursively.
If the object has an object_id, the object will be added to “ret” to be returned. If that object has children, they will be added to the “stack” in order to be: 1) Checked to see if has an object_id, if so then add to “ret” 2) Have children that will also be checked
- property all_objects¶
Get a LabelledDict that indexed all child objects and their children by object ID.
- append(arg)¶
- property children¶
- property container_source¶
The source of this Container
- property data¶
- data_type = 'VectorData'¶
- property description¶
a description for this column
- extend(ar, **kwargs)¶
Add all elements of the iterable arg to the end of this VectorData.
Each subclass of VectorData should have its own extend method to ensure functionality and efficiency.
- Parameters:
arg – The iterable to add to the end of this VectorData
- property fields¶
Subclasses use this class attribute to add properties to autogenerate. fields allows for lists and for dicts with the keys {‘name’, ‘child’, ‘required_name’, ‘doc’, ‘settable’}. 1. name: The name of the field property 2. child: A boolean value to set the parent/child relationship between the field property and the container. 3. required_name: The name the field property must have such that name matches required_name. 4. doc: Documentation of the field property 5. settable: If true, a setter function is created so that the field can be changed after creation.
- generate_new_id(recurse=True)¶
Changes the object ID of this Container and all of its children to a new UUID string.
- Parameters:
recurse (
bool) – whether or not to change the object ID of this container’s children
- get(key, **kwargs)¶
Retrieve elements from this VectorData
- Parameters:
key – Selection of the elements
kwargs – Ignored
- get_ancestor(data_type=None)¶
Traverse parent hierarchy and return first instance of the specified data_type
- Parameters:
data_type (
str) – the data_type to search for
- get_ancestors()¶
- classmethod get_fields_conf()¶
- get_meanings()¶
Get the MeaningsTable associated with this VectorData column, if one exists.
- Returns:
- The MeaningsTable for this column, or None if this column
is not part of a DynamicTable or has no associated MeaningsTable.
- Return type:
MeaningsTable
- get_read_io()¶
Get the io object used to read this container.
If self.read_io is None, this function will iterate through the parents and return the first io object found on a parent container
- Returns:
The
HDMFIOobject used to read this container. Returns None in case no io object is found, e.g., in case this container has not been read from file.
- property hed¶
Return the HED annotation template for this column.
- property modified¶
- property name¶
The name of this Container
- namespace = 'ndx-hed'¶
- neurodata_type = 'HedDurationVector'¶
- property object_id¶
- property parent¶
The parent Container of this Container
- property read_io¶
The
HDMFIOobject used for reading the container.This property will typically be None if this Container is not a root Container (i.e., if parent is not None). Use get_read_io instead if you want to retrieve the
HDMFIOobject used for reading from the parent container.
- reset_parent()¶
Reset the parent of this Container to None and remove the Container from the children of its parent.
Use with caution. This can result in orphaned containers and broken links.
- set_data_io(data_io_class: type[DataIO], data_io_kwargs: dict, data_chunk_iterator_class: type[AbstractDataChunkIterator] | None = None, data_chunk_iterator_kwargs: dict = None) None¶
Apply DataIO object to the data held by this Data object.
- Parameters:
data_io_class (type[DataIO]) – The DataIO to apply to the data held by this Data.
data_io_kwargs (dict) – The keyword arguments to pass to the DataIO.
data_chunk_iterator_class (type[AbstractDataChunkIterator]) – The DataChunkIterator to use for the DataIO. If None, no DataChunkIterator is used.
data_chunk_iterator_kwargs (dict) – The keyword arguments to pass to the DataChunkIterator.
Notes
If data_chunk_iterator_class is not None, the data is wrapped in the DataChunkIterator before being wrapped in the DataIO. This allows for rewriting the backend configuration of hdf5 datasets.
- set_modified(modified=True)¶
- Parameters:
modified (
bool) – whether or not this Container has been modified
- property shape¶
Get the shape of the data represented by this container :return: Shape tuple :rtype: tuple of ints
- transform(func)¶
Transform data from the current underlying state.
This function can be used to permanently load data from disk, or convert to a different representation, such as a torch.Tensor
- Parameters:
func (
function) – a function to transform data
- classmethod type_hierarchy()¶
- property unit¶
The unit of measurement for the durations, fixed to ‘seconds’.
Validation Utilities¶
HedNWBValidator¶
- class HedNWBValidator(*args: Any, **kwargs: Any)[source]¶
Bases:
- __init__(hed_metadata: HedLabMetaData)[source]¶
Initialize the HedNWBValidator with HED metadata.
- Parameters:
hed_metadata (HedLabMetaData) – The HED lab metadata holding the schema and the definitions. A constructed HedLabMetaData has already loaded and checked both, so nothing is validated here.
- Raises:
ValueError – If hed_metadata is not an instance of HedLabMetaData
BIDS Conversion Utilities¶
These utilities support bidirectional conversion between BIDS events and NWB EventsTable format.
- extract_definitions(sidecar_data: dict, hed_schema: HedSchema | HedSchemaGroup) tuple[source]¶
Extracts definitions from a HED sidecar JSON data using the provided HED schema.
- Parameters:
sidecar_data (dict) – A dictionary representing the loaded HED Sidecar JSON data.
hed_schema (HedSchema or HedSchemaGroup) – The HED schema object for validation and processing.
- Returns:
- A tuple containing:
DefinitionDict: A dictionary of definitions extracted from the sidecar.
list: A list of validation issues found during extraction.
- Return type:
- extract_meanings(sidecar_data: dict) dict[source]¶
Converts a HED sidecar JSON data to a meanings dictionary.
- Parameters:
sidecar_data (dict) – A dictionary representing the loaded HED Sidecar JSON data.
- Returns:
- A meanings dictionary with keys “categorical” and “value”
”categorical”: dict mapping column names to their raw sidecar column-info dict (Levels and/or HED). A MeaningsTable cannot be built here because, as of PyNWB 4.0.0, a MeaningsTable requires the target VectorData column object; the MeaningsTable is created later in get_events_table once the column exists.
”value”: dict mapping column names to HED strings
- Return type:
- get_bids_dataframe(table: DynamicTable) DataFrame[source]¶
Converts the data of a DynamicTable to a BIDS-style dataframe, the TSV half of get_bids_tabular().
This reads every column of the table. A
TimestampVectorDatacolumn is renamed toonsetso that downstream BIDS-HED validation treats the table as a timeline (temporal) file. Missing cells (NaN, None, the empty string) become the textn/a, as in a BIDS TSV file, so every column of the returned dataframe has dtype object.- Parameters:
table (DynamicTable) – The table to convert.
- Returns:
The table data with BIDS column names (onset, duration, etc.)
- Return type:
pd.DataFrame
- Raises:
ValueError – If a BIDS column name would occur twice, that is the table has a column named
onsetas well as a TimestampVectorData namedtimestamp(TABULAR_INVALID, HED specification 3.2.10).
- get_bids_tabular(table: DynamicTable, hed_metadata: HedLabMetaData = None) tuple[source]¶
Converts a DynamicTable to a BIDS-style tabular representation (DataFrame and JSON sidecar).
Works for any DynamicTable (an EventsTable or a plain DynamicTable). It is not meant for a MeaningsTable, whose HED is consumed while assembling the table whose column it annotates. The dataframe is get_bids_dataframe(table) and the sidecar is get_json_hed_dict(table, hed_metadata); call those directly to get only one half (the sidecar half does not read the table’s data).
- Parameters:
table (DynamicTable) – The table to convert.
hed_metadata (HedLabMetaData, optional) – The HED lab metadata supplying the definitions, passed through to get_json_hed_dict(). If None (the default), the sidecar carries no definitions and any
Def/references in the table cannot be resolved from it alone.
- Returns:
- A tuple containing:
pd.DataFrame: The table data with BIDS column names (onset, duration, etc.)
dict: The JSON sidecar data as returned by get_json_hed_dict().
- Return type:
- get_categorical_meanings(target_column: VectorData, column_info: dict) MeaningsTable[source]¶
Converts a categorical column info dict to a MeaningsTable annotating a target column.
As of PyNWB 4.0.0, a MeaningsTable is bound to the VectorData column it annotates and its name is derived automatically as “{target_column.name}_meanings”.
- Parameters:
target_column (VectorData) – The column object this MeaningsTable annotates. Must already be a column of the DynamicTable that the MeaningsTable will be added to.
column_info (dict) – The column info dictionary from the sidecar (Levels and/or HED).
- Returns:
The constructed MeaningsTable object (name “{target_column.name}_meanings”).
- Return type:
MeaningsTable
- get_events_table(name: str, description: str, df: DataFrame, meanings: dict) EventsTable[source]¶
Converts a pandas DataFrame and meanings dictionary to an EventsTable.
- Parameters:
name (str) – The name of the EventsTable.
description (str) – The description of the EventsTable.
df (pd.DataFrame) – The DataFrame containing event data.
meanings (dict) – The meanings dictionary with keys “categorical” and “value”. The “categorical” values are raw sidecar column-info dicts (see extract_meanings).
- Returns:
The constructed EventsTable object. Categorical columns are stored as plain VectorData columns, each annotated by a MeaningsTable attached to the table. The onset column becomes the TimestampVectorData “timestamp”; the duration column becomes a HedDurationVector when meanings gives it a value template, a DurationVectorData otherwise.
- Return type:
EventsTable
- Raises:
ValueError – If the DataFrame repeats a column name, if meanings gives the onset column any HED or Levels, or if meanings makes the duration column categorical.
- get_json_hed_dict(table: DynamicTable, hed_metadata: HedLabMetaData = None) dict[source]¶
Builds the BIDS-style JSON sidecar dictionary of a DynamicTable directly from its columns.
This is the metadata half of get_bids_tabular(). It reads the column objects and the MeaningsTable annotating each categorical column, so it needs neither a dataframe nor pandas.
- Each entry maps a column name to a BIDS sidecar column-info dict, which may contain:
“Description”: the column’s description, if it has one.
“HED”: for a HedValueVector or HedDurationVector, its value template (a string containing a single “#”); for a categorical column, a dict mapping each value to its HED string.
“Levels”: for a categorical column, a dict mapping each value to its meaning.
- Columns contribute as follows:
HedTags (the “HED” column): omitted entirely. Its cells are HED strings, so the column is self-describing, and “HED” is a reserved sidecar key that must not name an entry.
HedValueVector and HedDurationVector: the HED template, unless the template is empty or “n/a”.
A plain TimestampVectorData/DurationVectorData: description only. The entry keeps the column’s own name, so an EventsTable “timestamp” column is keyed “timestamp” even though get_bids_tabular() renames it to “onset” in the dataframe.
Any other column: the Levels and HED of the MeaningsTable annotating it, if any.
Columns with no metadata are omitted, so the result is empty for a table with no descriptions, no MeaningsTables, and no HED template columns.
In NWB, HED definitions live in a HedLabMetaData object, which is the only place extra definitions may come from. When one is given, its definitions are exported under the “definitions” key as
{"HED": {"defList": <definitions>}}– a sidecar entry that names no column, which is how BIDS carries definitions. The resulting sidecar is then self-contained: theDef/references in the table can be resolved without supplying the definitions separately. If the HedLabMetaData holds no definitions, no entry is added.- Parameters:
table (DynamicTable) – The table to extract the sidecar metadata from. As with get_bids_tabular(), this is not meant for a MeaningsTable, whose HED is consumed while assembling the table whose column it annotates.
hed_metadata (HedLabMetaData, optional) – The HED lab metadata supplying the definitions. If None (the default), the sidecar carries no definitions.
- Returns:
The JSON sidecar data with column metadata, levels, HED annotations, and definitions.
- Return type:
- Raises:
ValueError – If hed_metadata is given but is not a HedLabMetaData instance.
ValueError – If the definitions would overwrite the entry of a column named “definitions”.
- get_levels_and_hed(meanings_table: MeaningsTable) tuple[source]¶
Extracts the BIDS “Levels” and “HED” dictionaries from a MeaningsTable without using pandas.
- Parameters:
meanings_table (MeaningsTable) – The MeaningsTable annotating a categorical column. Its “value” and “meaning” columns supply the levels; an optional HedTags column named “HED” supplies the per-value HED annotations.
- Returns:
- A tuple containing:
dict: Levels – maps each value to its meaning (empty string if there is no meaning). The keys are the values as text (
_value_text), as a BIDS sidecar holds them.dict: HED – maps each value, as text, to its HED string, omitting missing (None, NaN, or empty) annotations. Empty if the MeaningsTable has no HED column. An explicit
n/ais kept (the sidecar annotates the value as having no HED); a value with an empty cell, whichget_categorical_meaningswrites for a value onlyLevelslists, is not a key under HED, so the rule for keys under HED (specification 3.2.9.2) does not apply to it.
- Return type:
- read_bids_tsv(path) DataFrame[source]¶
Reads a BIDS TSV file into a DataFrame the way hedtools reads it for validation.
Every cell is kept as text exactly as written: a double quote is an ordinary character, a line is split on every tab, and only an empty cell and
n/aare missing (NaN);NA,null, ornanare values. Column names are the header cells exactly as written too: the header line is read as a data row, so pandas does not rename a repeated name (HED,HED.1) or a blank one (Unnamed: 1), and a padded, repeated, or blank name reachesget_events_tableand validation as it is in the file (HED specification 3.2.10).pd.read_csv(path, sep="\t")with its defaults removes quotes, readsNAas missing, and renames repeated and blank names, so a file converted with it is not the file hedtools validates.- Parameters:
path (str or path-like or file-like) – The TSV file, UTF-8 (a byte order mark is removed).
- Returns:
The file’s columns, labeled by the header cells, every value a string or NaN.
- Return type:
pd.DataFrame
- Raises:
pandas.errors.EmptyDataError – If the file has no header line.