HED benchmarks

Welcome to the HED benchmarks documentation! This repository holds benchmark case studies and shared benchmark tooling for Hierarchical Event Descriptors (HED) - a standardized framework for annotating events and experimental metadata in neuroscience and beyond.

The repository has two kinds of content:

• Benchmark case studies: self-contained benchmarks, one directory each under use_cases/ - performance benchmarks for HED search now, and application-oriented benchmarks (sleep staging, language scoring, epilepsy scoring) as they are developed.

• Shared tooling: the hedbench package, which will run any case study through a standardized JSON input/output format validated with JSON Schema.

• User guide
  • Installation
  • The case-study model
  • Running a benchmark
  • The standardized JSON format
  • Adding a new benchmark case study
  • Related resources
• Use cases
  • Synthetic search (working)
  • Sleep annotation source search (working)
  • Language scoring (planned)
  • Epilepsy scoring (planned)
• API
  • hedbench.runner
  • hedbench.validation

• Index