Skip to content

perf: reuse TDC peeler distance work and bound temporal neighbor searches - #4874

Open
JESUSROYETH wants to merge 1 commit into
SpikeInterface:mainfrom
JESUSROYETH:codex/perf-tdc-query-reuse-20261015
Open

JESUSROYETH wants to merge 1 commit into
SpikeInterface:mainfrom
JESUSROYETH:codex/perf-tdc-query-reuse-20261015

Conversation

@JESUSROYETH

Copy link
Copy Markdown
Contributor

TDC template matching repeats waveform-only squared terms across sparse templates, and scans the full spike array on every temporal-neighbor query. So this change reduces a complete sorting-to-curation pipeline by 10.91% median paired time on the recording below.

The peeler now computes the zero-template channel energy once per query, and only walks the union of candidate channels. Independent accumulators take care of the rest of the reduction, and an outward rounding check falls back to the original kernel whenever the float32 answer is uncertain. An operation-count guard also keeps low-reuse cases on the original kernel. Neighbor queries search sorted sample-time windows and then restore the original index order for amplitude fitting, so the output order doesn't change. Defaults and public signatures stay the same.

Five separate n2-standard-8 CPU VMs ran the first 300 s of public IBL CSH_ZAD_026/2020-09-04/001 probe00, 384 channels at 30 kHz. Each pair includes local compressed-input reading, default tridesclous2, its preprocessing/cache, a filtered/referenced saved analyzer, extensions, five quality metrics and saved example curation. Four workers, 1 s chunks, seed 42, NumPy 2.3.5/Numba 0.68; fresh processes include JIT compilation. Downloads and installation are excluded.

Trial Baseline (s) Proposed (s) Full pipeline reduction
1 510.95 452.04 11.530%
2 486.49 420.92 13.477%
3 504.71 449.64 10.912%
4 499.72 459.76 7.996%
5 506.12 461.21 8.874%

Arm order alternates, and the baseline-self repeat varied by 3.02%. Even discarding the gains measured in stages outside template matching — while still keeping their slowdowns — this leaves a 9.85% median full-pipeline saving.

Validation: four focused tests on x86/Numba 0.60 and ARM/Numba 0.68, 256 byte-level distance cases, temporal boundary and order checks, and real-data query comparisons. Ambiguous rounding and unsupported temporal inputs fall back to the old path. This only measures one recording and one sorter — complete sorter output stays stochastic regardless, and biological accuracy wasn't evaluated here.

@alejoe91 alejoe91 added sortingcomponents Related to sortingcomponents module performance Performance issues/improvements labels Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance issues/improvements sortingcomponents Related to sortingcomponents module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants