Repository navigation
perf: reuse TDC peeler distance work and bound temporal neighbor searches - #4874
Open
JESUSROYETH wants to merge 1 commit into
Open
JESUSROYETH wants to merge 1 commit into
JESUSROYETH wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TDC template matching repeats waveform-only squared terms across sparse templates, and scans the full spike array on every temporal-neighbor query. So this change reduces a complete sorting-to-curation pipeline by 10.91% median paired time on the recording below.
The peeler now computes the zero-template channel energy once per query, and only walks the union of candidate channels. Independent accumulators take care of the rest of the reduction, and an outward rounding check falls back to the original kernel whenever the float32 answer is uncertain. An operation-count guard also keeps low-reuse cases on the original kernel. Neighbor queries search sorted sample-time windows and then restore the original index order for amplitude fitting, so the output order doesn't change. Defaults and public signatures stay the same.
Five separate n2-standard-8 CPU VMs ran the first 300 s of public IBL CSH_ZAD_026/2020-09-04/001 probe00, 384 channels at 30 kHz. Each pair includes local compressed-input reading, default
tridesclous2, its preprocessing/cache, a filtered/referenced saved analyzer, extensions, five quality metrics and saved example curation. Four workers, 1 s chunks, seed 42, NumPy 2.3.5/Numba 0.68; fresh processes include JIT compilation. Downloads and installation are excluded.Arm order alternates, and the baseline-self repeat varied by 3.02%. Even discarding the gains measured in stages outside template matching — while still keeping their slowdowns — this leaves a 9.85% median full-pipeline saving.
Validation: four focused tests on x86/Numba 0.60 and ARM/Numba 0.68, 256 byte-level distance cases, temporal boundary and order checks, and real-data query comparisons. Ambiguous rounding and unsupported temporal inputs fall back to the old path. This only measures one recording and one sorter — complete sorter output stays stochastic regardless, and biological accuracy wasn't evaluated here.