Repository navigation
Add fir_filter step and FIR highpass and bandpass designs - #4855
Open
galenlynch wants to merge 4 commits into
Open
galenlynch wants to merge 4 commits into
galenlynch wants to merge 4 commits into
Conversation
fir_filter(recording, taps) applies any FIR kernel, shared or per channel, by overlap-save. Its margin is the kernel's support, so its output does not depend on chunking. Channels are filtered in blocks sized to keep each block's buffers in the L2 cache. design_fir_bandpass and design_fir_highpass design Kaiser-window kernels from a stated spec: -6 dB at the cutoffs, a given transition width at each edge, and a given stopband attenuation. Keeping design separate from application lets kernels be inspected or combined before use. The defaults (175 Hz lower transition, 45 dB stopband) pass at most 0.03% of the probe's specified noise power below 150 Hz, and no more than the default Butterworth between 150 and 300 Hz, on every channel of a Neuropixels 2.0 recording. A 400 Hz lower transition passed 2.4 times the Butterworth's power between 150 and 300 Hz. check_chunk_invariance in core.testing compares chunked reads with one whole read, so any preprocessor's seam error can be tested directly.
The core test jobs run without scipy and collect every test module, so a module-level scipy import failed collection there.
Contributor
Author
|
The test failures came from a module-level "Load fixtures from 0.105.1" fails for a reason outside this PR. In else:
rec_with_probe = rec.set_probe(probe) # new API returns a new recording; portable across versionsIn 0.105.1, else:
rec.set_probe(probe)
rec_with_probe = recThe job runs only for PRs that touch |
This was referenced Oct 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a step that applies any FIR kernel, and separate functions that design the default kernels, as suggested on #4563:
fir_filter(recording, taps)applies a FIR kernel by overlap-save, either one kernel for all channels or one per channel group. Its output doesn't depend on chunking.design_fir_highpass(freq_min, fs)anddesign_fir_bandpass(freq_min, freq_max, fs)design Kaiser-window kernels: −6 dB at the cutoffs, a 175 Hz transition at the low edge (and 2 kHz at the bandpass's high edge), and at least 45 dB down beyond. The default highpass has 463 taps and the default bandpass 503.Best merged after #4853: without it,
phase_shiftfollowed by this filter runs an FFT of a slow length.Speed. One thread, 1 s chunks of 384 channels, i9-12900K:
get_traces, int16 in and float32 out, best of 7; MS/s counts channel-samples. The IIR issosfiltfilt, which computes in float64.Saving 30 s through the filter keeps most of the gain and uses less RAM. Input and output are in RAM, so the time covers reading, filtering and writing but not the disk (median of 3; RAM is the peak above baseline):
In a chain of
phase_shift, the highpass and mediancommon_reference, the FIR makes saving 1.15–1.17 times faster with main'sphase_shift(given #4853), and 1.27–1.36 times faster with #4563's.Response. The defaults aim to be no worse than the default Butterworth. On 383 channels of a Neuropixels 2.0 recording, compared with the probe's noise spec (69 nV/√Hz):
Median spectrum over channels, relative to the probe's noise spec, for the 300 Hz highpass (top) and 300–6000 Hz bandpass (bottom).
Precision. A FIR has no poles, so float32 rounding errors don't build up, even at low cutoffs:
sosfiltfiltin float32rms error against a float64 reference, relative to the signal; 10 s of 32 channels.
Seams. Each output sample depends only on the
len(taps) - 1input samples around it, and the step reads exactly those, so chunked output equals whole-signal output up to float rounding. The PR addscheck_chunk_invariance(recording, chunk_size, atol)tocore.testingto check this for any preprocessor:Numbers
Largest seam error with 1 s chunks, on 10 s of a Neuropixels 2.0 probe (384 int16 channels, 2.34 µV per count, float32 output):
Shorter kernels miss the design goal. A 400 Hz transition (221 taps), for example, passes 2.2 times the Butterworth's power between 150 and 300 Hz.
Details
nissum_j taps[j] * x[n + center - j]. The defaultcentermakes a symmetric kernel zero-phase, andcenter=0makes the filter causal.add_reflect_padding=True, as infilter.offset_to_uVis scaled by the kernel's DC gain.scipy.fftandscipy.signal.A follow-up PR adds
method="fir"tohighpass_filterandbandpass_filter, for users who don't want to design a kernel.