Skip to content

Support zarr v3 - #4260

Open
alejoe91 wants to merge 77 commits into
SpikeInterface:mainfrom
alejoe91:zarrv3
Open

alejoe91 wants to merge 77 commits into
SpikeInterface:mainfrom
alejoe91:zarrv3

Conversation

@alejoe91

@alejoe91 alejoe91 commented Dec 11, 2025 •

Copy link
Copy Markdown
Member

Fixes #4014 - UPDATED 16/10/2026

Implement bulk changes from zarr v2 to zarr v3

  • create_dataset -> create_array
  • Serialization of non JSON serializable objects (recording/sorting/extensions) not supported anymore
  • Compressors use zarr.codecs, which wrap numcodecs compressor (e.g., Blosc -> BloscCodec)
  • Codec Pipeline: improved generation of codec pipeline to automatically chain filters (ArrayArrayCodec), serializers (standard unless an ArrayBytesCodec is used - e.g. Wavpack), and compressors (BytesBytesCodec)
  • Sharding: new sharding option in the zarr save functions (save(..., format="zarr")), with tests on chunks and shards
  • Backward compatibility: loading objects written with zarr v2 (recordings, sortings, templates, SortingAnalyzer with extensions), including zarr_class_info, channel_ids dtype handling, and storage_options
  • NWB / zarr:
    • zarr links are read from _LINKS for pynwb
    • zarr children iteration is wrapped to skip arrays whose metadata zarr v3 can't parse
    • streaming zarr (stream_mode="zarr") now works with zarr>=3 and its test is re-enabled
    • NWB-zarr tests only run the hdf5 backend when hdmf-zarr is not installed
  • Python: requires-python >= 3.11. The hdmf-zarr>=0.14.0 extras are restricted to python_version>='3.12', since hdmf-zarr 0.14.0 requires Python >=3.12. NWB-zarr (pynwb) is therefore unavailable on 3.11, while plain zarr reading/writing and zarr streaming work everywhere.
  • Removed the requires_zarr_write test marker (zarr writing is always available) and cleaned up tests
  • CI:
    • cross-version serialization workflow checks v2 -> v3 compatibility (including dev) on all supported versions
    • numpy pinned <2.5 in the cross-serialization tests
    • deepinterpolation action disabled
  • Docs: updated the compression section

Depends on:

@alejoe91 alejoe91 added dependencies Issue/PR that is related to dependencies core Changes to core module labels Dec 11, 2025
@alejoe91 alejoe91 closed this Mar 20, 2026
@alejoe91
alejoe91 deleted the zarrv3 branch March 20, 2026 09:41
@alejoe91
alejoe91 restored the zarrv3 branch March 20, 2026 09:41
@alejoe91 alejoe91 reopened this Mar 23, 2026
@alejoe91 alejoe91 added this to the 0.105.0 milestone Mar 24, 2026
Comment thread src/spikeinterface/core/zarrextractors.py
@alejoe91 alejoe91 mentioned this pull request Mar 26, 2026
Comment thread pyproject.toml Outdated
@alejoe91

alejoe91 commented Oct 16, 2026 •

Copy link
Copy Markdown
Member Author

@h-mayorquin not this is Python 3.11, with optional install of hdmf-zarr. Good to merge on my end!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Changes to core module dependencies Issue/PR that is related to dependencies

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrating to zarr v3

2 participants