Skip to content

Add a reusable test suite for the backends - #323

Draft
lkdvos wants to merge 5 commits into
ci-loadfrom
testsuite
Draft

lkdvos wants to merge 5 commits into
ci-loadfrom
testsuite

Conversation

@lkdvos

@lkdvos lkdvos commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Adds a reusable test suite for TensorOperations backends, in the spirit of the GPUArrays.jl and MatrixAlgebraKit.jl test suites, and fixes the bugs it found in the backends. Stacked on #320.

Test suite

test/testsuite/TestSuite.jl tests tensoradd!, tensortrace!, tensorcontract! and their allocating versions for a given backend:

include(joinpath(pkgdir(TensorOperations), "test", "testsuite", "TestSuite.jl"))
TestSuite.test_backend(MyBackend(); arraytype = CuArray, eltypes = (Float32, ComplexF32))
  • Reference results: every result is compared against a reference backend, BaseCopy() by default, run on CPU copies of the same inputs, so GPU backends are checked against CPU results. BaseCopy itself is checked against straightforward loop implementations (TestSuite.LoopReference()).
  • Capabilities: the TestSuite.Options fields describe what a backend supports: views, negative strides, Adjoint/Transpose inputs, size-zero dimensions, Diagonal arguments, rejecting aliased outputs, and the combinations of mixed element types to test (() for none).
  • Edge cases:
    • C filled with NaN whenever β is zero, to check strong-zero semantics, for both Zero() and numeric zeros
    • One()/Zero() and a zero α
    • zero-dimensional arrays, and dimensions of size zero and one
    • full traces and traces without traced indices
    • inner, outer and full contractions
    • layouts that cannot be fused into a matrix multiplication
    • strided views, including negative strides
    • Adjoint/Transpose inputs, combined with conjA
    • Diagonal arguments, including diagonal-times-diagonal into a Diagonal
    • mixed element types and BigFloat
    • invalid indices and sizes, and aliased outputs
    • inputs are checked to stay unchanged
  • Where it runs:
    • test/core/backends/: BaseCopy, BaseView, StridedNative, StridedBLAS (also with the manual and buffer allocators) and DefaultBackend.
    • extensions/tblis.jl: TBLIS.
    • extensions/gpuarrays.jl: the Strided backends on JLArray/CuArray, and cuTENSOR and DefaultBackend on CuArray.
  • Removed duplicates: the per-operation tests of methods.jl, tblis.jl and gpuarrays.jl that the suite covers.

Bugs found and fixed

Backend Bug
BaseCopy, BaseView trace over a size-zero dimension threw a BoundsError
TBLIS trace over a size-zero dimension returned NaN or an unscaled result
cuTENSOR no index or size checks: an invalid permutation was silently accepted
cuTENSOR errors on size-zero dimensions, and on a trace without traced indices
StridedNative contraction did not reject an output that aliases an input
DefaultBackend on CuArray selected cuTENSOR for mixed element types, which cuTENSOR does not support; it now falls back to the Strided backends

The size-zero cases with an output that may not be read are handled through a shared _scale! helper, which overwrites C for β = 0.

Not supported, and switched off through the options: mixed element types and negative strides for cuTENSOR, mixed element types for TBLIS, and Diagonal arguments for StridedBLAS.

Other changes

  • Test dependencies: moved to test/Project.toml (with [workspace] and [sources]), as in TensorKit and MatrixAlgebraKit.
  • cuBLAS: added as a test dependency, since StridedBLAS on CuArray needs Strided's cuBLAS extension.
  • CI: ci.yml excludes testsuite from the test groups, and runtests.jl skips the suite's own files.

Strided

Requires Strided v2.6.7 (QuantumKitHub/Strided.jl#100): before that, StridedBLAS contractions over a size-zero dimension left NaNs for β = 0 (conjugated complex and non-BLAS element types), and failed on CuArray. The compat bound is raised to 2.6.7 in both Project.toml and test/Project.toml.

Testing

All suites pass locally with Julia 1.13, including on a local GPU (the core/backends/strided.jl suite also rerun against the released Strided v2.6.7). Each file was run in its own module through a scratch script, not through runtests.jl:

Test file Pass
core/backends/basecopy.jl 10294
core/backends/baseview.jl 6834
core/backends/strided.jl 27972
core/backends/default.jl 6834
core/methods.jl 92
extensions/tblis.jl 6417
extensions/gpuarrays.jl (JLArray and CuArray) 20398, plus 2 already marked broken

The suite is compile-heavy: the core group takes noticeably longer than before.

🤖 Generated with Claude Code

lkdvos and others added 5 commits November 2, 2026 11:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kends

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- cuTENSOR: check the indices and sizes of the arguments, and handle empty outputs,
  contractions and traces over dimensions of size zero, and traces without traced indices
- TBLIS: handle traces over dimensions of size zero
- StridedNative: reject outputs that alias an input of a contraction

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every backend is compared to BaseCopy on CPU copies of the inputs, which is itself compared
to plain loop implementations. Backends of other array types, e.g. GPU arrays, are therefore
compared to CPU results. The test suite replaces the per-backend tests of the primitive
operations, and adds many more edge cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant