Repository navigation
perf: add contiguous fast path to complex add/subtract fallback loop - #278
intel-python-devops wants to merge 1 commit into
Conversation
|
unsure if auto-generated PRs like this should be putting these changes into the CHANGELOG or not. Thoughts? |
Tough call, I would say ideally yes but I would want to make sure we have some kind of anti-AI-slop-text skill or guard before that |
| *oi @OP@= ri; | ||
| return; | ||
| } | ||
| else if (contig) { |
There was a problem hiding this comment.
Wrong results for complex accumulate/cumsum in GCC builds.
What NumPy sends this branch:
np.add.accumulateandnp.cumsumcall the inner loop without = in1 + 1element and all strides contiguous (NumPy v2.5.3 ufunc_object.c L3043–3078).- So contig is
1andcan_vectorizeis 0. That lands in the new branch, where each output depends on the result two scalars earlier.
Why that breaks under GCC:
NPY_PRAGMA_VECTORbecomes#pragma GCC ivdep, which promises the compiler there is no such dependency. GCC then drops its runtime overlap check:-fopt-info-vecshows the new loop vectorized with no versioning, while the old loop was "versioned … because of possible aliasing".- The three existing uses of this pragma (lines 355, 514, 668) all run behind
DISJOINT_OR_SAMEchecks.fast_loop_macros.h:205says ivdep must only be used after that kind of check. The new comment itself says operands may be "overlapping".
The repro: a GCC build of the PR gives mkl_umath.add.accumulate on complex64 [1..8] = [1, 3, -3.62, 0.38, 5, 11, 7, 15]. Expected is [1, 3, 6, 10, …]. The result includes uninitialized output memory, so it's nondeterministic.
Shipped builds use icx, so they're fine today. But meson.build accepts GCC, the README says "a C compiler", and a plain pip install . on Linux would use GCC.
| else if (contig) { | |
| else if (can_vectorize) { |
This matches the other three pragma sites and the overlap guard NumPy itself uses before its fast path. It still covers np.add(Z, C, Z).
| *oi @OP@= ri; | ||
| return; | ||
| } | ||
| else if (contig) { |
There was a problem hiding this comment.
We need a real benchmark numbers in the PR before it merges, including both out-of-place with no overlap and in-place paths
| *oi @OP@= ri; | ||
| return; | ||
| } | ||
| else if (contig) { |
There was a problem hiding this comment.
The PR text and code comment don't match what was measured:
- "Bit-for-bit identical for every stride, overlap…": false.
- "Lets the compiler autovectorize": only true for complex64. For complex128, icx reports "seems inefficient" and only unrolls the loop. This adds 2 new -Wpass-failed warnings for the CDOUBLE add and subtract loops.
- No measurements: the PR gives no benchmark numbers.
np.add(Z, C, Z)isn't an overlap case: the output is the same array as an input, which can_vectorize already allows.
| *oi @OP@= ri; | ||
| return; | ||
| } | ||
| else if (contig) { |
There was a problem hiding this comment.
Missing tests coverage for the new branch
| const @ftype@ *ip1 = (@ftype@ *)args[0]; | ||
| const @ftype@ *ip2 = (@ftype@ *)args[1]; | ||
| @ftype@ *op1 = (@ftype@ *)args[2]; |
There was a problem hiding this comment.
| const @ftype@ *ip1 = (@ftype@ *)args[0]; | |
| const @ftype@ *ip2 = (@ftype@ *)args[1]; | |
| @ftype@ *op1 = (@ftype@ *)args[2]; | |
| const @ftype@ *ip1 = (const @ftype@ *)args[0]; | |
| const @ftype@ *ip2 = (const @ftype@ *)args[1]; | |
| @ftype@ *op1 = (const @ftype@ *)args[2]; |
| *oi @OP@= ri; | ||
| return; | ||
| } | ||
| else if (contig) { |
There was a problem hiding this comment.
Need to populate the changelog
Note
This pull request is entirely AI-generated. Please review thoroughly.
else if (contig)branch between the pairwise-sum reduction case and the genericBINARY_LOOPfallback. When inputs/output are contiguous but the array is below VML_ASM_THRESHOLD (or otherwise ineligible for the oneMKL call), the new branch walks the real and imaginary components as one flat run of 2n ftype elements with a plain indexed loop under NPY_PRAGMA_VECTOR, instead of the char-stepped BINARY_LOOP that dereferences real/imag through pointer casts one complex element at a time.Complex add/subtract previously fell through to the fully strided BINARY_LOOP for every call that misses the oneMKL threshold, even when both inputs and the output are contiguous and simply too small (or overlapping) for the VML call — exactly the case hit repeatedly by workloads like the Mandelbrot kernel's np.add(Z, C, Z) as the working array shrinks below VML_ASM_THRESHOLD each iteration. Since +/- is applied strictly componentwise in the same forward element order as the original loop, flattening real/imag into one typed-array loop is bit-for-bit identical for every stride, overlap and special-value case, but gives the compiler a much better shot at autovectorizing than pointer-cast access through char* increments. The ASV micro-benchmarks only exercise unary float32/float64 ufuncs above the oneMKL threshold, so this path is not directly covered there; BenchMandelbrot2 in the npbench suite is the closest exercised proxy, since it repeatedly calls np.add/np.multiply on shrinking complex128 arrays, many iterations of which fall below the 100000-element VML_ASM_THRESHOLD for complex add.