Repository navigation
Test by-element sdot/udot codegen on ARM - #9544
Open
alexreinking wants to merge 1 commit into
Open
alexreinking wants to merge 1 commit into
alexreinking wants to merge 1 commit into
Conversation
Lock in that a dot product against a 4-byte group broadcast from a register-staged vector lowers to the by-element form `sdot/udot vD.4s, vN.16b, vM.4b[lane]`, for either operand order. This relies on CodeGen_ARM moving the broadcast to the second operand, which had no test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
I'm adding this test because Copilot spent well over an hour discovering this recipe even after searching the repo for key terms. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9544 +/- ##
=======================================
Coverage 71.05% 71.05%
=======================================
Files 262 262
Lines 81084 81084
Branches 19770 19770
=======================================
+ Hits 57615 57618 +3
+ Misses 17502 17488 -14
- Partials 5967 5978 +11 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
alexreinking
marked this pull request as ready for review
October 10, 2026 12:07
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
correctness_arm_dot_product_by_element, which locks in by-elementsdot/udotcodegen on ARM (sdot vD.4s, vN.16b, vM.4b[lane], thevdotq_laneq_s32instruction).The test compiles an 8-row gemv with weights interleaved 4 rows at a time in 4-byte groups, as in ggml's q4_0_4x4 and KleidiAI's dotprod kernels. It stages 16 activation bytes in a register and unrolls 4 groups.
It requires the by-element form on all four lanes, for
sdotandudot, with the broadcast operand on either side of the multiply. On an AArch64 host with dotprod it also checks values against a scalar reference.This relies on
CodeGen_ARMmoving the broadcast to the second operand ofdot_product(since 8aa778c), which had no test. With that swap removed, the test fails (found 0 lanes).Test-only; no compiler changes.
Validation on macOS arm64:
ctest --test-dir build/macOS -R correctness_arm_dot_product_by_elementpasses, and fails with the swap removed.Authored by GitHub Copilot