Skip to content

fix(ci): fail the test job on compile errors and bump RoslynTestKit to 1.5.1 - #592

Merged
Arthurvdv merged 3 commits into
mainfrom
fix/ci-fail-test-job-on-compile-errors
Oct 20, 2026
Merged

Arthurvdv merged 3 commits into
mainfrom
fix/ci-fail-test-job-on-compile-errors

Conversation

@Arthurvdv

@Arthurvdv Arthurvdv commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Summary

The scheduled build is green although two net10.0 test legs (Test v16.10.44.19458, Test v17.10.44.19457) compile none of the cop test projects.

Root cause: the CI compile floor for net10.0 moved down to SDK 16.10.44.19458 when Microsoft published net10.0 VSIX builds for older AL majors (2026-10-18). ALCops.RoslynTestKit 1.5.0 references 18.0.41.39415, so every leg with an SDK below that fails with CS1705. Every Run *.Test step was continue-on-error, which hid the compile error: no trx, nothing reported, job green.

This PR:

  • builds each test project in its own step without continue-on-error, then runs dotnet test --no-build (still continue-on-error) so a compile error fails the job while test failures still let every cop x AL version report;
  • bumps ALCops.RoslynTestKit to 1.5.1 (net10.0 lib compiled against 16.10.44.19458, the lowest marketplace net10.0 release);
  • pins the local net10.0 SDK in .vscode/Setup-BCDevTools.ps1 to the same 16.10 floor, and updates the rules docs.

Commits

  1. fix(ci): fail the test job when a test project does not compile - the CI gate and docs. Pushed first on purpose: its run is the failing-first proof of the gate.
  2. chore: bump RoslynTestKit to 1.5.1 and pin the local net10.0 SDK to the 16.10 floor - the fix.
  3. chore(ci): name the build steps in the no-trx error and keep the floor doc to current state - review follow-up.

Developers

Re-run .vscode/Setup-BCDevTools.ps1 so the local net10.0 SDK matches the 16.10 floor.

Test plan

  • Locally with RoslynTestKit 1.5.0 against SDK 16.10.44.19458: dotnet test ALCops.sln fails with CS1705
  • Run of commit 1 (39409566734): Test v16.10.44.19458 and Test v17.10.44.19457 fail at Build ALCops.ApplicationCop.Test with CS1705, all other legs green, merge/report steps still run
  • Locally with RoslynTestKit 1.5.1: dotnet test ALCops.sln passes
  • Run of commit 2 (39410728615): all legs green, seven Build ALCops.*.Test steps per leg, no CS1705, "Test results" check green
  • Run of commit 3 green

Review notes

  • A hard Build step skips the later cops on that leg: deliberate. A compile error must fail the job at its own step; the other legs plus merge and report still run (run 39409566734), and compile failures are systemic (same SDK), so skipped cops carry no extra signal.
  • Download/Restore are soft but Build is hard: not a regression. A restore failure made the leg useless before as well; it used to surface later via the missing-trx check.
  • The NuGet branch of Setup-BCDevTools.ps1 is unused but kept, so the pin can move back to a NuGet source when the floor does.
  • The seven repeated step blocks are the pre-existing layout and out of scope.

🤖 Generated with Claude Code

Arthurvdv and others added 3 commits October 20, 2026 12:40
Every test step in build-test.yml was continue-on-error, so a compile
error in a test project (CS1705 from RoslynTestKit referencing a newer
SDK than the leg's floor) produced no trx and was silently skipped; the
daily build stayed green for weeks.

Each test project is now built in its own step without continue-on-error,
so a compile error fails the job at that step. `dotnet test --no-build`
(same NavTargetFramework and Configuration, so it runs the exact build
output) stays continue-on-error, so test failures still do not stop the
remaining cops and AL versions from reporting together.

The rules docs now describe the gate and the RoslynTestKit floor
contract: the per-TFM floor moves down when Microsoft publishes a
net10.0 build for an older AL major, and the remedy for CS1705 is
re-pinning and releasing RoslynTestKit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…he 16.10 floor

RoslynTestKit 1.5.1 compiles its net10.0 lib against SDK 16.10.44.19458,
the lowest marketplace net10.0 release, so it binds on every leg at or
above the CI compile floor.

Setup-BCDevTools.ps1 now pins net10.0 to the same 16.10 VSIX so the
local compile floor equals the CI floor.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r doc to current state

The no-trx error in Merge Test Results now points at the Build steps as
well as the Setup and Run steps. The floor row in get-bc-devtools.md
drops its dated history, states the mechanism and current values, and
explains why RoslynTestKit must be compiled against an SDK at or below
the floor (CS1705).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Arthurvdv
Arthurvdv merged commit 88f251a into main Oct 20, 2026
52 checks passed
@Arthurvdv
Arthurvdv deleted the fix/ci-fail-test-job-on-compile-errors branch October 20, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant