Skip to content

refactor(deps): move off the deprecated v1 BSON driver entirely - #1497

Merged
ako merged 5 commits into
mainfrom
claude/relaxed-goldberg-q2vra6
Oct 20, 2026
Merged

ako merged 5 commits into
mainfrom
claude/relaxed-goldberg-q2vra6

Conversation

@ako

@ako ako commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Why

go.mongodb.org/mongo-driver v1.17.10 is removed. mxcli now runs on mongo-driver/v2 v2.9.2 alone, and go version -m bin/mxcli lists only v2.

It was the last component on a customer's blocking dependency scan, flagged for CVE-2026-107325, which has no fix on the v1 line and never will — v1.17.10 is that line's final release and the module is marked deprecated. mxcli was never exposed (the panic is in bsoncore.Array.Validate, which no file imported, as established in ako/mxcli#1493) but the finding would have recurred on every scan with no upgrade able to clear it.

This is not a find-and-replace

The two drivers decode nested documents into different Go types, and the type assertions mxcli reads them back with were written against v1's. Measured on identical bytes:

destination v1 nested v2 nested
bson.D primitive.D bson.D equivalent, no action
bson.M primitive.M bson.D breaks .(bson.M)
map[string]any map[string]any bson.D breaks .(map[string]any)

A failed type assertion is not an error, so the value is silently dropped and the next write emits a document with a property missing — which Studio Pro refuses to merge (the canon.CompletePropertySets class, mendixlabs#1373).

The repo's own tests caught every instance. scheduledevents wrote 13 properties where Studio Pro writes 14; TestNoopRerun_ReportsUnchanged tripped the settings guard ("refusing to write a settings document that would drop every part"); the Atlas layout copies lost their icon and image lookups.

The approach: use the switch v2 already has

Rather than audit the 618 type assertions in the migrated files — where a wrong one compiles clean — this uses v2's own decoder options. New leaf package modelsdk/bsoncompat:

helper mechanism restores
UnmarshalDocM DefaultDocumentM() nested bson.M, v1's shape
UnmarshalDocMap DefaultDocumentMap() nested map[string]any
DMap — the bson.D.Map() v2 removed

Applied at 73 call sites, found by how each Unmarshal target was declared, not by reading the call: 39 bson.M destinations and 34 map[string]any.

Three things that are easy to get wrong, all documented at the helpers:

  • The two Unmarshal helpers are not interchangeable. bson.M is a named type (type M map[string]any), so a bson.M value does not satisfy .(map[string]any). mdl/executor's TestResolveWebServiceName_ReadsMapDecodedDocuments already existed for that exact confusion — "cost a debugging round" — and caught this migration too, because it pins its own precondition rather than passing vacuously. Worth copying that pattern.
  • A bson.D destination needs neither helper; applying one there re-introduces the bug inverted.
  • bsoncompat cannot live in modelsdk/codec: codec imports modelsdk/mpr, which imports canon and meta, whose own tests could then not import codec at all (import cycle not allowed in test). It depends on nothing inside mxcli so every layer can use it.

Guards against regression

Re-adding v1 is not a build error, and nothing else would notice until the next dependency scan.

  • TestNoV1DriverImports — walks the tree for the v1 import path
  • TestGoModDoesNotRequireV1Driver — reads go.mod, which is what the scanner reads
  • TestDecodeShapes — pins all three nested shapes, and fails loudly if a driver release makes a helper unnecessary rather than passing vacuously

Controls: with a v1 import and the go.mod require restored, the first names modelsdk/element/zz_ctl_v1.go and the second names the go.mod line.

These guards earned their keep immediately. Merging origin/main brought in ako/mxcli#1495 (ALTER NAVIGATION / ALTER MENU), whose 7 new files import v1. The merge was conflict-free so nothing else flagged them; TestNoV1DriverImports failed on the merged tree and named them. They're migrated in the second commit — all bson.D destinations, so import-path-only.

Third commit: the dead test that blocked go mod tidy

go mod tidy could not run in this repo at all, which is why the v1 requirement was first dropped by hand. The cause was not a dependency problem: mdl/exprcheck/roundtrip_test.go imported mdl/backend/mpr (the legacy engine, deleted) plus executor.GetHierarchyForMining and executor.DescribeMicroflowToString (both removed). It never broke a build because it sat behind //go:build roundtrip — a tag no Makefile target and no CI job has ever passed, and the only file in the tree using it. So it rotted through the engine deletion and two API removals without one red build, and tidy — which considers every build tag — was the only thing that still saw it.

Ported rather than deleted, to mdl/executor/exprcheck_roundtrip_test.go under the integration tag: describing every microflow and re-parsing each expression with the robust parser catches grammar gaps (a missing parsePrimary branch, funcTable entry or slot constraint) that the unit tests cannot, since those only feed the parser expressions someone thought to write down. It now runs under make test-integration-executor, and fails rather than passing silently when the fixture yields no microflows or no expression slots.

With that gone, go mod tidy runs clean and pruned 35 more stale go.sum lines than the hand-edit had — so v1's removal is now the tool's work rather than mine.

This is a third concern on the branch. It is folded in because the proper tidy of go.mod/go.sum in this PR depends on it; happy to split it out if you'd rather review it separately.

Verification

  • go build ./... clean, go vet clean (also under -tags integration), gofmt clean
  • 106 test packages pass, 0 failures (on the merged tree, with origin/main including feat(navigation): alter navigation / alter menu — item-level menu edits #1495)
  • Zero v1 imports remain
  • go mod tidy exits 0; go mod verify — all modules verified
  • go version -m bin/mxcli lists only mongo-driver/v2 v2.9.2
  • Findings recorded; make check-findings passes

The integration (parity) and integration (roundtrip) suites are the real gate here — they exercise the byte-exactness this change could plausibly disturb. Worth waiting for them specifically. integration (executor) is what exercises the ported test; it skips in a container without an mx-test-projects/ checkout, which is why I could not run it locally.

One note for the reviewer

bsonv2 aliases are left as they are — 39 references in files that only ever imported v2 and named it bsonv2 to distinguish it from v1. The name is still accurate, so renaming would be churn in an already large diff.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM

go.mongodb.org/mongo-driver v1.17.10 is removed. mxcli now runs on
mongo-driver/v2 v2.9.2 alone, and `go version -m bin/mxcli` lists only
v2.

It was the last component left on a customer's blocking dependency scan,
flagged for CVE-2026-107325, which has no fix on the v1 line and never
will: v1.17.10 is that line's final release and the module is marked
deprecated. mxcli was never exposed — the panic is in
bsoncore.Array.Validate, which no file imported — but the finding would
have recurred on every scan, with no upgrade able to clear it.

The migration is not a find-and-replace. The two drivers decode nested
documents into different Go types, and the type assertions mxcli reads
them back with were written against v1's. Measured on identical bytes:

  destination        v1 nested        v2 nested
  bson.D             primitive.D      bson.D          equivalent
  bson.M             primitive.M      bson.D          BREAKS
  map[string]any     map[string]any   bson.D          BREAKS

A failed type assertion is not an error, so the value is silently
dropped and the next write emits a document with a property missing,
which Studio Pro refuses to merge (canon.CompletePropertySets,
mendixlabs#1373). The repo's own tests caught every instance:
scheduledevents wrote 13 properties where Studio Pro writes 14,
TestNoopRerun_ReportsUnchanged tripped the settings guard, and the Atlas
layout copies lost their icon and image lookups.

Rather than audit the 618 type assertions in the migrated files — where
a wrong one compiles clean — this uses the switch v2's decoder already
has. New leaf package modelsdk/bsoncompat:

  UnmarshalDocM    DefaultDocumentM()   -> nested bson.M, v1's shape
  UnmarshalDocMap  DefaultDocumentMap() -> nested map[string]any
  DMap                                  -> the bson.D.Map() v2 removed

Applied at 73 call sites found by how each Unmarshal target was
DECLARED, not by reading the call: 39 bson.M destinations and 34
map[string]any. The two helpers are not interchangeable — bson.M is a
named type, so a bson.M value does not satisfy .(map[string]any) — and a
bson.D destination needs neither, since a helper there re-introduces the
bug inverted. mdl/executor's TestResolveWebServiceName_ReadsMapDecodedDocuments
already existed for that exact confusion and caught this migration too,
because it pins its own precondition instead of passing vacuously.

bsoncompat depends on nothing inside mxcli. It cannot live in
modelsdk/codec: codec imports modelsdk/mpr, which imports canon and
meta, whose own tests could then not import codec at all ("import cycle
not allowed in test").

Guards, both verified by control (each names the offending file or
go.mod line): TestNoV1DriverImports walks the tree for the v1 import and
TestGoModDoesNotRequireV1Driver reads go.mod, since re-adding either is
not a build error and nothing else would notice until the next scan.
TestDecodeShapes pins all three nested shapes and fails loudly if a
driver release makes a helper unnecessary.

Note `go mod tidy` cannot run in this repo at all — mdl/exprcheck/
roundtrip_test.go (//go:build roundtrip) imports the long-deleted
mdl/backend/mpr — so the requirement was dropped with `go mod edit
-droprequire` and a manual go.sum prune. That dead test file is
pre-existing and left untouched here.

Verified: go build ./... clean, go vet clean, gofmt clean, 105 test
packages pass with 0 failures, `go mod verify` reports all modules
verified, and the built binary links only mongo-driver/v2 v2.9.2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
Merging origin/main brought in PR #1495 (ALTER NAVIGATION / ALTER MENU),
whose 7 new and changed files import the v1 BSON driver. The merge itself
was conflict-free, so nothing flagged them — which is exactly what
TestNoV1DriverImports is for: it failed on the merged tree and named
them.

All 7 decode into bson.D destinations, which need no bsoncompat helper:
v2 nests bson.D inside bson.D just as v1 nested primitive.D, so their
type assertions are already correct. The change is the import path and
primitive.X -> bson.X only.

Verified on the merged tree: go build ./... clean, go vet clean, gofmt
clean, 106 test packages pass with 0 failures, zero v1 imports remain,
and the built binary links only mongo-driver/v2 v2.9.2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
mdl/exprcheck/roundtrip_test.go could not compile. It imported three
things that no longer exist: mdl/backend/mpr (the legacy engine,
deleted), executor.GetHierarchyForMining and
executor.DescribeMicroflowToString.

It never broke a build because it sat behind `//go:build roundtrip` — a
tag no Makefile target and no CI job has ever passed, and the only file
in the tree that uses it. So it rotted through the legacy-engine
deletion and two executor API removals without one red build. The only
command that still saw it was `go mod tidy`, which considers every build
tag and failed with "no matching versions for query latest" on the
deleted package — a message that reads as a dependency problem rather
than a dead test. That is why the v1 BSON driver had to be dropped from
go.mod with `go mod edit -droprequire` and a hand-pruned go.sum earlier
on this branch.

Ported to mdl/executor/exprcheck_roundtrip_test.go under the
`integration` tag rather than deleted: the assertion is one the unit
tests cannot make. Describing every microflow and re-parsing each
expression with the robust parser catches grammar gaps — a missing
parsePrimary branch, a missing funcTable entry, a missing slot
constraint — where the unit tests only ever feed the parser expressions
someone thought to write down.

It lives in package executor because that is where the real-project
harness already is (setupTestEnv, describeMDL) and because
`make test-integration-executor` actually runs it. The import cycle the
original file's header warned about does not arise in this direction:
mdl/exprcheck does not import mdl/executor. The removed
GetHierarchyForMining is replaced by building the containerID -> module
name map from backend.ListModules().

Two vacuity guards added, because the fixture project lives in a
separate repo (mx-test-projects/) and TestMain skips the whole suite
when it is absent: the test now fails if no microflow could be described
or if no expression slot was found, rather than reporting success
against an empty walk.

With the dead import gone, `go mod tidy` runs clean and pruned 35 more
stale go.sum lines than the hand-edit had, so the v1 driver's removal is
now the tool's work rather than mine.

Verified: go build ./... clean, go vet clean (also with -tags
integration), gofmt clean, 106 test packages pass with 0 failures,
`go mod tidy` exits 0, `go mod verify` reports all modules verified. The
ported test compiles under the integration tag but SKIPS in this
container, which has no mx-test-projects checkout — CI's
`integration (executor)` job is what exercises it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
CI's integration (executor) job failed the round-trip test ported in the
previous commit, and failed it on the vacuity guard rather than on a
hint:

  described 1 microflow(s) but found no expression slots; either the
  test project lost its expressions or the walk below no longer matches
  the AST

The guard was right and the test was wrong. A microflow's ContainerID is
its FOLDER when it sits in one, not its module, so the flat map I built
from ListModules resolved almost nothing — and an unresolved module name
meant the microflow was skipped before it was ever described. That is
precisely the job the removed executor.GetHierarchyForMining did, and
replacing it with a flat lookup is what broke the walk.

NewContainerHierarchyFromBackend is the live replacement: FindModuleID
walks up through folders to the owning module. It is exported and this
test is in package executor, so it needs no new seam.

Measured locally against testdata/expr-checker/minimal.mpr, which this
container does have (the integration fixture, mx-test-projects/, it does
not — hence the first push going out unvalidated):

  resolved by flat ListModules map  :  1/16 microflows
  resolved by hierarchy FindModuleID: 16/16 microflows

and the full describe -> reparse -> walk chain then yields 16 microflows
described and 11 SourceExpr slots, where it previously found 0. The
other 36 expression nodes are LiteralExpr and VariableExpr, which the
walk deliberately ignores: they are already structured, so there is no
raw source text for the robust parser to re-check.

So both halves of the guard's complaint are closed, and the thresholds
it asserts (at least one microflow described, at least one expression
slot) are met with margin rather than exactly.

Verified: go build ./... clean, go vet clean (also with -tags
integration), gofmt clean, 106 test packages pass with 0 failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
ako pushed a commit to ako/mxcli-rest that referenced this pull request Oct 20, 2026
PR ako/mxcli#1497 moves off the deprecated v1 BSON driver across 309 files.
Its own description names the risk: the two drivers decode nested documents
into different Go types, a failed type assertion is not an error, and the
dropped value emits a document with a property missing -- the exact defect
FINDINGS #68 measured and #69 drove to zero here.

Tested as an A/B rather than a survey. Two copies of the committed project
seeded from git archive, all 14 scripts run under a main build and a PR build:
0 of 444 unit files differ, every Unit row including ContentsHash is identical,
and the only difference in the whole .mpr is the _Transaction GUID -- which the
same binary also changes between two writes, so it is per-write noise.

The property-name measurement against `mx convert -p` is still empty in all
three directions, and `mx diff` completes with 64 units Modified and zero
changeDetails. `go version -m` on the two binaries shows v1.17.10 and v2.9.2 for
main, only v2.9.2 for the PR -- the CVE-flagged module is absent from the
artifact a scanner reads.

Two units (the domain model and Home_Web) change against the committed tree,
but the main build rewrites exactly the same two with the same bytes, so they
belong to main's 159 commits -- most likely "keep a layout's stored canvas size
on rewrite" landing on #69's layout. Committed here.

Verified under the PR build: 14 scripts 0 errors 0 deprecations and all pass
--v1=error, exec idempotent, mx check 0 errors, fix hashes 0 mismatches,
fix property-sets already complete, rates lane green, lint 269 under all three
binaries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ADv48FWQnmV8F8oVFN5i8Z

ako commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

Tested this against the Formula1 solution — two Mendix 11.14.0 apps, 1,060 units, 41 MDL scripts. No regression found.

Your description names the failure mode as a silently dropped property on the next write, which is a claim about written bytes, so I compared bytes. Head 36f8cc426 against base 4c0e494e8 (main), both built here, with the base as a control throughout.

The premise

go version -m mxcli   (PR head)   dep go.mongodb.org/mongo-driver/v2 v2.9.2
go version -m mxcli   (base/main) dep go.mongodb.org/mongo-driver     v1.17.10
                                  dep go.mongodb.org/mongo-driver/v2  v2.9.2

Zero v1 import paths in the tree, go.mod requires only v2, and the control build pulling v1.17.10 while this one does not is the claim in its clearest form. Your three guards pass, and TestDecodeShapes is written so it cannot pass vacuously — the trap subtest t.Fatals if the driver ever starts behaving, naming the reason. scheduledevents and TestNoopRerun_ReportsUnchanged pass here too.

Same bytes, where bytes are deterministic

Re-exec of the whole set against the already-provisioned project, from identical committed bytes, with each build:

                        writes   files touched
base run 1 (main)          20          23
base run 2 (main)          20          23
PR head                    20          23

Write lists identical, line for line. The byte comparison needs a noise floor first, because two runs of the same binary don't agree:

same binary, two runs:   7 of 23 files differ
PR vs base:              7 of 23 files differ — exactly the same 7

So all 16 files whose content is deterministic at all are byte-identical to main, and the 7 that differ are the 7 that differ anyway (five pages are Replaced rather than spliced, so a rebuild renumbers element IDs — my project's scripts, not yours).

For the specific failure mode, after the PR build had done its writes:

Formula1Backend   Checked 519 unit(s) … Every unit already has the property names Studio Pro writes.
Formula1Frontend  Checked 541 unit(s) … Every unit already has the property names Studio Pro writes.

Both fix hashes-clean, unit counts unchanged.

Your three named areas, round-tripped

described and the output re-executed:

  • Scheduled event (the 13-vs-14 properties case) → Unchanged scheduled event: Formula1Backend.SE_NarrativeBuild
  • Settings (the "would drop every part" guard) → 4 documents already in sync, guard not tripped
  • Icon lookups → Unchanged navigation profile 'Responsive', all 8 menu items keeping their Atlas_Core.Atlas.* icons, hyphenated quoted ones included ("time-clock", "org-chart", "user-neutral-group", "layout-list")

Also: both script sets check with 0 errors, both apps build and boot at build #1 with no CE codes, all six published OData services return 200 with real rows through x-api-key microflow auth, and playwright check passes with svg.marks>=1.

Two notes, neither about this PR

  • The full re-exec writes 20 documents on main as much as on this branch, so that non-idempotency is pre-existing. Chasing it found a real gap on my side of the line: a relative MetadataUrl ('./contracts/f1-metadata.xml') is stored resolved to an absolute file:///workspaces/… URL, so the committed model carries the authoring container's path and five OData clients churn whenever the checkout path changes. Filed separately rather than here.
  • playwright check <url> fails a page it has just signed into and rendered; the same command without the URL passes #1216 (playwright check <url> failing a page it has signed into and rendered) reproduces unchanged on this build, as expected for an unrelated branch.

One caveat on scope: integration (executor) skips here too, so the ported exprcheck round-trip test is the part I could not exercise. Everything above is the written-model side.

Recorded as §93 of FINDINGS.md in ako/mxcli-formula1.


Generated by Claude Code

@ako
ako merged commit fa57680 into main Oct 20, 2026
31 checks passed
ako pushed a commit that referenced this pull request Oct 20, 2026
main is red. The merge of #1497, which removed
go.mongodb.org/mongo-driver v1 from go.mod, landed just after #1498
added mdl/backend/pagemutator/container_rendermode_test.go — a file
written against v1. The package now fails to compile at all:

  container_rendermode_test.go:8:2: no required module provides package
  go.mongodb.org/mongo-driver/bson
  FAIL github.com/mendixlabs/mxcli/mdl/backend/pagemutator [setup failed]

and TestNoV1DriverImports names the file, which is what it is for.

Neither PR is at fault. #1498 was authored and tested against a main
that still required v1, the two touched different files so nothing
conflicted, and each was green on its own base. The combination is what
breaks.

The file uses bson.D only, so this is the import path and nothing else:
v2 nests bson.D inside bson.D exactly as v1 nested primitive.D, so its
assertions are already correct and it needs no modelsdk/bsoncompat
helper.

Checked the whole tree rather than just the file CI named, since `go
test` stops at the first [setup failed] package and a second offender
would only surface on the next run: this is the only one.

Verified: go build ./... clean, go vet clean, gofmt clean, 106 test
packages pass with 0 failures, the two packages that failed in CI
(mdl/backend/pagemutator and modelsdk/bsoncompat) both pass, and `go mod
tidy` exits 0 leaving go.mod and go.sum unchanged — so v1 stays out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
ako added a commit that referenced this pull request Oct 20, 2026
fix(test): point container_rendermode_test at the v2 BSON driver

Unbreaks main. #1497 removed go.mongodb.org/mongo-driver v1 from go.mod
just after #1498 added a test file written against it, so
mdl/backend/pagemutator stopped compiling and TestNoV1DriverImports
named the file. Neither PR is at fault: each was green on its own base
and the two touched different files, so nothing conflicted — only the
combination breaks.

The file uses bson.D only, so this is the import path alone; v2 nests
bson.D inside bson.D exactly as v1 nested primitive.D, needing no
modelsdk/bsoncompat helper. Whole tree checked, not just the file CI
named: this is the only one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants