Repository navigation
refactor(deps): move off the deprecated v1 BSON driver entirely - #1497
Conversation
go.mongodb.org/mongo-driver v1.17.10 is removed. mxcli now runs on mongo-driver/v2 v2.9.2 alone, and `go version -m bin/mxcli` lists only v2. It was the last component left on a customer's blocking dependency scan, flagged for CVE-2026-107325, which has no fix on the v1 line and never will: v1.17.10 is that line's final release and the module is marked deprecated. mxcli was never exposed — the panic is in bsoncore.Array.Validate, which no file imported — but the finding would have recurred on every scan, with no upgrade able to clear it. The migration is not a find-and-replace. The two drivers decode nested documents into different Go types, and the type assertions mxcli reads them back with were written against v1's. Measured on identical bytes: destination v1 nested v2 nested bson.D primitive.D bson.D equivalent bson.M primitive.M bson.D BREAKS map[string]any map[string]any bson.D BREAKS A failed type assertion is not an error, so the value is silently dropped and the next write emits a document with a property missing, which Studio Pro refuses to merge (canon.CompletePropertySets, mendixlabs#1373). The repo's own tests caught every instance: scheduledevents wrote 13 properties where Studio Pro writes 14, TestNoopRerun_ReportsUnchanged tripped the settings guard, and the Atlas layout copies lost their icon and image lookups. Rather than audit the 618 type assertions in the migrated files — where a wrong one compiles clean — this uses the switch v2's decoder already has. New leaf package modelsdk/bsoncompat: UnmarshalDocM DefaultDocumentM() -> nested bson.M, v1's shape UnmarshalDocMap DefaultDocumentMap() -> nested map[string]any DMap -> the bson.D.Map() v2 removed Applied at 73 call sites found by how each Unmarshal target was DECLARED, not by reading the call: 39 bson.M destinations and 34 map[string]any. The two helpers are not interchangeable — bson.M is a named type, so a bson.M value does not satisfy .(map[string]any) — and a bson.D destination needs neither, since a helper there re-introduces the bug inverted. mdl/executor's TestResolveWebServiceName_ReadsMapDecodedDocuments already existed for that exact confusion and caught this migration too, because it pins its own precondition instead of passing vacuously. bsoncompat depends on nothing inside mxcli. It cannot live in modelsdk/codec: codec imports modelsdk/mpr, which imports canon and meta, whose own tests could then not import codec at all ("import cycle not allowed in test"). Guards, both verified by control (each names the offending file or go.mod line): TestNoV1DriverImports walks the tree for the v1 import and TestGoModDoesNotRequireV1Driver reads go.mod, since re-adding either is not a build error and nothing else would notice until the next scan. TestDecodeShapes pins all three nested shapes and fails loudly if a driver release makes a helper unnecessary. Note `go mod tidy` cannot run in this repo at all — mdl/exprcheck/ roundtrip_test.go (//go:build roundtrip) imports the long-deleted mdl/backend/mpr — so the requirement was dropped with `go mod edit -droprequire` and a manual go.sum prune. That dead test file is pre-existing and left untouched here. Verified: go build ./... clean, go vet clean, gofmt clean, 105 test packages pass with 0 failures, `go mod verify` reports all modules verified, and the built binary links only mongo-driver/v2 v2.9.2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
Merging origin/main brought in PR #1495 (ALTER NAVIGATION / ALTER MENU), whose 7 new and changed files import the v1 BSON driver. The merge itself was conflict-free, so nothing flagged them — which is exactly what TestNoV1DriverImports is for: it failed on the merged tree and named them. All 7 decode into bson.D destinations, which need no bsoncompat helper: v2 nests bson.D inside bson.D just as v1 nested primitive.D, so their type assertions are already correct. The change is the import path and primitive.X -> bson.X only. Verified on the merged tree: go build ./... clean, go vet clean, gofmt clean, 106 test packages pass with 0 failures, zero v1 imports remain, and the built binary links only mongo-driver/v2 v2.9.2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
mdl/exprcheck/roundtrip_test.go could not compile. It imported three things that no longer exist: mdl/backend/mpr (the legacy engine, deleted), executor.GetHierarchyForMining and executor.DescribeMicroflowToString. It never broke a build because it sat behind `//go:build roundtrip` — a tag no Makefile target and no CI job has ever passed, and the only file in the tree that uses it. So it rotted through the legacy-engine deletion and two executor API removals without one red build. The only command that still saw it was `go mod tidy`, which considers every build tag and failed with "no matching versions for query latest" on the deleted package — a message that reads as a dependency problem rather than a dead test. That is why the v1 BSON driver had to be dropped from go.mod with `go mod edit -droprequire` and a hand-pruned go.sum earlier on this branch. Ported to mdl/executor/exprcheck_roundtrip_test.go under the `integration` tag rather than deleted: the assertion is one the unit tests cannot make. Describing every microflow and re-parsing each expression with the robust parser catches grammar gaps — a missing parsePrimary branch, a missing funcTable entry, a missing slot constraint — where the unit tests only ever feed the parser expressions someone thought to write down. It lives in package executor because that is where the real-project harness already is (setupTestEnv, describeMDL) and because `make test-integration-executor` actually runs it. The import cycle the original file's header warned about does not arise in this direction: mdl/exprcheck does not import mdl/executor. The removed GetHierarchyForMining is replaced by building the containerID -> module name map from backend.ListModules(). Two vacuity guards added, because the fixture project lives in a separate repo (mx-test-projects/) and TestMain skips the whole suite when it is absent: the test now fails if no microflow could be described or if no expression slot was found, rather than reporting success against an empty walk. With the dead import gone, `go mod tidy` runs clean and pruned 35 more stale go.sum lines than the hand-edit had, so the v1 driver's removal is now the tool's work rather than mine. Verified: go build ./... clean, go vet clean (also with -tags integration), gofmt clean, 106 test packages pass with 0 failures, `go mod tidy` exits 0, `go mod verify` reports all modules verified. The ported test compiles under the integration tag but SKIPS in this container, which has no mx-test-projects checkout — CI's `integration (executor)` job is what exercises it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
CI's integration (executor) job failed the round-trip test ported in the previous commit, and failed it on the vacuity guard rather than on a hint: described 1 microflow(s) but found no expression slots; either the test project lost its expressions or the walk below no longer matches the AST The guard was right and the test was wrong. A microflow's ContainerID is its FOLDER when it sits in one, not its module, so the flat map I built from ListModules resolved almost nothing — and an unresolved module name meant the microflow was skipped before it was ever described. That is precisely the job the removed executor.GetHierarchyForMining did, and replacing it with a flat lookup is what broke the walk. NewContainerHierarchyFromBackend is the live replacement: FindModuleID walks up through folders to the owning module. It is exported and this test is in package executor, so it needs no new seam. Measured locally against testdata/expr-checker/minimal.mpr, which this container does have (the integration fixture, mx-test-projects/, it does not — hence the first push going out unvalidated): resolved by flat ListModules map : 1/16 microflows resolved by hierarchy FindModuleID: 16/16 microflows and the full describe -> reparse -> walk chain then yields 16 microflows described and 11 SourceExpr slots, where it previously found 0. The other 36 expression nodes are LiteralExpr and VariableExpr, which the walk deliberately ignores: they are already structured, so there is no raw source text for the robust parser to re-check. So both halves of the guard's complaint are closed, and the thresholds it asserts (at least one microflow described, at least one expression slot) are met with margin rather than exactly. Verified: go build ./... clean, go vet clean (also with -tags integration), gofmt clean, 106 test packages pass with 0 failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
PR ako/mxcli#1497 moves off the deprecated v1 BSON driver across 309 files. Its own description names the risk: the two drivers decode nested documents into different Go types, a failed type assertion is not an error, and the dropped value emits a document with a property missing -- the exact defect FINDINGS #68 measured and #69 drove to zero here. Tested as an A/B rather than a survey. Two copies of the committed project seeded from git archive, all 14 scripts run under a main build and a PR build: 0 of 444 unit files differ, every Unit row including ContentsHash is identical, and the only difference in the whole .mpr is the _Transaction GUID -- which the same binary also changes between two writes, so it is per-write noise. The property-name measurement against `mx convert -p` is still empty in all three directions, and `mx diff` completes with 64 units Modified and zero changeDetails. `go version -m` on the two binaries shows v1.17.10 and v2.9.2 for main, only v2.9.2 for the PR -- the CVE-flagged module is absent from the artifact a scanner reads. Two units (the domain model and Home_Web) change against the committed tree, but the main build rewrites exactly the same two with the same bytes, so they belong to main's 159 commits -- most likely "keep a layout's stored canvas size on rewrite" landing on #69's layout. Committed here. Verified under the PR build: 14 scripts 0 errors 0 deprecations and all pass --v1=error, exec idempotent, mx check 0 errors, fix hashes 0 mismatches, fix property-sets already complete, rates lane green, lint 269 under all three binaries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADv48FWQnmV8F8oVFN5i8Z
|
Tested this against the Formula1 solution — two Mendix 11.14.0 apps, 1,060 units, 41 MDL scripts. No regression found. Your description names the failure mode as a silently dropped property on the next write, which is a claim about written bytes, so I compared bytes. Head The premiseZero v1 import paths in the tree, Same bytes, where bytes are deterministicRe-exec of the whole set against the already-provisioned project, from identical committed bytes, with each build: Write lists identical, line for line. The byte comparison needs a noise floor first, because two runs of the same binary don't agree: So all 16 files whose content is deterministic at all are byte-identical to main, and the 7 that differ are the 7 that differ anyway (five pages are For the specific failure mode, after the PR build had done its writes: Both Your three named areas, round-tripped
Also: both script sets Two notes, neither about this PR
One caveat on scope: Recorded as §93 of Generated by Claude Code |
main is red. The merge of #1497, which removed go.mongodb.org/mongo-driver v1 from go.mod, landed just after #1498 added mdl/backend/pagemutator/container_rendermode_test.go — a file written against v1. The package now fails to compile at all: container_rendermode_test.go:8:2: no required module provides package go.mongodb.org/mongo-driver/bson FAIL github.com/mendixlabs/mxcli/mdl/backend/pagemutator [setup failed] and TestNoV1DriverImports names the file, which is what it is for. Neither PR is at fault. #1498 was authored and tested against a main that still required v1, the two touched different files so nothing conflicted, and each was green on its own base. The combination is what breaks. The file uses bson.D only, so this is the import path and nothing else: v2 nests bson.D inside bson.D exactly as v1 nested primitive.D, so its assertions are already correct and it needs no modelsdk/bsoncompat helper. Checked the whole tree rather than just the file CI named, since `go test` stops at the first [setup failed] package and a second offender would only surface on the next run: this is the only one. Verified: go build ./... clean, go vet clean, gofmt clean, 106 test packages pass with 0 failures, the two packages that failed in CI (mdl/backend/pagemutator and modelsdk/bsoncompat) both pass, and `go mod tidy` exits 0 leaving go.mod and go.sum unchanged — so v1 stays out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM
fix(test): point container_rendermode_test at the v2 BSON driver Unbreaks main. #1497 removed go.mongodb.org/mongo-driver v1 from go.mod just after #1498 added a test file written against it, so mdl/backend/pagemutator stopped compiling and TestNoV1DriverImports named the file. Neither PR is at fault: each was green on its own base and the two touched different files, so nothing conflicted — only the combination breaks. The file uses bson.D only, so this is the import path alone; v2 nests bson.D inside bson.D exactly as v1 nested primitive.D, needing no modelsdk/bsoncompat helper. Whole tree checked, not just the file CI named: this is the only one.
Why
go.mongodb.org/mongo-driverv1.17.10 is removed. mxcli now runs onmongo-driver/v2 v2.9.2alone, andgo version -m bin/mxclilists only v2.It was the last component on a customer's blocking dependency scan, flagged for CVE-2026-107325, which has no fix on the v1 line and never will — v1.17.10 is that line's final release and the module is marked deprecated. mxcli was never exposed (the panic is in
bsoncore.Array.Validate, which no file imported, as established in ako/mxcli#1493) but the finding would have recurred on every scan with no upgrade able to clear it.This is not a find-and-replace
The two drivers decode nested documents into different Go types, and the type assertions mxcli reads them back with were written against v1's. Measured on identical bytes:
bson.Dprimitive.Dbson.Dbson.Mprimitive.Mbson.D.(bson.M)map[string]anymap[string]anybson.D.(map[string]any)A failed type assertion is not an error, so the value is silently dropped and the next write emits a document with a property missing — which Studio Pro refuses to merge (the
canon.CompletePropertySetsclass, mendixlabs#1373).The repo's own tests caught every instance.
scheduledeventswrote 13 properties where Studio Pro writes 14;TestNoopRerun_ReportsUnchangedtripped the settings guard ("refusing to write a settings document that would drop every part"); the Atlas layout copies lost their icon and image lookups.The approach: use the switch v2 already has
Rather than audit the 618 type assertions in the migrated files — where a wrong one compiles clean — this uses v2's own decoder options. New leaf package
modelsdk/bsoncompat:UnmarshalDocMDefaultDocumentM()bson.M, v1's shapeUnmarshalDocMapDefaultDocumentMap()map[string]anyDMapbson.D.Map()v2 removedApplied at 73 call sites, found by how each
Unmarshaltarget was declared, not by reading the call: 39bson.Mdestinations and 34map[string]any.Three things that are easy to get wrong, all documented at the helpers:
Unmarshalhelpers are not interchangeable.bson.Mis a named type (type M map[string]any), so abson.Mvalue does not satisfy.(map[string]any).mdl/executor'sTestResolveWebServiceName_ReadsMapDecodedDocumentsalready existed for that exact confusion — "cost a debugging round" — and caught this migration too, because it pins its own precondition rather than passing vacuously. Worth copying that pattern.bson.Ddestination needs neither helper; applying one there re-introduces the bug inverted.bsoncompatcannot live inmodelsdk/codec: codec importsmodelsdk/mpr, which importscanonandmeta, whose own tests could then not import codec at all (import cycle not allowed in test). It depends on nothing inside mxcli so every layer can use it.Guards against regression
Re-adding v1 is not a build error, and nothing else would notice until the next dependency scan.
TestNoV1DriverImports— walks the tree for the v1 import pathTestGoModDoesNotRequireV1Driver— readsgo.mod, which is what the scanner readsTestDecodeShapes— pins all three nested shapes, and fails loudly if a driver release makes a helper unnecessary rather than passing vacuouslyControls: with a v1 import and the
go.modrequire restored, the first namesmodelsdk/element/zz_ctl_v1.goand the second names thego.modline.These guards earned their keep immediately. Merging
origin/mainbrought in ako/mxcli#1495 (ALTER NAVIGATION / ALTER MENU), whose 7 new files import v1. The merge was conflict-free so nothing else flagged them;TestNoV1DriverImportsfailed on the merged tree and named them. They're migrated in the second commit — allbson.Ddestinations, so import-path-only.Third commit: the dead test that blocked
go mod tidygo mod tidycould not run in this repo at all, which is why the v1 requirement was first dropped by hand. The cause was not a dependency problem:mdl/exprcheck/roundtrip_test.goimportedmdl/backend/mpr(the legacy engine, deleted) plusexecutor.GetHierarchyForMiningandexecutor.DescribeMicroflowToString(both removed). It never broke a build because it sat behind//go:build roundtrip— a tag no Makefile target and no CI job has ever passed, and the only file in the tree using it. So it rotted through the engine deletion and two API removals without one red build, andtidy— which considers every build tag — was the only thing that still saw it.Ported rather than deleted, to
mdl/executor/exprcheck_roundtrip_test.gounder theintegrationtag: describing every microflow and re-parsing each expression with the robust parser catches grammar gaps (a missingparsePrimarybranch,funcTableentry or slot constraint) that the unit tests cannot, since those only feed the parser expressions someone thought to write down. It now runs undermake test-integration-executor, and fails rather than passing silently when the fixture yields no microflows or no expression slots.With that gone,
go mod tidyruns clean and pruned 35 more stalego.sumlines than the hand-edit had — so v1's removal is now the tool's work rather than mine.This is a third concern on the branch. It is folded in because the proper
tidyofgo.mod/go.sumin this PR depends on it; happy to split it out if you'd rather review it separately.Verification
go build ./...clean,go vetclean (also under-tags integration),gofmtcleanorigin/mainincluding feat(navigation): alter navigation / alter menu — item-level menu edits #1495)go mod tidyexits 0;go mod verify— all modules verifiedgo version -m bin/mxclilists onlymongo-driver/v2 v2.9.2make check-findingspassesThe
integration (parity)andintegration (roundtrip)suites are the real gate here — they exercise the byte-exactness this change could plausibly disturb. Worth waiting for them specifically.integration (executor)is what exercises the ported test; it skips in a container without anmx-test-projects/checkout, which is why I could not run it locally.One note for the reviewer
bsonv2aliases are left as they are — 39 references in files that only ever imported v2 and named itbsonv2to distinguish it from v1. The name is still accurate, so renaming would be churn in an already large diff.🤖 Generated with Claude Code
https://claude.ai/code/session_01Rer1mzP69XxbXKxQqheipM