Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"area":"mdl/executor","date":"2026-10-20","symptom":"`column (Attribute: Name, Filterable: true)` on a Data grid 2: the original report was a silent drop; on current main check refuses it, but with a project the message lists the column's properties without the `filter` slot, and without one it suggests `did you mean FilterType?` — a key the column does not have","cause":"The refusal (MDL-WIDGET01 / MDL-WIDGET07) built its hint from property names only: the item's widgets slots (`content`, `filter`) are not properties, so neither the property list nor nearestKey over the builtin union could name the slot the author was reaching for","file":"`mdl/executor/validate_widgets.go` (`widgetSlotHint`), `mdl/executor/validate_widget_embedded_schema.go` (`widgetSlots`)","insight":"Re-run an old silent-drop report on main before fixing it: #177 had already become a refusal, and what was left was the message. A did-you-mean over property names cannot reach a slot, so a key whose prefix is a slot key (`Filterable` → `filter`) needs its own answer naming the body form and the widget types the slot routes (itemSlotAcceptedChildTypes, already per widget). Measured on Mendix 11.13.0: the suggested `column (Attribute: X) { textfilter tf }` builds 0 errors. Control: both new assertions fail on main with `its properties are attribute, header (Caption)` and `did you mean FilterType`","refs":["mendixlabs/mxcli#177","#1230"]}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"area":"mdl/executor","date":"2026-10-20","symptom":"`container ctnMain (RenderMode: 'Main')` in `create or modify layout` passes check and exec, then `describe layout` prints `container ctnMain (Class: 'app-main')` — reported as \"the value isn't stored\"; `alter layout … set (RenderMode: …)` refused with `property \"RenderMode\" is not a property of this built-in widget`","cause":"The write was fine: `bson dump` showed `RenderMode: \"Main\"` on the stored Forms$DivContainer. DESCRIBE's DivContainer branch never read RenderMode (only DynamicText did), the ALTER `rendermode` case routed only dynamic texts and sent containers to the pluggable setter, and buildContainerV3 cast any string straight to the enum, so `'main'` or `'Banner'` was stored verbatim","file":"`mdl/executor/cmd_pages_describe_parse.go`, `cmd_pages_describe_output.go`, `cmd_pages_builder_v3_layout.go` (`buildContainerV3`), `mdl/backend/pagemutator/mutator.go` (`setContainerRenderModeMut`), `sdk/pages/pages_widgets_container.go` (`CanonicalContainerRenderMode`)","insight":"A report that a value \"isn't stored\" is usually read off `describe`; run `bson dump` before touching the write path — here it split a suspected write drop into a read-side gap in one command, and the builder had been right all along. The sdk enum had drifted too (`Div`, `Form`; Mendix's set is Div…Address in modelsdk/gen/pages), so take the member list from the generated enum rather than the hand-written one. Control: the four new tests fail on main with `describe dropped the container's RenderMode`, `RenderMode = \"main\", want Main`, `'Banner' must be refused` and `property \"RenderMode\" not found on this widget`","refs":["mendixlabs/mxcli#1582","#1487"]}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"area":"mdl/executor","date":"2026-10-20","symptom":"Marketplace Calendar 2.6.1 page passes check and exec, then `mx check` fails `[CE0642] \"Property 'Editable' is required.\"` and `\"Property 'Show event date range' is required.\"`; `editable: 'true'` is accepted with no warning and changes nothing, `showEventDate:` warns MDL-WIDGET06 \"will be dropped\" although it is written, and `describe page` prints neither","cause":"Four hops, each dropping the same value: (1) GenerateFromMPK's createDefaultWidgetValue had no Expression case, so `defaultValue=\"true\"` was written as an empty expression; (2) GenerateDefJSON emitted no mapping for a top-level `expression` property, leaving it in KnownProperties (MDL-WIDGET06) and written only by the explicit pass; (3) that pass skips builtin names, and `editable` is also the widget-level editability, which the Calendar's package does not declare — so the widget's own `editable` was unreachable in every spelling; (4) extractExplicitProperties had no Expression branch","file":"`modelsdk/widgets/augment.go` (`createDefaultWidgetValue`), `mdl/executor/widget_defs.go` (`GenerateDefJSON`, generator v19), `mdl/executor/widget_engine.go` (`resolveMapping`, `applyOperation`), `mdl/executor/pluggable_system_props.go` (`editableIsOwnProperty`), `mdl/executor/cmd_pages_describe_pluggable.go`","insight":"A builtin MDL key and a widget's own property of the same name collide silently: the builtin always wins, and when the widget does not declare the matching system property the builtin is a no-op too. Decide the routing in the def generator, which reads the .mpk's system properties, so check and build read one answer. The cheapest probe was the reporter's table — `showEventDate:` made the CE0642 go away while `editable:` did not, which already says the explicit pass writes expressions and something else eats `editable`. Make the template write declared defaults first: that alone makes every Calendar page build, before any MDL names the property. Measured on Mendix 11.13.0 with Calendar 245304 v2.6.1: main → 2× CE0642; this branch → 0 errors with neither, both 'true' and both false/'false', and describe → exec reports Unchanged","refs":["mendixlabs/mxcli#1578","#1483"]}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"area": "mdl/backend/pagemutator", "date": "2026-10-20", "symptom": "main went red immediately after ako/mxcli#1497 (the v1 BSON driver removal) merged: `mdl/backend/pagemutator/container_rendermode_test.go:8:2: no required module provides package go.mongodb.org/mongo-driver/bson`, the package failing [setup failed], plus `--- FAIL: TestNoV1DriverImports ... imported again by 1 file(s)`", "cause": "Not a defect in either PR. ako/mxcli#1498 was authored and tested against a main that still required the v1 driver, and merged just before #1497 removed it. Neither PR conflicted with the other — different files — so nothing flagged the combination, and each was green on its own base. The file uses bson.D only, so the fix is the import path alone", "file": "`mdl/backend/pagemutator/container_rendermode_test.go` — v1 bson import -> v2", "insight": "Removing a module the whole repo imports is not finished when the PR is green: any PR already open at that moment is still written against the old module, and a merge of two non-conflicting PRs can break main where neither breaks it alone. This happened TWICE in one change — first caught in-branch when merging origin/main pulled in #1495's 7 new v1 imports, then again post-merge from #1498 — so expect it rather than being surprised. TestNoV1DriverImports is what names the file both times, and its value is exactly here: a re-added import is not a build error while the module is still required, and after removal the compile error names only the package, not the reason. The structural fix is a repo setting, not code: require branches to be up to date before merging, for the window around any dependency removal. When it does happen, check the whole tree (`grep -rn '\"go.mongodb.org/mongo-driver/bson'`) rather than only the file CI named — CI stops at the first [setup failed] package and a second offender would surface only on the next run", "refs": ["ako/mxcli#1497", "ako/mxcli#1498", "ako/mxcli#1495"]}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"area": "mdl/exprcheck", "date": "2026-10-20", "symptom": "`go mod tidy` could not run in the repo at all: \"finding module for package github.com/mendixlabs/mxcli/mdl/backend/mpr ... no matching versions for query latest\". Surfaced while dropping the v1 BSON driver from go.mod, which needs tidy; the workaround was `go mod edit -droprequire` plus hand-pruning go.sum", "cause": "mdl/exprcheck/roundtrip_test.go imported three things that no longer exist: mdl/backend/mpr (the legacy engine, deleted), executor.GetHierarchyForMining and executor.DescribeMicroflowToString. It never failed a build because it sat behind `//go:build roundtrip` — a tag NO Makefile target and no CI job ever passes, and the only file in the tree that uses it. go mod tidy considers every build tag, so it was the one command that still saw the file", "file": "`mdl/exprcheck/roundtrip_test.go` deleted; ported to `mdl/executor/exprcheck_roundtrip_test.go` under the `integration` tag", "insight": "A build tag nothing runs is not a disabled test, it is a deleted test that still looks present — `grep -rl 'go:build <tag>'` returning exactly one file, with no Makefile target naming that tag, is the signal. This one rotted through the legacy-engine deletion AND two executor API removals without a single red build, and the only thing that ever noticed was `go mod tidy`, whose failure reads as a module-resolution problem rather than a dead test. So when tidy complains about a package that was deliberately deleted, look for a test behind an orphan tag, not for a dependency problem. Porting beat deleting: the assertion (describe every microflow, re-parse each expression, expect zero hints from the robust parser) covers grammar gaps the unit tests cannot, since they only feed the parser expressions someone thought to write. Port target is `package executor` under the `integration` tag, where setupTestEnv/describeMDL already provide the real-project harness and `make test-integration-executor` actually runs it; the import cycle the original file's header warned about does not arise in that direction, because mdl/exprcheck does not import mdl/executor. The removed executor.GetHierarchyForMining is replaced by building the containerID->module-name map from backend.ListModules(). Two vacuity guards matter here because the fixture project lives in a separate repo (mx-test-projects/) and TestMain SKIPS the whole suite when it is absent: assert the microflow count is non-zero and the expression-slot count is non-zero, or an empty fixture passes silently. After the port, `go mod tidy` ran clean and pruned 35 more stale go.sum lines than the hand-edit had", "refs": ["ako/mxcli#1497"]}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"area": "modelsdk/bsoncompat", "date": "2026-10-20", "symptom": "go.mongodb.org/mongo-driver v1 stayed on a customer's blocking dependency scan (CVE-2026-107325) with no fix available on the v1 line ever — v1.17.10 is its last release and the module is deprecated. Dropping it meant moving 107 non-test and 179 test files off v1 bson, through the BSON serialization core", "cause": "v1 and v2 decode nested documents into DIFFERENT Go types, and the type assertions mxcli reads them back with were written against v1. Measured on identical bytes: destination bson.D nests primitive.D under v1 and bson.D under v2 (equivalent, no action); destination bson.M nests primitive.M under v1 and bson.D under v2 (BREAKS `doc[k].(bson.M)`); destination map[string]any nests map[string]any under v1 and bson.D under v2 (BREAKS `doc[k].(map[string]any)`). A failed type assertion is not an error, so the value is silently dropped and the next write emits a document with a property missing — unmergeable in Studio Pro (canon.CompletePropertySets, #1373). v2 also removed bson.D.Map()", "file": "`modelsdk/bsoncompat/unmarshal.go` (new leaf package: UnmarshalDocM, UnmarshalDocMap, DMap), 294 files repointed at v2, `go.mod`/`go.sum` v1 requirement dropped", "insight": "Do NOT audit the type assertions — 618 of them in the migrated files, and a wrong one compiles clean. v2's decoder already has the switch: `Decoder.DefaultDocumentM()` restores v1's nested-bson.M and `DefaultDocumentMap()` restores v1's nested-map[string]any, both at every depth including inside arrays. That turns a 1-2 day assertion audit into 2 helpers plus 73 mechanical call-site swaps (39 bson.M destinations + 34 map[string]any), found by grepping each file for how its Unmarshal target was DECLARED, not by reading the call. The two helpers are NOT interchangeable: bson.M is a named type (type M map[string]any), so a bson.M value fails `.(map[string]any)` and vice versa — mdl/executor's TestResolveWebServiceName_ReadsMapDecodedDocuments exists because that exact confusion already cost a debugging round, and it caught this migration too (it pins its own precondition, so it reports \"this test no longer pins the trap it was written for\" rather than passing vacuously — copy that pattern). bson.D destinations need NO helper, and applying one there re-introduces the bug inverted. Placement is load-bearing: the helpers cannot live in modelsdk/codec, which imports modelsdk/mpr, which imports canon and meta — whose own tests then could not import codec (\"import cycle not allowed in test\"); a leaf package depending only on the driver is the only placement every layer can use. Mechanically: the rewrite is a script (284 files in minutes), and the only real breakage is files that imported BOTH drivers with v2 aliased — they end up with two identical imports. Guard against regression with a test that walks the tree for the v1 import AND reads go.mod, since re-adding either is not a build error; verified by control (names the file and the go.mod line). go mod tidy cannot run in this repo at all — mdl/exprcheck/roundtrip_test.go, //go:build roundtrip, imports the long-deleted mdl/backend/mpr — so the requirement was dropped with `go mod edit -droprequire` plus a manual go.sum prune, proven by `go version -m bin/mxcli` showing only v2", "refs": ["CVE-2026-107325", "ako/mxcli#1372", "ako/mxcli#1493", "mendixlabs/mxcli#1373"]}
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [Unreleased]

### Fixed

- **`go mod tidy` runs again, and the expression round-trip test runs for the first time** — `mdl/exprcheck/roundtrip_test.go` imported `mdl/backend/mpr` (the legacy engine, deleted) plus `executor.GetHierarchyForMining` and `executor.DescribeMicroflowToString` (both removed). It never broke a build because it sat behind `//go:build roundtrip`, a tag no Makefile target and no CI job has ever passed — the only file in the tree using it. `go mod tidy` considers every build tag, so it was the one command that still saw the file, and it failed with "no matching versions for query latest" on the deleted package, which reads as a dependency problem rather than a dead test. The test is ported to `mdl/executor/exprcheck_roundtrip_test.go` under the `integration` tag, where `setupTestEnv`/`describeMDL` already provide the real-project harness and `make test-integration-executor` actually runs it; the removed hierarchy helper is replaced by building the container-to-module map from `backend.ListModules()`. Its assertion is worth keeping: describing every microflow and re-parsing each expression with the robust parser catches grammar gaps — a missing `parsePrimary` branch, a missing `funcTable` entry, a missing slot constraint — that the unit tests cannot, since those only feed the parser expressions someone thought to write down. It now also fails rather than passing silently when the fixture project yields no microflows or no expression slots.

### Security

- **The deprecated v1 BSON driver is gone: mxcli runs on `mongo-driver/v2` alone** — `go.mongodb.org/mongo-driver` v1.17.10 is removed from `go.mod`, and `go version -m bin/mxcli` now lists only `mongo-driver/v2 v2.9.2`. It was the last component left on a customer's blocking dependency scan, flagged for CVE-2026-107325, which **has no fix on the v1 line and never will** — v1.17.10 is that line's final release and the module is marked deprecated, so no upgrade could ever have cleared it. mxcli was never exposed (the panic is in `bsoncore.Array.Validate`, which no file imported), but the finding would have recurred on every scan.
- **New `modelsdk/bsoncompat` — the decode shims that make v2 read like v1** — the two drivers decode nested documents into *different Go types*, and mxcli's type assertions were written against v1's. Measured on identical bytes: a `bson.D` destination nests `primitive.D` under v1 and `bson.D` under v2 (equivalent); a `bson.M` destination nests `primitive.M` under v1 but `bson.D` under v2; a `map[string]any` destination nests `map[string]any` under v1 but `bson.D` under v2. The last two silently break a `doc["Key"].(bson.M)` read — a failed type assertion is not an error, so the value is dropped and the next write emits a document with a property missing, which Studio Pro refuses to merge (the `canon.CompletePropertySets` class, mendixlabs/mxcli#1373). `UnmarshalDocM` and `UnmarshalDocMap` restore v1's shape at every depth, arrays included; `DMap` replaces the `bson.D.Map()` method v2 removed. The two Unmarshal helpers are **not** interchangeable — `bson.M` is a named type, so a `bson.M` value does not satisfy `.(map[string]any)` — and a `bson.D` destination needs neither, since applying one there re-introduces the bug inverted. All three shapes are pinned by tests that fail loudly if a future driver release makes a helper unnecessary, rather than passing vacuously.
- **A guard keeps v1 from coming back** — `TestNoV1DriverImports` walks the tree for the v1 import path and `TestGoModDoesNotRequireV1Driver` reads `go.mod`, because re-adding either is not a build error and nothing else would notice until the next dependency scan. Verified by control: both name the offending file and the offending `go.mod` line.

### Security

- **Dependency scan follow-up: `klauspost/compress` and `go-mssqldb` raised to their fixed versions** — `github.com/klauspost/compress` v1.19.2 -> v1.20.3 and `github.com/microsoft/go-mssqldb` v1.10.0 -> v1.11.2. A second scan of the published `.exe`, run after the mongo-driver upgrade, still blocked the release. `klauspost/compress` v1.19.2 predates GHSA-fpmh-frq8-7rj8 (a zstd `DecodeAll` crash on crafted input, fixed in v1.20.2) — the earlier upgrade had only raised it to whatever `mongo-driver/v2` declared, which is not a security floor; it is now an explicit requirement. As before, that module is compiled into no mxcli build (no `go.sum` entry, no `go version -m` line), so the bump moves only the requirement graph the scanner reads. `go-mssqldb` is a blank import in `sql/drivers.go` for driver registration and mxcli calls none of its API, so the bump has no API surface; `TestExternalDriversRegistered` now pins that contract, which had no coverage, and fails with `driver "sqlserver" is not registered` when the import is dropped.
Expand Down
5 changes: 3 additions & 2 deletions cmd/bsondump/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ package main

import (
"fmt"
"github.com/mendixlabs/mxcli/modelsdk/bsoncompat"
"os"

"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/v2/bson"
)

func main() {
Expand All @@ -21,7 +22,7 @@ func main() {
os.Exit(1)
}
var doc bson.M
if err := bson.Unmarshal(data, &doc); err != nil {
if err := bsoncompat.UnmarshalDocM(data, &doc); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
Expand Down
2 changes: 1 addition & 1 deletion cmd/check_bson/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
"fmt"
"os"

"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/v2/bson"
_ "modernc.org/sqlite"
)

Expand Down
2 changes: 1 addition & 1 deletion cmd/compare_pages/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
"os"
"strings"

"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/v2/bson"
)

func main() {
Expand Down
5 changes: 3 additions & 2 deletions cmd/debug/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@ import (
"database/sql"
"encoding/json"
"fmt"
"github.com/mendixlabs/mxcli/modelsdk/bsoncompat"
"os"
"path/filepath"

"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/v2/bson"
_ "modernc.org/sqlite"
)

Expand Down Expand Up @@ -46,7 +47,7 @@ func main() {
}

var doc bson.M
if err := bson.Unmarshal(content, &doc); err != nil {
if err := bsoncompat.UnmarshalDocM(content, &doc); err != nil {
fmt.Fprintf(os.Stderr, "Failed to unmarshal BSON: %v\n", err)
os.Exit(1)
}
Expand Down
2 changes: 1 addition & 1 deletion cmd/dump_page/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"path/filepath"
"strings"

"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/v2/bson"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion cmd/extract_page/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"os"
"strings"

"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/v2/bson"
_ "modernc.org/sqlite"
)

Expand Down
Loading
Loading