fix: describe round-trip gaps on pages (gallery sort over association, by-key combo box / filter settings)

.claude/skills/fix-issue/findings/mdl-executor/2026-10-21-describe-combobox-filter-bykey-settings.json

@@ -0,0 +1 @@
+{"area":"mdl/executor","date":"2026-10-21","symptom":"`describe page` omitted a combo box's `customEditability` / `customEditabilityExpression` and a data grid text/number filter's `defaultValue` expression and `delay`, though exec stored them (bson dump shows `Expression: $currentObject/Locked`); the report's own generic widget (`entitypicker … Placeholder:`) and its MDL-WIDGET06 \"not yet persisted\" warning reproduce only on the pre-#1578 binary","cause":"Widgets with a dedicated describe reader (isKnownCustomWidgetType) never ran extractExplicitProperties, and only Data grid 2 and Gallery had been given the by-key reader (#1573); the combo box and filter branch printed its keyword set and nothing else. Separately, nonDefaultPropsByKey compared with the schema DefaultValue, but exec writes the embedded template's value for an absent property — combo box readOnlyStyle is `bordered` in the template against a default of `text`","file":"`mdl/executor/cmd_pages_describe_bykey.go` (sharedBranchByKeySkip, templateWrittenDefaults, extractSharedBranchByKeyProps, withoutUndescribedComboBoxModes), `cmd_pages_describe_parse.go`, `cmd_pages_describe_output.go` (generic branch no longer claims a known widget with explicit props), `widget_known_props_from_mpk.go` + `validate_widgets.go` (enrichKnownPropertiesFromTemplates for check without -p)","insight":"Retest an old report against a pre-fix worktree before theorising: the reporter's exact shape was already fixed a day earlier, and only a probe on SHIPPED widgets with dedicated readers found what was still live. Compare by-key values with what exec writes when the key is absent (the template object), not the schema default — the corpus diff against a baseline build showed Studio Pro's own Administration combo boxes store `readOnlyStyle: text`, which describe omitted and an edit-replay would have flipped to `bordered`. Settings of a mode DESCRIBE does not spell (combo box database/static) are named in a NOT re-executable note, not printed into a combo box of another mode. And a key describe prints must pass `check` WITHOUT -p too: the hand-crafted defs only learn the widget's keys from the project's .mpk, so the embedded template now stands in when there is none. Not done: datagrid.json's template stores columnsFilterable=false against default true — the #1573 reader still compares with the default there","refs":["mendixlabs/mxcli#1586","ako/mxcli#1491","mendixlabs/mxcli#1578"],"test":"mdl/executor/cmd_pages_describe_shared_bykey_test.go"}

.claude/skills/fix-issue/findings/mdl-executor/2026-10-21-describe-gallery-sort-over-association.json

@@ -0,0 +1 @@
+{"area":"mdl/executor","date":"2026-10-21","symptom":"`describe page` printed a gallery whose Database source runs over an association as `gallery g1 (DataSource: $currentObject/Probe.Item_Parent, …)` — no `sort by`, no `where`, although the stored CustomWidgets$CustomWidgetXPathSource carries a GridSortBar; and `create page` refused the faithful spelling with \"`database from $currentObject/…` is a LIST VIEW source reached over an association, and gallery cannot store one\"","cause":"parseDataSource sent an XPath source with an IndirectEntityRef to parseEntityPathSource only for Forms$ListViewXPathSource (#721 L5); the pluggable CustomWidgetXPathSource fell to parseContextSource (the #941 reading), which yields an association source and drops XPath and sort — and replays as a Forms$AssociationSource, a different retrieve. On the write side checkDatabaseOverAssociationIsOnAListView refused the form for every non-list-view widget on 10+, and customWidgetDataSourceToGen wrote only a DirectEntityRef","file":"`mdl/executor/cmd_pages_describe_datasource.go` (parseDataSource), `mdl/executor/pluggable_database_source.go` (buildPluggableDataSourceV3), `mdl/executor/cmd_pages_builder_v3.go` (check removed; buildDataSourceV3 refusal reworded), `mdl/backend/modelsdk/widget_write.go` (customWidgetDataSourceToGen: IndirectEntityRef + SourceVariable)","insight":"The 9.x fix for the same shape (#1399) already had the whole write path — buildDatabaseOverAssociationV3 and the version gate — so the 10+ case was a one-branch change once the reader stopped treating the XPath source as an association; check which source TYPE a reading replays to, not just whether the text parses. The #941 association reading was itself lossy (type change on replay) and only looked faithful because a source with no sort replays to the same rows. Measured on 11.12.1: the reporter's shape authored directly in MDL, mx check 0 errors, describe -> exec `Unchanged page`; control with the association name byte-patched to a nonexistent one -> CE1613 at Gallery 'g1', so mx check does read the path. A hop with no stored destination keeps the association reading","refs":["mendixlabs/mxcli#1599","ako/mxcli#1509"],"test":"mdl/executor/pluggable_xpath_over_association_test.go; mdl/backend/modelsdk/cw_xpath_source_over_association_test.go"}

.github/workflows/nightly.yml

@@ -40,7 +40,7 @@ jobs:
 
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
 
       - name: Cache ANTLR4 JAR
         uses: actions/cache@v6
@@ -89,7 +89,7 @@ jobs:
 
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
 
       # mxbuild 9 draws through System.Drawing; without libgdiplus every mx call
       # dies with "The type initializer for 'Gdip' threw an exception."
@@ -144,7 +144,7 @@ jobs:
       - uses: actions/checkout@v7
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
       - name: Cache ANTLR4 JAR
         uses: actions/cache@v6
         with:
@@ -175,7 +175,7 @@ jobs:
 
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
 
       - uses: oven-sh/setup-bun@v2
 

.github/workflows/push-test.yml

@@ -23,7 +23,7 @@ jobs:
       - uses: actions/checkout@v7
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
       - name: Test the tunnel seam
         shell: bash
         # Scoped with -run to the seam's own tests. The full test binaries are still
@@ -78,7 +78,7 @@ jobs:
       - uses: actions/checkout@v7
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
       - name: Test the Windows process helpers
         shell: bash
         # Scoped with -run: the rest of the docker package's tests assert POSIX
@@ -122,7 +122,7 @@ jobs:
           submodules: true
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
       - name: Cache ANTLR4 JAR
         uses: actions/cache@v6
         with:
@@ -256,7 +256,7 @@ jobs:
           submodules: ${{ matrix.suite == 'roundtrip' || matrix.suite == 'parity' }}
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
       - name: Cache ANTLR4 JAR
         uses: actions/cache@v6
         with:

.github/workflows/release.yml

@@ -17,7 +17,7 @@ jobs:
           fetch-depth: 0
       - uses: actions/setup-go@v7
         with:
-          go-version: '1.27.2'
+          go-version: '1.27.3'
       - uses: oven-sh/setup-bun@v2
       - name: Cache ANTLR4 JAR
         uses: actions/cache@v6

docs-site/src/language/widget-types.md

@@ -210,7 +210,12 @@ trailing association with no entity gets the end opposite the context.
 
 It is a different source from `$Task/System.WorkflowUserTask_Assignees` — an
 **association** source, which follows the association in memory and has no
-XPath, sort or search. List views only; on any other widget it is refused.
+XPath, sort or search.
+
+A gallery, a data grid and any other pluggable widget take the same form for
+their "Database" source, with its XPath and sort (`search by` stays a list view
+search bar). Widgets with a source of their own, such as a chart series or a
+dynamic image, refuse it.
 
 #### Specialization templates
 

docs-wiki/bug-patterns/describe-round-trip-gaps.md

@@ -8,6 +8,7 @@ sources:
   - .claude/skills/fix-issue/findings/mdl-executor/
   - mdl/executor/cmd_workflows.go
   - mdl/executor/cmd_pages_describe_pluggable.go
+  - mdl/executor/cmd_pages_describe_bykey.go
   - mdl/executor/cmd_microflows_show_crossed.go
   - mdl/executor/cmd_microflows_normalize.go
   - mdl/executor/describe_graph_invariant_test.go
@@ -153,6 +154,17 @@ The same third state appears without translations: an argument field Studio Pro
 stores as `Argument: ""` is neither the value `empty` nor absence, and a
 describer that conflates it with either cannot round-trip to `Unchanged`.
 
+**"Default" means what exec writes for the key left out, not the schema's
+`DefaultValue`.** A by-key reader prints what differs from the default, and the
+two disagree wherever the embedded template stores something else — a combo
+box's `readOnlyStyle` is `bordered` in the template against a default of
+`text`. Compared with the schema, every mxcli-written widget prints a value
+nobody authored (*invents*), and every Studio Pro one storing the schema default
+prints nothing and replays as the template's value (*silently drops*). A reader
+for a widget with a dedicated describer has to run alongside it, or every key
+outside its keyword set is lost — and what it prints must also pass `check`
+without `-p`, where only the built-in definition knows the widget's keys.
+
 **Fixing one half is worse than the bug.** Where a describer has two defects at
 once — say, quoting *and* a missing property — shipping the quoting fix alone
 turns unparseable output into output that parses cleanly while silently dropping

go.mod

@@ -2,7 +2,7 @@ module github.com/mendixlabs/mxcli
 
 go 1.26.0
 
-toolchain go1.27.2
+toolchain go1.27.3
 
 require (
 	github.com/alecthomas/chroma/v2 v2.26.1

mdl-examples/bug-tests/1520-describe-snippet-root-datasource.mdl

@@ -0,0 +1,28 @@
+-- ============================================================================
+-- Bug test: mendixlabs/mxcli#1520 — DESCRIBE SNIPPET omits the root data
+-- source of a snippet
+-- ============================================================================
+--
+-- A data view inside a snippet, sourced from a snippet parameter, stores the
+-- parameter in Forms$PageVariable.SnippetParameter with PageParameter empty.
+-- DESCRIBE read PageParameter alone and printed the data view with no
+-- DataSource, so exec of the described text failed with CE0488 (fixed with
+-- ako/mxcli#721 L3; this script keeps the shape the report names).
+--
+-- To verify:
+--   1. mxcli exec this script, then: describe snippet Bug1520S.S
+--      -> dataview dv (DataSource: $View, …)
+--   2. exec that output -> "Unchanged snippet Bug1520S.S".
+-- ============================================================================
+
+mdl 1;
+
+create module Bug1520S;
+
+create persistent entity Bug1520S.Rec (Label: String(50));
+
+create snippet Bug1520S.S (Params: ($View: Bug1520S.Rec)) {
+  dataview dv (DataSource: $View) {
+    textbox tb (Attribute: Label)
+  }
+};

mdl-examples/bug-tests/1586-describe-combobox-filter-expression-props.mdl

@@ -0,0 +1,54 @@
+-- ============================================================================
+-- Bug test: mendixlabs/mxcli#1586 — DESCRIBE PAGE omits expression-type
+-- pluggable widget properties
+-- ============================================================================
+--
+-- A combo box and the data grid filters have their own DESCRIBE readers, which
+-- knew only the keys MDL has keywords for. Every other setting — written by
+-- exec's explicit-property pass — described back as nothing: a combo box's
+-- `customEditabilityExpression`, a text filter's `defaultValue` expression and
+-- `delay`. A describe -> edit -> exec cycle reset them to their defaults.
+--
+-- (A generic pluggable widget's top-level expression, the report's own
+-- `entitypicker … Placeholder:`, is covered by mendixlabs/mxcli#1578.)
+--
+-- To verify:
+--   1. mxcli exec this script, then: describe page Bug1586.Edit and
+--      describe page Bug1586.Grid
+--      -> combobox cb (…, customEditability: 'conditionally',
+--                      customEditabilityExpression: $currentObject/Locked)
+--      -> textfilter tf (defaultValue: '''abc''', delay: 300)
+--   2. exec each output -> "Unchanged page".
+--   3. mx check: 0 errors.
+-- ============================================================================
+
+mdl 1;
+
+create module Bug1586;
+
+create persistent entity Bug1586.Parent (Name: String(50));
+create persistent entity Bug1586.Rec (Code: String(20), Amount: Integer, Locked: Boolean);
+create association Bug1586.Rec_Parent from Bug1586.Rec to Bug1586.Parent;
+
+create page Bug1586.Edit (Title: 'Edit', Layout: Atlas_Core.Atlas_Default, Params: ($Rec: Bug1586.Rec)) {
+  dataview dv (DataSource: $Rec) {
+    combobox cb (
+      Attribute: Bug1586.Rec_Parent,
+      DataSource: database from Bug1586.Parent,
+      CaptionAttribute: Name,
+      customEditability: conditionally,
+      customEditabilityExpression: '$currentObject/Locked'
+    )
+  }
+};
+
+create page Bug1586.Grid (Title: 'Grid', Layout: Atlas_Core.Atlas_Default) {
+  datagrid dg (DataSource: database from Bug1586.Rec) {
+    column (Attribute: Code, Caption: 'Code') {
+      textfilter tf (defaultValue: '''abc''', delay: 300)
+    }
+    column (Attribute: Amount, Caption: 'Amount') {
+      numberfilter nf (defaultValue: 5)
+    }
+  }
+};

mdl-examples/bug-tests/1599-describe-gallery-sort-over-association.mdl

@@ -0,0 +1,41 @@
+-- ============================================================================
+-- Bug test: mendixlabs/mxcli#1599 — DESCRIBE PAGE drops a gallery's sort when
+-- its "Database" source runs over an association
+-- ============================================================================
+--
+-- Studio Pro stores a gallery bound to "Database" over an association from its
+-- context object as a CustomWidgets$CustomWidgetXPathSource whose EntityRef is
+-- an IndirectEntityRef, with its XPath and sort bar beside it. DESCRIBE read it
+-- as the association source `$currentObject/Assoc`, without the sort; replayed,
+-- that is a Forms$AssociationSource — a different retrieve. MDL could not
+-- author the shape at all: `create page` refused `database from
+-- $currentObject/…` on a gallery.
+--
+-- To verify:
+--   1. mxcli exec this script, then: describe page Bug1599.P1
+--      -> gallery g1 (DataSource: database from
+--           $currentObject/Bug1599.Item_Parent/Bug1599.Item sort by Code asc, …)
+--   2. exec that output -> "Unchanged page Bug1599.P1".
+--   3. mx check: 0 errors (measured on 11.12.1).
+-- ============================================================================
+
+mdl 1;
+
+create module Bug1599;
+
+create persistent entity Bug1599.Parent (Name: String(50));
+create persistent entity Bug1599.Item (Code: String(20), Amount: Integer);
+create association Bug1599.Item_Parent from Bug1599.Item to Bug1599.Parent;
+
+create page Bug1599.P1 (Title: 'P', Layout: Atlas_Core.Atlas_Default) {
+  listview lv (DataSource: database from Bug1599.Parent) {
+    gallery g1 (
+      DataSource: database from $currentObject/Bug1599.Item_Parent/Bug1599.Item where [Amount > 0] sort by Code asc,
+      Selection: None
+    ) {
+      template {
+        dynamictext x (Content: '{1}', ContentParams: ({1} = Code))
+      }
+    }
+  }
+};

mdl/backend/modelsdk/cw_xpath_source_over_association_test.go

@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package modelsdkbackend
+
+import (
+	"testing"
+
+	"go.mongodb.org/mongo-driver/v2/bson"
+
+	"github.com/mendixlabs/mxcli/modelsdk/codec"
+	"github.com/mendixlabs/mxcli/sdk/pages"
+)
+
+// mendixlabs/mxcli#1599. On 10+ a gallery's "Database" source reached over an
+// association is a CustomWidgets$CustomWidgetXPathSource whose EntityRef is an
+// IndirectEntityRef of the hops, with its sort bar beside it — the shape the
+// report's bson dump shows. The writer stored only a DirectEntityRef, so the
+// hops (and with them the context restriction) could not be written at all.
+func TestCustomWidgetXPathSourceToGen_OverAssociation(t *testing.T) {
+	el, err := customWidgetDataSourceToGen(&pages.DatabaseSource{
+		EntityName: "Probe.Item",
+		EntitySteps: []pages.AttributeRefStep{{
+			Association: "Probe.Item_Parent", DestinationEntity: "Probe.Item",
+		}},
+		ContextVariable: "Parent",
+		Sorting: []*pages.GridSort{{
+			AttributePath: "Probe.Item.Code", Direction: "Ascending",
+		}},
+	})
+	if err != nil {
+		t.Fatalf("customWidgetDataSourceToGen: %v", err)
+	}
+	raw, err := (&codec.Encoder{}).Encode(el)
+	if err != nil {
+		t.Fatalf("encode: %v", err)
+	}
+	var d bson.D
+	if err := bson.Unmarshal(raw, &d); err != nil {
+		t.Fatalf("unmarshal: %v", err)
+	}
+	if got := lookup(d, "$Type"); got != "CustomWidgets$CustomWidgetXPathSource" {
+		t.Fatalf("$Type = %v, want CustomWidgets$CustomWidgetXPathSource", got)
+	}
+	ref, _ := lookup(d, "EntityRef").(bson.D)
+	if lookup(ref, "$Type") != "DomainModels$IndirectEntityRef" {
+		t.Fatalf("EntityRef = %v, want an IndirectEntityRef", ref)
+	}
+	steps, _ := lookup(ref, "Steps").(bson.A)
+	if len(steps) != 2 {
+		t.Fatalf("Steps = %v, want one hop", steps)
+	}
+	step, _ := steps[1].(bson.D)
+	if lookup(step, "Association") != "Probe.Item_Parent" || lookup(step, "DestinationEntity") != "Probe.Item" {
+		t.Errorf("step = %v, want Probe.Item_Parent → Probe.Item", step)
+	}
+	sv, _ := lookup(d, "SourceVariable").(bson.D)
+	if lookup(sv, "PageParameter") != "Parent" {
+		t.Errorf("SourceVariable = %v, want page parameter Parent", sv)
+	}
+	bar, _ := lookup(d, "SortBar").(bson.D)
+	if items, _ := lookup(bar, "SortItems").(bson.A); len(items) != 2 {
+		t.Errorf("SortBar.SortItems = %v, want one sort item", items)
+	}
+}

mdl/backend/modelsdk/widget_write.go

@@ -1850,7 +1850,22 @@ func customWidgetDataSourceToGen(ds pages.DataSource) (element.Element, error) {
 		assignID(src)
 		src.SetForceFullObjects(false)
 		src.SetXPathConstraint(d.XPathConstraint)
-		if d.EntityName != "" {
+		switch {
+		case len(d.EntitySteps) > 0:
+			// "Database" reached over associations from a context object: the
+			// hops as an IndirectEntityRef inside the XPath source, the start
+			// parameter in its SourceVariable — the shape listViewSourceToGen
+			// writes, and the one Studio Pro stores for a gallery or grid
+			// bound this way (mendixlabs/mxcli#1599).
+			src.SetEntityRef(pageEntityRefToGen(d.EntitySteps))
+			if d.ContextVariable != "" {
+				kind := ""
+				if d.IsSnippetParameter {
+					kind = "snippet"
+				}
+				src.SetSourceVariable(sourceVariableToGen(d.ContextVariable, kind))
+			}
+		case d.EntityName != "":
 			ref := genDm.NewDirectEntityRef()
 			assignID(ref)
 			ref.SetEntityQualifiedName(d.EntityName)