fix(check): accept a Data grid 2's own settings without a project

.claude/skills/fix-issue/findings/mdl-executor/2026-10-17-check-without-project-reports-datagrid-keys-as-dropped.json

@@ -0,0 +1 @@
+{"area": "mdl/executor", "date": "2026-10-17", "symptom": "`mxcli check pers.mdl` without `-p` warned `widget dgAttr (datagrid) property configurationAttribute is not recognized and will be silently dropped on write [MDL-WIDGET07]` (and the same for configurationStorageType, storeFiltersInPersonalization, refreshInterval, columnsHidable, showEmptyPlaceholder, a column's minWidth/minWidthLimit/allowEventPropagation/exportValue), although `exec -p` stores each of them and `check -p` accepts them (mendixlabs/mxcli#1573). An Expression-typed key (rowClass, a column's columnClass) was worse: MDL-WIDGET42 ERROR \"a plain-value property cannot hold the expression\", so a script `exec` takes failed `check` outright, and so did `describe` output carrying one.", "cause": "DATAGRID has no embedded definition (sdk/widgets/definitions holds combobox, gallery, the filters, image): its .def.json is generated from the project's own package, and an embedded one would take precedence over that package for every write because RefreshWidgetDefinitions skips an id the registry already holds. With no project the keyword therefore resolved to no definition, and validateWidgetTreeIn fell through to validateStaticWidgetUnknownProps, whose vocabulary is the union of what the built-in widgets' builders read and describe prints; validateWidgetExpressionValues, given no definition and no item mapping, called every key a plain-value property.", "fix": "validate_widget_embedded_schema.go: for a keyword widget the registry has no definition for (DATAGRID, and its `column`), read the property list of the template mxcli embeds for it (widgets.GetTemplate; the grid's own keys, the `columns` item type for a column). A key the template declares (or the `<key>Params` of a text template) is accepted by MDL-WIDGET07, and its declared type decides MDL-WIDGET42 (Expression takes an expression). A key it does not declare still draws MDL-WIDGET07 with a did-you-mean from the template's names, now worded as what is true: it cannot be verified without a project. Nothing changes with a project, or for a built-in widget.", "file": "mdl/executor/validate_widget_embedded_schema.go", "test": "mdl/executor/validate_widget_embedded_schema_test.go (every key the template declares, derived from it); mdl-examples/bug-tests/1573-datagrid-by-key-settings-check-no-project.mdl", "insight": "`make check-mdl` runs `check` with no project, so the symptom is invisible to the usual loop (`check -p`) and shows only when a script is checked in a repo or by a tool without a project; the reproduction needs the pair `check` vs `check -p` on one file. The embedded template is the 11.6 baseline, kept conservative on purpose (widget-engine.md), so it is evidence of what a key IS and never of what it is not: the Data grid 2 template carries the same 40 grid keys and 21 column keys as the 11.12.5 package (3.4.0), but a newer widget version can add more, which is why an undeclared key is reported as unverifiable rather than as dropped, and why the misspelled-key case keeps its warning. Adding an embedded datagrid.def.json looks like the obvious fix and is the wrong one: it would change every write, not just the check."}

CHANGELOG.md

@@ -6,6 +6,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Fixed
+
+- **`check` without a project no longer reports a Data grid 2's own settings as dropped, or refuses an expression on one** (mendixlabs/mxcli#1573) — `exec -p` stores `configurationStorageType`, `configurationAttribute`, `storeFiltersInPersonalization`, `refreshInterval`, `columnsHidable`, `rowClass`, a column's `minWidth` and the rest of the widget's own keys, and `check -p` accepts them, but `check` alone warned MDL-WIDGET07 "not recognized and will be silently dropped on write" for each, and refused an Expression-typed one (`rowClass`, a column's `columnClass`) as an MDL-WIDGET42 error, so a script `exec` takes failed `make check-mdl`. Data grid 2 has no embedded definition (it is generated from the project's package, and an embedded one would take precedence over that package on every write), so the check judged its keys against the built-in widgets' vocabulary. It now reads the property list of the Data grid 2 template mxcli embeds: a key the template declares is accepted and its declared type decides MDL-WIDGET42. A key it does not declare still warns — a misspelled `configurationStorageTyp` too, with the spelling it probably meant — but says what is true without a project: that the key cannot be verified, and is dropped on write unless the installed widget declares it. With `-p`, and for every other widget, nothing changes.
+
 ## [0.27.0] - 2026-10-15
 
 Headline: **`exec` is all or nothing: when a statement fails at write time, the whole run is rolled back, so a script never leaves a project half-changed.** Alongside it: a wave of `describe` → `check` → `exec` fixes from round-trip runs over real apps (the Siemens FSM template, the GenAI showcase, Marketplace modules), Mendix 9 projects can take microflows, nanoflows and pluggable widgets again, `check -p` resolves many more of the references `exec` and the build would refuse, the classic `filemanager` widget and role-based `Visible: roles (…)` are in MDL, and the companion draws a module's domain model. The MongoDB driver is upgraded for the October 2026 BSON advisories.

docs-site/src/internals/widget-engine.md

@@ -228,6 +228,8 @@ Definitions are loaded in priority order. Higher-priority definitions override l
 
 Definitions are looked up by MDL name (case-insensitive): `COMBOBOX`, `GALLERY`, or any custom name.
 
+**DATAGRID has no embedded definition, on purpose.** Its `.def.json` is generated from the project's own package, and an embedded one would sit ahead of that package for every write (`RefreshWidgetDefinitions` skips a widget id the registry already holds). So without a project the registry has no definition for it, and `mxcli check` reads the property list of the template mxcli embeds for the widget (`sdk/widgets/templates/`, via `embeddedSchemaFor`): a key the template declares is accepted and its declared type decides whether an expression is allowed (MDL-WIDGET42). The template is the oldest-supported baseline, so a key it does not declare is reported as one that cannot be verified without a project (MDL-WIDGET07), not as one that is dropped.
+
 ### Validation at Load Time
 
 When a definition is loaded, it is validated:

mdl-examples/bug-tests/1573-datagrid-by-key-settings-check-no-project.mdl

@@ -0,0 +1,83 @@
+mdl 1;
+-- ============================================================================
+-- Bug: `mxcli check` without a project called a Data grid 2's own settings
+-- "not recognized and silently dropped" (mendixlabs/mxcli#1573)
+-- ============================================================================
+--
+-- A Data grid 2 takes every one of its settings by the widget's own property
+-- key. `exec -p` stores each of these and `check -p` accepts them. `check`
+-- without a project — which is what `make check-mdl` runs on this file — warned
+-- MDL-WIDGET07 for each of them ("property `configurationAttribute` is not
+-- recognized and will be silently dropped on write"), and REFUSED the two
+-- Expression-typed ones as MDL-WIDGET42 ("a plain-value property cannot hold
+-- the expression"), which makes the file fail `check` outright.
+--
+-- Without a project DATAGRID has no definition (it is generated from the
+-- project's own package), so the check judged its keys against the vocabulary
+-- of the built-in widgets. It now reads the property list of the Data grid 2
+-- template mxcli embeds: a key that template declares is accepted, and an
+-- Expression-typed one takes an expression. A key it does not declare is still
+-- reported, now as one that cannot be verified without a project — the
+-- template is a baseline, not the installed widget.
+--
+-- This file must pass `mxcli check` with no MDL-WIDGET07 and no MDL-WIDGET42.
+-- The negative case (a misspelled key still warns) is a Go test:
+-- mdl/executor/validate_widget_embedded_schema_test.go.
+-- ============================================================================
+
+CREATE MODULE BugDgCheck;
+
+CREATE OR MODIFY PERSISTENT ENTITY BugDgCheck.Ord (
+  Code: String(20),
+  Amount: Decimal,
+  Open: Boolean
+);
+
+CREATE OR MODIFY PERSISTENT ENTITY BugDgCheck.GridPref (
+  Config: String(unlimited)
+);
+
+-- Keyword settings and settings by key, grid and column.
+CREATE PAGE BugDgCheck.Orders_Settings (Title: 'Settings', Layout: Atlas_Core.Atlas_Default) {
+  datagrid dgSettings (
+    DataSource: database BugDgCheck.Ord,
+    PageSize: 25,
+    Pagination: virtualScrolling,
+    Selection: Single,
+    refreshInterval: 30,
+    columnsHidable: false,
+    showEmptyPlaceholder: custom,
+    rowClass: if $currentObject/Open then 'row-open' else 'row-closed',
+    exportDialogLabel: 'Exporting'
+  ) {
+    column (Attribute: Code, Caption: 'Code', Sortable: true, columnClass: if $currentObject/Open then 'is-open' else '') { textfilter tf }
+    column (Attribute: Amount, ShowContentAs: customContent, ColumnWidth: autoFit, minWidth: manual, minWidthLimit: 150, allowEventPropagation: false) {
+      dynamictext amt (Content: 'amt')
+    }
+  }
+};
+
+-- Personalization kept in the browser.
+CREATE PAGE BugDgCheck.Orders_Local (Title: 'Local', Layout: Atlas_Core.Atlas_Default) {
+  datagrid dgLocal (
+    DataSource: database BugDgCheck.Ord,
+    configurationStorageType: localStorage,
+    storeFiltersInPersonalization: true
+  ) {
+    column (Attribute: Code) { textfilter tf }
+  }
+};
+
+-- Personalization kept in an attribute of the surrounding object.
+CREATE PAGE BugDgCheck.Orders_Attribute (Title: 'Attribute', Layout: Atlas_Core.Atlas_Default, Params: ( $Pref: BugDgCheck.GridPref )) {
+  dataview dvPref (DataSource: $Pref) {
+    datagrid dgAttr (
+      DataSource: database BugDgCheck.Ord,
+      configurationStorageType: attribute,
+      configurationAttribute: Config,
+      storeFiltersInPersonalization: true
+    ) {
+      column (Attribute: Code) { textfilter tf }
+    }
+  }
+};

mdl/executor/validate_widget_embedded_schema.go

@@ -0,0 +1,184 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package executor
+
+import (
+	"sort"
+	"strings"
+	"sync"
+
+	"github.com/mendixlabs/mxcli/mdl/ast"
+	"github.com/mendixlabs/mxcli/sdk/widgets"
+)
+
+// What a check without a project knows about Data grid 2.
+//
+// A widget's properties come from its package (.mpk), which only a project
+// has. The widgets mxcli embeds a definition for (combobox, gallery, the grid
+// filters, image) are known without one. DATAGRID is not among them: its
+// definition is generated from the project's own package, and embedding one
+// would put it ahead of that package for every write
+// (RefreshWidgetDefinitions skips an id the registry already holds).
+//
+// So without a project the widget had no definition at all, and the check fell
+// back on the vocabulary of the built-in widgets — a union of what their
+// builders read and what describe prints. Every Data grid 2 setting outside it
+// was reported as "not recognized and will be silently dropped on write"
+// (MDL-WIDGET07) although exec writes it, and an Expression-typed one was an
+// ERROR (MDL-WIDGET42, "a plain-value property cannot hold the expression").
+// `mxcli check` of a script `exec` takes, and `describe` prints, failed.
+//
+// What mxcli does have is the template it embeds for the widget, which declares
+// every property. That is a baseline (the oldest supported Mendix version, see
+// widget-engine.md), so it is evidence of what a key IS, never of what it is
+// not: a key it declares is accepted, a key it does not is reported as one that
+// cannot be verified without a project.
+
+// embeddedWidgetSchema is the property list of a widget template mxcli embeds.
+type embeddedWidgetSchema struct {
+	// Title and Version are the template's: "Data grid 2", "11.6.3".
+	Title   string
+	Version string
+	// Names are the declared property keys, in declaration order.
+	Names []string
+	// types maps a lower-cased key to its declared type ("Expression",
+	// "TextTemplate", "Boolean", …).
+	types map[string]string
+}
+
+// declares reports whether the template declares key, case-insensitively — or
+// the `<key>Params` companion of a text-template property, which carries its
+// `{N}` parameters.
+func (s *embeddedWidgetSchema) declares(key string) bool {
+	if s == nil {
+		return false
+	}
+	lower := strings.ToLower(key)
+	if _, ok := s.types[lower]; ok {
+		return true
+	}
+	if base, found := strings.CutSuffix(lower, "params"); found {
+		return s.types[base] == "TextTemplate"
+	}
+	return false
+}
+
+// operation is the engine operation that writes key ("expression",
+// "texttemplate", "attribute", "primitive"), or "" for a kind with none.
+func (s *embeddedWidgetSchema) operation(key string) (string, bool) {
+	if s == nil {
+		return "", false
+	}
+	t, ok := s.types[strings.ToLower(key)]
+	if !ok {
+		return "", false
+	}
+	return operationForValueType(t), true
+}
+
+// pluggableWidgetIDOfKeyword is the id of the pluggable widget an MDL keyword
+// names, whatever the Mendix version (DATAGRID → Data grid 2), or "".
+func pluggableWidgetIDOfKeyword(keyword string) string {
+	upper := strings.ToUpper(keyword)
+	for _, m := range keywordDispatchTable {
+		if m.Keyword != upper {
+			continue
+		}
+		for _, b := range m.Bindings {
+			if b.Kind == bindingKindPluggable && b.WidgetID != "" {
+				return b.WidgetID
+			}
+		}
+	}
+	return ""
+}
+
+var embeddedSchemas sync.Map // widgetID + "/" + listKey -> *embeddedWidgetSchema (nil when there is none)
+
+// embeddedSchemaFor returns the embedded schema for the keyword widget w —
+// the grid's own properties for `datagrid`, a column's for a `column` inside
+// one — or nil when w is not a keyword widget the registry lacks a definition
+// for (everything else is judged exactly as before).
+func embeddedSchemaFor(w, parent *ast.WidgetV3) *embeddedWidgetSchema {
+	if w == nil || w.TypeIsGeneric || explicitWidgetID(w) != "" {
+		return nil
+	}
+	if id := pluggableWidgetIDOfKeyword(w.Type); id != "" {
+		return loadEmbeddedSchema(id, "")
+	}
+	if strings.EqualFold(w.Type, "column") && parent != nil && !parent.TypeIsGeneric && explicitWidgetID(parent) == "" {
+		if id := pluggableWidgetIDOfKeyword(parent.Type); id != "" {
+			return loadEmbeddedSchema(id, "columns")
+		}
+	}
+	return nil
+}
+
+// loadEmbeddedSchema reads the property list of the embedded template for
+// widgetID — or, with listKey, the item type of that object-list property.
+func loadEmbeddedSchema(widgetID, listKey string) *embeddedWidgetSchema {
+	cacheKey := widgetID + "/" + listKey
+	if v, ok := embeddedSchemas.Load(cacheKey); ok {
+		s, _ := v.(*embeddedWidgetSchema)
+		return s
+	}
+	var schema *embeddedWidgetSchema
+	if tmpl, err := widgets.GetTemplate(widgetID); err == nil && tmpl != nil {
+		if propTypes := embeddedTemplatePropertyTypes(tmpl.Type, listKey); len(propTypes) > 0 {
+			schema = &embeddedWidgetSchema{Title: tmpl.Name, Version: tmpl.Version, types: map[string]string{}}
+			if listKey != "" {
+				schema.Title += " column"
+			}
+			for _, pt := range propTypes {
+				key, _ := pt["PropertyKey"].(string)
+				if key == "" {
+					continue
+				}
+				kind := ""
+				if vt, ok := pt["ValueType"].(map[string]any); ok {
+					kind, _ = vt["Type"].(string)
+				}
+				schema.Names = append(schema.Names, key)
+				schema.types[strings.ToLower(key)] = kind
+			}
+			sort.Strings(schema.Names)
+		}
+	}
+	embeddedSchemas.Store(cacheKey, schema)
+	return schema
+}
+
+// embeddedTemplatePropertyTypes lists the property types of a template's widget type —
+// or of the item type of its object-list property listKey.
+func embeddedTemplatePropertyTypes(widgetType map[string]any, listKey string) []map[string]any {
+	objType, _ := widgetType["ObjectType"].(map[string]any)
+	list := embeddedPropertyTypeMaps(objType)
+	if listKey == "" {
+		return list
+	}
+	for _, pt := range list {
+		if key, _ := pt["PropertyKey"].(string); key != listKey {
+			continue
+		}
+		vt, _ := pt["ValueType"].(map[string]any)
+		itemType, _ := vt["ObjectType"].(map[string]any)
+		return embeddedPropertyTypeMaps(itemType)
+	}
+	return nil
+}
+
+// embeddedPropertyTypeMaps is an object type's PropertyTypes as maps; the typed-array
+// marker that opens the array is not one.
+func embeddedPropertyTypeMaps(objType map[string]any) []map[string]any {
+	var out []map[string]any
+	if objType == nil {
+		return nil
+	}
+	arr, _ := objType["PropertyTypes"].([]any)
+	for _, e := range arr {
+		if m, ok := e.(map[string]any); ok {
+			out = append(out, m)
+		}
+	}
+	return out
+}