From 9775527df6928e05aaefabfa101ad3d7760995a1 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 7 Aug 2026 21:29:54 +0000 Subject: [PATCH 01/31] feat(test): implement @cleanup rollback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The @cleanup annotation has documented rollback as its default since the runner shipped, but TestCase.Cleanup was parsed and then used nowhere: every test committed. The after-startup runner had no seam to implement it — tests execute inside the startup action, so there is no context the runner owns. The test endpoint creates that seam, because it builds the IContext each test runs on. The handler now wraps the call in startTransaction()/rollbackTransaction() when the runner asks for it, in a finally so a throwing test — the one most likely to leave half-written data — is rolled back too. @cleanup none commits, for when the writes are the point. Verified against Postgres rather than the endpoint's own claim: a suite with one rollback test and one @cleanup none test, run against an emptied table, leaves exactly the "none" row behind. Same microflow, same run, only the annotation differs. Two failure modes this closes rather than opens: - An unknown strategy (@cleanup rollbak) is now a parse error. Treating it as "not rollback" would leave the data behind while the run still reported a clean pass. Rejected at parse time, so --list catches it and no runtime is booted for a file that cannot run correctly. The .mdl and .md parsers are separate code paths and both are covered — the first version of this only reached one of them. - A rollback that fails is reported per test and summarised at the end, never swallowed. --verbose tags every result [rolled back] / [committed] / [ROLLBACK FAILED]. An endpoint too old to know the parameter is called out specifically, since --attach can meet one. Rollback applies to --local and --attach; Docker keeps committing, and the docs say so. It matters most under --attach, where the database belongs to the developer's running app. Each new test was verified to fail against a stubbed guard. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .claude/skills/fix-issue.md | 1 + .claude/skills/mendix/test-microflows.md | 42 ++++ CLAUDE.md | 2 +- cmd/mxcli/cmd_test_run.go | 6 + cmd/mxcli/syntax/features_misc.go | 8 +- cmd/mxcli/testrunner/cleanup_strategy.go | 89 +++++++ cmd/mxcli/testrunner/cleanup_strategy_test.go | 217 ++++++++++++++++++ cmd/mxcli/testrunner/client.go | 19 +- cmd/mxcli/testrunner/client_test.go | 29 +++ cmd/mxcli/testrunner/endpoint.go | 31 +++ cmd/mxcli/testrunner/parser.go | 8 + cmd/mxcli/testrunner/runner_endpoint.go | 22 +- docs-site/src/tools/running-tests.md | 37 +++ .../doctype-tests/cleanup-rollback.test.mdl | 56 +++++ 14 files changed, 560 insertions(+), 7 deletions(-) create mode 100644 cmd/mxcli/testrunner/cleanup_strategy.go create mode 100644 cmd/mxcli/testrunner/cleanup_strategy_test.go create mode 100644 mdl-examples/doctype-tests/cleanup-rollback.test.mdl diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index f03e16d82..5427fbebf 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -401,3 +401,4 @@ extracting `OffsetExpression`/`LimitExpression`. | `ALTER MODULE X ADD JAR DEPENDENCY (…)` succeeds, `list jar dependencies` reports it, the build is **green** — and the runtime throws `SQLException: No JDBC driver found in app for URL`. `deployment/build.gradle` has no dependencies block and `find deployment -iname '**'` returns nothing | Not a bad write. Declaring and resolving are **separate steps**: the model records the coordinate, and `mx sync-java-dependencies ` is what downloads it into `vendorlib/`. Studio Pro runs that when you edit Module Settings; nothing headless was running it. Confirmed on 11.12.1 — a full `mxbuild --target=deploy` resolves nothing, and the sync command then fetches the jar | `cmd/mxcli/docker/javadeps.go` (`SyncJavaDependencies`, `UnvendoredJarDependencies`), `cmd/mxcli/cmd_sync_java_deps.go` (`mxcli sync-java-deps [--check]`), `cmd/mxcli/docker/runlocal.go` (vendors before boot), `mdl/executor/cmd_modules.go` (`warnUnvendoredJarDependencies`) | **How to find the missing step**: the reporter's open question was "does mxbuild skip Maven resolution, or does mxcli write it somewhere MxBuild cannot read?" — neither. `strings mx.dll | grep -i dependenc` surfaced `ISyncJavaDependenciesRunner`/`SkipManagedDependencySync`, and `mx --help` listed `sync-java-dependencies`. When a model-level write "works" but the artefact never appears, check whether the **toolset** has a separate command for it before suspecting the write. Wired at three levels so the gap cannot stay silent: the executor says so the moment it writes an unvendored coordinate, `run --local` resolves it before boot, and `--check` exits non-zero as a build gate. Resolution needs network, so every call site is best-effort with an actionable message. Tests `cmd/mxcli/docker/javadeps_test.go`. mxcli-formula1 #12 | | `$Total = 5;` does not parse — `no viable alternative at input '$Total=5'` — while `DECLARE $Total Integer = 0;` does, and so do `$X = HEAD($List)`, `$X = create M.E (…)` and `$X = execute database query …`. The error names the token, not the missing keyword | Assignment existed only as a **prefix** on specific activity statements (`(VARIABLE EQUALS)?` on CALL/CREATE/RETRIEVE/…), plus a `SET $Var = expression` statement. A plain value therefore required `SET`, which nothing in the error or the surrounding syntax suggested | `mdl/grammar/domains/MDLMicroflow.g4` (`setStatement : SET? …`), `cmd/mxcli/syntax/features_microflow.go` | Make the guessable form work rather than improve the error: `SET` is now optional and both spellings produce the same `MfSetStmt`/`ChangeVariableAction`. **Prove a grammar relaxation causes no regressions with a control binary, not by reading**: `git stash` the `.g4`, `make grammar`, build `bin/mxcli-control`, sweep every `mdl-examples/**/*.mdl` with both — 13 scripts fail, the *same* 13, all pre-existing. ANTLR's adaptive prediction picks the activity-prefixed alternatives over `setStatement` on its own; no ordering change was needed. Executed against a real .mpr, mxbuild reports 0 errors. Tests `mdl/visitor/visitor_microflow_bare_assign_test.go` (bare and keyword forms must agree on the AST, not merely both parse). mxcli-formula1 #13 | | `mxcli test tests/ -p app/App.mpr` fails with "no such file or directory" for a `tests/` that sits right next to the `.mpr` | Test paths resolved against the process CWD only. Defensible in isolation, but mxcli otherwise encourages naming the project (`-p`) rather than standing in its directory, and project auto-discovery searches outward — so the two conventions collide and the failure looks like a missing directory | `cmd/mxcli/cmd_test_run.go` (`resolveTestPaths`) | Fall back to project-relative **only when the CWD-relative path does not exist**: a `tests/` in both places must resolve to the one the user is standing in, since silently preferring the project's copy would run the wrong suite. A path that exists in neither is passed through unchanged so the error names what was typed, not a rewritten path the user never mentioned. Tests `cmd/mxcli/cmd_test_run_paths_test.go`. mxcli-formula1 #13 | +| A test annotated `@cleanup rollback` (or with no `@cleanup` at all — rollback is the documented default) still leaves its rows in the database; a misspelled strategy like `@cleanup rollbak` does the same, silently, while the run reports PASS | `TestCase.Cleanup` was parsed and then used nowhere. The after-startup runner had no seam to implement it — tests run inside the startup action, so there is no context the runner owns. The test endpoint creates that seam: it builds the `IContext` each test runs on | `cmd/mxcli/testrunner/endpoint.go` (the handler's execute block), `cmd/mxcli/testrunner/cleanup_strategy.go` | Wrap the call in `ctx.startTransaction()` … `ctx.rollbackTransaction()` in a **finally** (a throwing test is the one most likely to leave half-written data), gated on a `rollback=1` query parameter the client sends per test. Report `rolledBack`/`rollbackError` in the response and warn per test — a rollback that fails silently is worse than none. Reject an unknown `@cleanup` value at **parse** time so `--list` catches it too. Verify against the database, not the endpoint's own claim: run one test with rollback and one with `@cleanup none` in the same suite and query Postgres — the `none` row must be the only survivor | diff --git a/.claude/skills/mendix/test-microflows.md b/.claude/skills/mendix/test-microflows.md index 8d1534bab..6133b2c85 100644 --- a/.claude/skills/mendix/test-microflows.md +++ b/.claude/skills/mendix/test-microflows.md @@ -93,6 +93,48 @@ The markdown format turns your tests into living documentation. | `@throws` | Expect error | `@throws 'validation failed'` | | `@cleanup` | Rollback strategy | `@cleanup rollback` (default) or `@cleanup none` | +### `@cleanup` — what happens to a test's data + +**`rollback` is the default**, so by default a test's database writes do not +survive it. The endpoint opens a transaction around the call and rolls it back +afterwards, including when the test throws. + +```mdl +/** + * @test creating an order does not leak + * @expect $result = 'ok' + */ +$result = CALL MICROFLOW Sales.CreateOrder(Amount = 100); +/ + +/** + * @test seed data the next test needs + * @cleanup none + */ +$result = CALL MICROFLOW Sales.SeedCatalogue(); +/ +``` + +Use `@cleanup none` when the writes are the point — seeding a fixture, or +inspecting the result in the running app afterwards. + +Two things worth knowing: + +- **`--local` only.** Rollback needs the test endpoint, which owns the context + the test runs in. The Docker / `--legacy-runner` path executes tests inside + the after-startup action and has no such seam, so it always commits. +- **A rollback that fails is reported, loudly.** The run prints a `WARNING` per + affected test and a summary line, because the alternative — data left behind + while the suite still says PASS — is the failure mode this annotation exists + to prevent. `--verbose` tags every test with `[rolled back]`, `[committed]` or + `[ROLLBACK FAILED]`. + +A misspelled strategy (`@cleanup rollbak`) is a **parse error**, not a silent +fallback to committing. + +Rollback matters most under `--attach`, where the database is the one your dev +app is using. + --- ## Running Tests diff --git a/CLAUDE.md b/CLAUDE.md index 088cd61be..cea1e09ee 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -610,7 +610,7 @@ Full syntax tables for all MDL statements (microflows, pages, security, navigati - LSP server with hover, go-to-definition, completion, diagnostics, symbols, folding - VS Code extension (`vscode-mdl`) with context menu commands (Run/Check/Selection) - Docker build integration (`mxcli docker build`) with PAD patching (Phase 1) -- Warm test loop (`mxcli test --local [--watch]`, `--attach`, `run --local --test-endpoint`): local test runs go through a **token-guarded HTTP endpoint** registered by a generated Java custom request handler, instead of compiling the suite into the project's after-startup microflow. One microflow per test, resolved by name at request time from `Core.getMicroflowNames()` and invoked with `Core.microflowCall(...).execute(...)` — so a throwing test fails only itself (not the boot), results are returned rather than scraped from the runtime log, and each test has its own variable scope. The handler **survives `reload_model`** (after-startup does not re-run, the JVM is unchanged), which is what makes `--watch` possible: ~30s first run, then ~2s from an edit — to a test *or* to the microflow under test — to a verdict. `--attach` skips even that boot by running against an app already up under `run --local --test-endpoint`, driving that process's serve + admin APIs over loopback; it uses **that app's database**, only ever adds/removes its own test microflows, and refuses a change needing a restart. Security: the handler is **not registered at all** without `MXCLI_TEST_TOKEN` in the runtime env (so a project that kept the `MxTest` module through a failed cleanup is inert in production), the token is constant-time compared, non-loopback callers are refused, `/list` is clamped to the test namespace, and only `MxTest.Test_*` may be invoked. The token reaches the runtime via its environment and is never written into the project. Docker keeps the after-startup runner (`--legacy-runner` selects it locally). Packages: `cmd/mxcli/testrunner/` (`endpoint.go`, `client.go`, `watch.go`, `host.go`, `handshake.go`). See `docs/15-testing/SPIKE_test_endpoint_request_handler.md` +- Warm test loop (`mxcli test --local [--watch]`, `--attach`, `run --local --test-endpoint`): local test runs go through a **token-guarded HTTP endpoint** registered by a generated Java custom request handler, instead of compiling the suite into the project's after-startup microflow. One microflow per test, resolved by name at request time from `Core.getMicroflowNames()` and invoked with `Core.microflowCall(...).execute(...)` — so a throwing test fails only itself (not the boot), results are returned rather than scraped from the runtime log, and each test has its own variable scope. Owning the `IContext` is also what finally makes **`@cleanup rollback`** (the annotation's documented default, previously parsed and ignored) real: the handler wraps the call in `startTransaction()`/`rollbackTransaction()`, so a test's writes do not survive it — verified against Postgres, with `@cleanup none` as the in-run control. A rollback that fails is reported per test and summarised, never silent; an unknown strategy is a parse error. The handler **survives `reload_model`** (after-startup does not re-run, the JVM is unchanged), which is what makes `--watch` possible: ~30s first run, then ~2s from an edit — to a test *or* to the microflow under test — to a verdict. `--attach` skips even that boot by running against an app already up under `run --local --test-endpoint`, driving that process's serve + admin APIs over loopback; it uses **that app's database**, only ever adds/removes its own test microflows, and refuses a change needing a restart. Security: the handler is **not registered at all** without `MXCLI_TEST_TOKEN` in the runtime env (so a project that kept the `MxTest` module through a failed cleanup is inert in production), the token is constant-time compared, non-loopback callers are refused, `/list` is clamped to the test namespace, and only `MxTest.Test_*` may be invoked. The token reaches the runtime via its environment and is never written into the project. Docker keeps the after-startup runner (`--legacy-runner` selects it locally). Packages: `cmd/mxcli/testrunner/` (`endpoint.go`, `client.go`, `watch.go`, `host.go`, `handshake.go`). See `docs/15-testing/SPIKE_test_endpoint_request_handler.md` - Warm local dev loop (`mxcli run --local [--watch] [--screenshot]`): Docker-free `mxbuild --serve` + standalone runtime, hot `reload_model` for behavioural changes and restart+DDL for structural ones (chosen from the serve build's `restartRequired`). Bundles the browser client (`web/dist/` via mxbuild's rollup runner, which the serve Deploy target skips) so Mendix 11.x apps render in a browser. `--watch` keeps an incremental rollup bundler hot (CHOKIDAR_USEPOLLING for container fs; ~3-4s page re-bundle, skipped for model-only edits) and watches only model source (`.mpr`+`mprcontents/`). `--ensure-db` provisions the local Postgres + app database if missing; `--setup` does the non-blocking prerequisites (cache mxbuild+runtime, ensure DB) and exits — `mxcli init` wires it into a Claude Code SessionStart hook so a fresh/reaped web session self-bootstraps, and `docs-site/src/tools/bootstrap-prompt.md` is the empty-repo seed prompt. `--screenshot` captures a Playwright PNG each change (pixel-perfect page loop), with `--screenshot-url` deep links (repeatable for multi-page sets, one PNG per page) and `--screenshot-user`/`--screenshot-password` form login (session saved as Playwright storage state, reused via `screenshot --load-storage`). See `docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md` - External browser preview (`mxcli run --hub ` + `mxcli tunnel-hub`): the app stays local and reverse-tunnels out over a single 443 connection (embedded chisel) to a static relay, so it is reachable in a browser at a public URL — works from egress-only environments (Claude Code web), verified live through the session's MITM egress proxy. `run --hub` implies `--local`, boots the runtime with `ApplicationRootUrl` set to the assigned URL (so the SPA/`originURI` work under the public origin), resolves the control proxy honouring `NO_PROXY`, and retries forever. `mxcli tunnel-hub --domain ` is the **multi-tenant** relay: a registry keyed by prefix/project/solution/branch/worktree (stable URLs on reconnect) fronts many previews at per-subdomain hosts (`[prefix-]project[-branch].`; main collapses to the project) over one 443 with per-subdomain autocert, a registration API (`/api/register|status|deregister|backends|sessions`), and an availability overview at `hub./` **grouped by Claude Code session** (`/api/sessions`): each session lists the endpoints it exposed and links back to its `claude.ai/code` conversation. Client identity flags: `--hub-prefix`/`--hub-project`/`--hub-solution`/`--hub-branch`/`--hub-worktree` (project + branch auto-detected); `--hub-session` groups a session's endpoints (auto-detected from `CLAUDE_CODE_REMOTE_SESSION_ID`). Past sessions are retained: a durable per-session endpoint history (`--sessions-file`, default `~/.mxcli/hub-sessions.json`) survives restarts and reaping, and is pruned after `--session-retention` (default 30d) — so the overview shows offline sessions too (`SessionLog` in `cmd/mxcli/tunnelhub/sessions.go`). Package: `cmd/mxcli/tunnelhub/`. See `docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md` (slices 3–4) - Tunnel-hub GitHub authentication (opt-in, gated on `--github-oauth-client-id`; absent = today's open hub): **viewer plane** — GitHub OAuth web flow + HMAC-signed SSO session cookie (`Domain=.`), owner-checked previews (`--require-auth` default on → 302 to login / 403 non-owner; soft mode filters the listing only), `/api/backends` filtered to the viewer (unauthenticated → 401), admin "signed in as" via `/api/whoami`. **Registration plane** — durable, hashed hub API keys (`--keys-file`, default `~/.mxcli/hub-keys.json`, survive restarts) presented as `X-Hub-Key` → stamps `Backend.Owner`; shared `X-Hub-Secret` still works as an owner-less fallback. **Key issuance** — the hub's `/cli` browser page mints a key from the session cookie (no PAT; the device flow was removed as Claude Code containers block GitHub's device endpoints), rotate-by-default + count + revoke-all; `mxcli auth hub login --token ` is the headless path; `run --hub` reads `MXCLI_HUB_KEY` (env → `~/.mxcli/auth.json`) and degrades to local-only if registration fails. Append-only JSONL audit trail (`--audit-log`, no secrets). Packages: `cmd/mxcli/tunnelhub/` (+`audit/`), `cmd/mxcli/hubauth/`. See `docs/11-proposals/PROPOSAL_hub_authentication.md` diff --git a/cmd/mxcli/cmd_test_run.go b/cmd/mxcli/cmd_test_run.go index 56f059b1c..f5e5eb45a 100644 --- a/cmd/mxcli/cmd_test_run.go +++ b/cmd/mxcli/cmd_test_run.go @@ -46,6 +46,12 @@ Because each test is its own microflow invoked on its own, a test that throws fails only itself instead of ending the run, and results are returned rather than recovered from the runtime log. +It also makes @cleanup real. By default (@cleanup rollback) each test runs in a +transaction the endpoint rolls back afterwards, so its database writes do not +survive — use @cleanup none when the writes are the point. The Docker path +always commits: it runs tests inside the after-startup action and has no +context of its own to roll back. + The endpoint is only reachable from loopback, only with a per-run token passed to the runtime through its environment (never written into your project), and will only ever invoke the generated MxTest.Test_* microflows. With no token in diff --git a/cmd/mxcli/syntax/features_misc.go b/cmd/mxcli/syntax/features_misc.go index 85560be1e..fd5bd761c 100644 --- a/cmd/mxcli/syntax/features_misc.go +++ b/cmd/mxcli/syntax/features_misc.go @@ -292,7 +292,13 @@ Annotations: @expect $var = value Assert variable equals value @expect $obj/Attr = val Assert entity attribute @throws 'message' Expect error - @cleanup rollback|none Cleanup strategy (default: rollback) + @cleanup rollback|none What happens to the test's database writes. + rollback (the default) wraps the test in a + transaction and rolls it back, so nothing it + wrote survives — including when it throws. + none lets the writes commit. --local only: + the Docker path always commits. An unknown + value is a parse error, not a silent commit. How --local runs tests: one microflow per test, invoked by name over a token-guarded HTTP endpoint the app registers at boot. A test that throws diff --git a/cmd/mxcli/testrunner/cleanup_strategy.go b/cmd/mxcli/testrunner/cleanup_strategy.go new file mode 100644 index 000000000..f3c981995 --- /dev/null +++ b/cmd/mxcli/testrunner/cleanup_strategy.go @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testrunner + +import ( + "fmt" + "io" + "sort" + "strings" +) + +// Cleanup strategies for the @cleanup annotation. +// +// Rollback is the default and always was — the annotation has documented it +// since the runner shipped. It only became real with the test endpoint: the +// endpoint owns the context each test runs in, so it can open a transaction +// around the call and roll it back. The after-startup runner has no such seam, +// which is why the annotation sat parsed-but-unused. +const ( + // CleanupRollback wraps the test in a transaction and rolls it back, so its + // database writes do not survive. The default. + CleanupRollback = "rollback" + // CleanupNone lets the test's writes commit and persist. + CleanupNone = "none" +) + +// cleanupStrategies is the set of accepted @cleanup values. +var cleanupStrategies = map[string]string{ + CleanupRollback: "wrap the test in a transaction and roll it back (default)", + CleanupNone: "let the test's writes commit and persist", +} + +// validateCleanup rejects an unrecognised @cleanup value. +// +// Silently treating a typo as "not rollback" is the worst outcome available: +// `@cleanup rollbak` would leave the test's data in the database while the run +// still reported a clean pass, and nothing anywhere would say why. An unknown +// value is a mistake in the test file, so it is an error. +func validateCleanup(value string) error { + if value == "" || cleanupStrategies[value] != "" { + return nil + } + valid := make([]string, 0, len(cleanupStrategies)) + for k := range cleanupStrategies { + valid = append(valid, k) + } + sort.Strings(valid) + return fmt.Errorf("unknown @cleanup strategy %q (expected one of: %s)", value, strings.Join(valid, ", ")) +} + +// rollsBack reports whether a test's writes should be rolled back. +// +// An empty strategy means the annotation was absent, which is the default — +// rollback. Anything unrecognised has already been rejected by validateCleanup, +// so this never has to guess. +func rollsBack(tc TestCase) bool { + return tc.Cleanup == "" || tc.Cleanup == CleanupRollback +} + +// reportRollbackFailure explains why a requested rollback did not happen. +// +// Two causes are worth telling apart. The endpoint may not support rollback at +// all — with --attach the app is hosted by whatever mxcli started it, which can +// predate this feature — and that is a different fix from a transaction the +// runtime refused to roll back. +func reportRollbackFailure(w io.Writer, tc TestCase, rr *runResponse) { + switch { + case !rr.RollbackRequested: + fmt.Fprintf(w, " WARNING: %s ran without rollback — the app is hosting an older test endpoint\n"+ + " that ignores it. Restart the hosting 'mxcli run --local --test-endpoint'.\n", tc.Name) + case rr.RollbackError != "": + fmt.Fprintf(w, " WARNING: %s could not be rolled back: %s\n", tc.Name, rr.RollbackError) + default: + fmt.Fprintf(w, " WARNING: %s could not be rolled back (no reason reported)\n", tc.Name) + } +} + +// rollbackNote annotates a verbose result line with what happened to the +// transaction. +func rollbackNote(requested bool, rr *runResponse) string { + switch { + case !requested: + return " [committed]" + case rr.RolledBack: + return " [rolled back]" + default: + return " [ROLLBACK FAILED]" + } +} diff --git a/cmd/mxcli/testrunner/cleanup_strategy_test.go b/cmd/mxcli/testrunner/cleanup_strategy_test.go new file mode 100644 index 000000000..dcdc8146e --- /dev/null +++ b/cmd/mxcli/testrunner/cleanup_strategy_test.go @@ -0,0 +1,217 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testrunner + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestRollsBack(t *testing.T) { + tests := []struct { + name string + cleanup string + want bool + }{ + {"absent annotation defaults to rollback", "", true}, + {"explicit rollback", CleanupRollback, true}, + {"explicit none", CleanupNone, false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := rollsBack(TestCase{Cleanup: tt.cleanup}); got != tt.want { + t.Errorf("rollsBack(%q) = %v, want %v", tt.cleanup, got, tt.want) + } + }) + } +} + +// TestRollbackIsTheDefault pins the contract the annotation has always +// documented: a test with no @cleanup rolls back. It went unimplemented until +// the endpoint gave the runner a context of its own to open a transaction on. +func TestRollbackIsTheDefault(t *testing.T) { + if !rollsBack(TestCase{}) { + t.Error("a test with no @cleanup annotation does not roll back") + } +} + +func TestValidateCleanup(t *testing.T) { + for _, ok := range []string{"", CleanupRollback, CleanupNone} { + if err := validateCleanup(ok); err != nil { + t.Errorf("validateCleanup(%q) rejected a valid strategy: %v", ok, err) + } + } +} + +// TestValidateCleanupRejectsATypo pins the reason this validation exists at all. +// Treating an unrecognised value as "not rollback" would leave the test's data +// in the database while the run still reported a clean pass — the worst +// available outcome, because nothing anywhere would say why. +func TestValidateCleanupRejectsATypo(t *testing.T) { + err := validateCleanup("rollbak") + if err == nil { + t.Fatal("a misspelled strategy was accepted; it would silently skip the rollback") + } + if !strings.Contains(err.Error(), "rollbak") { + t.Errorf("error %q does not quote the offending value", err) + } + // The message has to say what IS allowed, or the user is left guessing. + for _, want := range []string{CleanupRollback, CleanupNone} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not list the valid strategy %q", err, want) + } + } +} + +// TestParserRejectsABadCleanup pins that the rejection happens at parse time, so +// --list catches it too and no runtime is booted for a test file that cannot be +// run correctly. +func TestParserRejectsABadCleanup(t *testing.T) { + body := `/** + * @test something + * @cleanup rollbak + */ +$r = CALL MICROFLOW Mod.A(); +/ +` + dir := t.TempDir() + path := filepath.Join(dir, "bad.test.mdl") + if err := os.WriteFile(path, []byte(body), 0o600); err != nil { + t.Fatalf("write: %v", err) + } + + if _, err := ParseTestFile(path); err == nil { + t.Fatal("a .test.mdl with a misspelled @cleanup parsed without error") + } +} + +// TestMarkdownParserRejectsABadCleanup covers the other file format — the two +// parsers are separate code paths and the first version of this validation only +// reached one of them. +func TestMarkdownParserRejectsABadCleanup(t *testing.T) { + body := "```mdl-test\n/**\n * @test something\n * @cleanup rollbak\n */\n$r = CALL MICROFLOW Mod.A();\n```\n" + dir := t.TempDir() + path := filepath.Join(dir, "bad.test.md") + if err := os.WriteFile(path, []byte(body), 0o600); err != nil { + t.Fatalf("write: %v", err) + } + + if _, err := ParseTestFile(path); err == nil { + t.Fatal("a .test.md with a misspelled @cleanup parsed without error") + } +} + +func TestEndpointJavaSupportsRollback(t *testing.T) { + for _, want := range []string{ + `"1".equals(request.getParameter("rollback"))`, + "ctx.startTransaction();", + "ctx.rollbackTransaction();", + } { + if !strings.Contains(endpointJava, want) { + t.Errorf("the handler is missing %q", want) + } + } +} + +// TestEndpointRollbackIsInAFinallyBlock pins that a test which throws still gets +// its transaction rolled back. Without the finally, a failing test would be +// exactly the one that leaves its half-written data behind. +func TestEndpointRollbackIsInAFinallyBlock(t *testing.T) { + execute := strings.Index(endpointJava, "Core.microflowCall(mf).execute") + finallyIdx := strings.Index(endpointJava, "} finally {") + rollbackIdx := strings.Index(endpointJava, "ctx.rollbackTransaction();") + + if finallyIdx < 0 { + t.Fatal("the execution is not wrapped in try/finally") + } + if !(execute < finallyIdx && finallyIdx < rollbackIdx) { + t.Errorf("the rollback is not in the finally block after execution (execute=%d finally=%d rollback=%d)", + execute, finallyIdx, rollbackIdx) + } +} + +// TestEndpointStartsTheTransactionBeforeExecuting pins the ordering: a +// transaction opened after the microflow ran would roll back nothing. +func TestEndpointStartsTheTransactionBeforeExecuting(t *testing.T) { + start := strings.Index(endpointJava, "ctx.startTransaction();") + execute := strings.Index(endpointJava, "Core.microflowCall(mf).execute") + if start < 0 || execute < 0 { + t.Fatal("a landmark is missing") + } + if start > execute { + t.Error("the transaction is started after the microflow runs, so it would roll back nothing") + } +} + +// TestEndpointReportsRollbackOutcome pins that a rollback which fails is +// reported rather than swallowed — otherwise the data stays and the run still +// says PASS. +func TestEndpointReportsRollbackOutcome(t *testing.T) { + for _, want := range []string{`\"rolledBack\":`, `\"rollbackRequested\":`, `\"rollbackError\":`} { + if !strings.Contains(endpointJava, want) { + t.Errorf("the response does not carry %s", want) + } + } +} + +func TestReportRollbackFailureDistinguishesCauses(t *testing.T) { + tc := TestCase{Name: "some test"} + + tests := []struct { + name string + resp runResponse + want string + }{ + { + name: "an endpoint that ignores the parameter", + resp: runResponse{RollbackRequested: false}, + want: "older test endpoint", + }, + { + name: "a runtime that refused", + resp: runResponse{RollbackRequested: true, RollbackError: "transaction already ended"}, + want: "transaction already ended", + }, + { + name: "no reason given", + resp: runResponse{RollbackRequested: true}, + want: "no reason reported", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var buf bytes.Buffer + reportRollbackFailure(&buf, tc, &tt.resp) + if !strings.Contains(buf.String(), tt.want) { + t.Errorf("warning %q does not mention %q", buf.String(), tt.want) + } + if !strings.Contains(buf.String(), tc.Name) { + t.Errorf("warning %q does not name the test", buf.String()) + } + }) + } +} + +func TestRollbackNote(t *testing.T) { + tests := []struct { + name string + requested bool + resp runResponse + want string + }{ + {"committed", false, runResponse{}, "committed"}, + {"rolled back", true, runResponse{RolledBack: true}, "rolled back"}, + {"failed", true, runResponse{}, "ROLLBACK FAILED"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := rollbackNote(tt.requested, &tt.resp); !strings.Contains(got, tt.want) { + t.Errorf("rollbackNote = %q, want it to contain %q", got, tt.want) + } + }) + } +} diff --git a/cmd/mxcli/testrunner/client.go b/cmd/mxcli/testrunner/client.go index 27253f415..63880f461 100644 --- a/cmd/mxcli/testrunner/client.go +++ b/cmd/mxcli/testrunner/client.go @@ -47,6 +47,13 @@ type runResponse struct { DurationMicros int64 `json:"durationMicros"` Result string `json:"result"` Error string `json:"error"` + // RollbackRequested echoes back whether the runner asked for a rollback, so a + // runner talking to an older endpoint that ignores the parameter can tell. + RollbackRequested bool `json:"rollbackRequested"` + // RolledBack reports that the transaction was actually rolled back. + RolledBack bool `json:"rolledBack"` + // RollbackError is why it was not. + RollbackError string `json:"rollbackError"` } // listResponse is the endpoint's reply to a list request. @@ -111,10 +118,16 @@ func (c *endpointClient) list() ([]string, error) { return lr.Microflows, nil } -// run executes one test microflow and returns the endpoint's reply. -func (c *endpointClient) run(mf string) (*runResponse, error) { +// run executes one test microflow and returns the endpoint's reply. With +// rollback set, the endpoint wraps the call in a transaction it rolls back, so +// the test's database writes do not survive. +func (c *endpointClient) run(mf string, rollback bool) (*runResponse, error) { + params := url.Values{"mf": {mf}} + if rollback { + params.Set("rollback", "1") + } var rr runResponse - if err := c.get("run", url.Values{"mf": {mf}}, &rr); err != nil { + if err := c.get("run", params, &rr); err != nil { return nil, err } return &rr, nil diff --git a/cmd/mxcli/testrunner/client_test.go b/cmd/mxcli/testrunner/client_test.go index a95b660f1..582ecc544 100644 --- a/cmd/mxcli/testrunner/client_test.go +++ b/cmd/mxcli/testrunner/client_test.go @@ -21,6 +21,8 @@ type fakeEndpoint struct { // seenTokens records what each request presented, so a test can assert the // client actually sends the token rather than the server merely allowing it. seenTokens []string + // rollbackParams records the rollback query parameter of each run request. + rollbackParams []string } func (f *fakeEndpoint) handler() http.Handler { @@ -45,6 +47,7 @@ func (f *fakeEndpoint) handler() http.Handler { } json.NewEncoder(w).Encode(listResponse{Microflows: names}) case strings.HasSuffix(r.URL.Path, "/run"): + f.rollbackParams = append(f.rollbackParams, r.URL.Query().Get("rollback")) mf := r.URL.Query().Get("mf") resp, ok := f.flows[mf] if !ok { @@ -215,3 +218,29 @@ func TestWaitReadyGivesUp(t *testing.T) { t.Errorf("error %q does not explain the endpoint never came up", err) } } + +// TestClientSendsTheRollbackParameter pins that the runner's per-test decision +// actually reaches the endpoint. Without the parameter the endpoint commits, and +// a test annotated for rollback would silently leave its data behind. +func TestClientSendsTheRollbackParameter(t *testing.T) { + fake, c := newFakeEndpoint(t, "tok", map[string]runResponse{ + testFlowPrefix + "test_1": {OK: true, Result: verdictPass}, + }) + + if _, err := c.run(testFlowPrefix+"test_1", true); err != nil { + t.Fatalf("run with rollback: %v", err) + } + if _, err := c.run(testFlowPrefix+"test_1", false); err != nil { + t.Fatalf("run without rollback: %v", err) + } + + if len(fake.rollbackParams) != 2 { + t.Fatalf("server saw %d run requests, want 2", len(fake.rollbackParams)) + } + if fake.rollbackParams[0] != "1" { + t.Errorf("rollback run sent rollback=%q, want \"1\"", fake.rollbackParams[0]) + } + if fake.rollbackParams[1] != "" { + t.Errorf("non-rollback run sent rollback=%q, want it absent", fake.rollbackParams[1]) + } +} diff --git a/cmd/mxcli/testrunner/endpoint.go b/cmd/mxcli/testrunner/endpoint.go index 86a2fea14..3f5cb4e30 100644 --- a/cmd/mxcli/testrunner/endpoint.go +++ b/cmd/mxcli/testrunner/endpoint.go @@ -235,10 +235,23 @@ com.mendix.core.Core.addRequestHandler("` + endpointPath + `", new com.mendix.ex return; } + // rollback=1 wraps the call in a transaction this handler owns and rolls + // it back afterwards, so the test's database writes do not survive it. + // The microflow joins that transaction rather than committing its own — + // Mendix contexts carry one transaction, and a nested start/end only + // adjusts its depth, so the outer rollback undoes everything inside. + boolean rollback = "1".equals(request.getParameter("rollback")); + long t0 = System.nanoTime(); com.mendix.systemwideinterfaces.core.IContext ctx = com.mendix.core.Core.createSystemContext(); Object result = null; String error = null; + String rollbackError = null; + boolean rolledBack = false; + + if (rollback) { + ctx.startTransaction(); + } try { result = com.mendix.core.Core.microflowCall(mf).execute(ctx); } catch (Throwable t) { @@ -246,6 +259,21 @@ com.mendix.core.Core.addRequestHandler("` + endpointPath + `", new com.mendix.ex while (root.getCause() != null && root.getCause() != root) root = root.getCause(); String msg = root.getMessage(); error = (msg == null || msg.isEmpty()) ? root.getClass().getName() : msg; + } finally { + if (rollback) { + // A rollback that silently fails leaves the data behind while the + // run still reports a clean pass, so its outcome is reported + // rather than swallowed. A microflow that already threw may have + // ended the transaction itself; that is not an error worth + // failing the test over, but it is worth saying. + try { + ctx.rollbackTransaction(); + rolledBack = true; + } catch (Throwable t) { + String msg = t.getMessage(); + rollbackError = (msg == null || msg.isEmpty()) ? t.getClass().getName() : msg; + } + } } long micros = (System.nanoTime() - t0) / 1000L; @@ -255,6 +283,9 @@ com.mendix.core.Core.addRequestHandler("` + endpointPath + `", new com.mendix.ex b.append(",\"durationMicros\":").append(micros); b.append(",\"result\":").append(result == null ? "null" : esc(String.valueOf(result))); if (error != null) b.append(",\"error\":").append(esc(error)); + b.append(",\"rollbackRequested\":").append(rollback); + b.append(",\"rolledBack\":").append(rolledBack); + if (rollbackError != null) b.append(",\"rollbackError\":").append(esc(rollbackError)); b.append('}'); out.write(b.toString()); out.flush(); diff --git a/cmd/mxcli/testrunner/parser.go b/cmd/mxcli/testrunner/parser.go index ae276ac81..aff53633b 100644 --- a/cmd/mxcli/testrunner/parser.go +++ b/cmd/mxcli/testrunner/parser.go @@ -134,6 +134,10 @@ func parseMDLTests(content string, sourcePath string) ([]TestCase, error) { continue } + if err := validateCleanup(annotations.Cleanup); err != nil { + return nil, fmt.Errorf("%s: test %q: %w", sourcePath, annotations.Test, err) + } + testID := fmt.Sprintf("test_%d", i+1) tests = append(tests, TestCase{ ID: testID, @@ -185,6 +189,10 @@ func parseMarkdownTests(content string, sourcePath string) ([]TestCase, error) { doc, body, _ := extractDocAndBody(blockContent, blockContent) annotations := parseAnnotations(doc) + if err := validateCleanup(annotations.Cleanup); err != nil { + return nil, fmt.Errorf("%s: test at line %d: %w", sourcePath, blockStart, err) + } + testNum++ testID := fmt.Sprintf("test_%d", testNum) diff --git a/cmd/mxcli/testrunner/runner_endpoint.go b/cmd/mxcli/testrunner/runner_endpoint.go index c3d67c0de..969c33d56 100644 --- a/cmd/mxcli/testrunner/runner_endpoint.go +++ b/cmd/mxcli/testrunner/runner_endpoint.go @@ -129,6 +129,9 @@ func runSuite(client *endpointClient, suite *TestSuite, opts RunOptions, w io.Wr result := &SuiteResult{Name: suite.Name, Started: time.Now()} fmt.Fprintf(w, "Running %d test(s) over the test endpoint...\n", len(suite.Tests)) + // leaked counts tests whose requested rollback did not happen. + leaked := 0 + for _, tc := range suite.Tests { flow := testFlowName(tc) if !present[flow] { @@ -141,7 +144,8 @@ func runSuite(client *endpointClient, suite *TestSuite, opts RunOptions, w io.Wr continue } - rr, err := client.run(flow) + rollback := rollsBack(tc) + rr, err := client.run(flow, rollback) if err != nil { // A transport failure is not a verdict. Report it against this test // and keep going; if the runtime died the rest will say so too. @@ -154,13 +158,27 @@ func runSuite(client *endpointClient, suite *TestSuite, opts RunOptions, w io.Wr continue } + // A rollback that was asked for and did not happen leaves the test's data + // in the database while the verdict still says PASS. The test itself is + // not wrong, so its verdict stands — but this must not pass in silence. + if rollback && !rr.RolledBack { + leaked++ + reportRollbackFailure(w, tc, rr) + } + res := toResult(tc, rr) result.Tests = append(result.Tests, res) if opts.Verbose { - fmt.Fprintf(w, " %s %s (%s)\n", res.Status, res.Name, res.Duration.Round(time.Millisecond)) + fmt.Fprintf(w, " %s %s (%s)%s\n", res.Status, res.Name, + res.Duration.Round(time.Millisecond), rollbackNote(rollback, rr)) } } + if leaked > 0 { + fmt.Fprintf(w, "\nWARNING: %d test(s) asked for @cleanup rollback and did not get it — "+ + "their writes are still in the database.\n", leaked) + } + result.Duration = time.Since(result.Started) return result, nil } diff --git a/docs-site/src/tools/running-tests.md b/docs-site/src/tools/running-tests.md index 4d45072d2..8fd1770f8 100644 --- a/docs-site/src/tools/running-tests.md +++ b/docs-site/src/tools/running-tests.md @@ -87,6 +87,43 @@ every request must present that token, non-loopback callers are refused, and it will only ever invoke the generated `MxTest.Test_*` microflows. The token is never written into your project. +### `@cleanup`: what happens to a test's data + +`rollback` is the **default**, so a test's database writes do not survive it. +The endpoint opens a transaction around the call and rolls it back afterwards, +including when the test throws. + +```mdl +/** + * @test creating an order does not leak + * @expect $result = 'ok' + */ +$result = CALL MICROFLOW Sales.CreateOrder(Amount = 100); +/ + +/** + * @test seed a fixture the app should keep + * @cleanup none + */ +$result = CALL MICROFLOW Sales.SeedCatalogue(); +/ +``` + +| Strategy | Effect | +|---|---| +| `rollback` (default) | The test's writes are rolled back, even if it throws | +| `none` | The writes commit and persist | + +Rollback needs the test endpoint, so it applies to `--local` and `--attach`. +The Docker / `--legacy-runner` path runs tests inside the after-startup action +and has no context of its own to roll back, so it always commits. + +A rollback that fails is reported per test and summarised at the end — data +left behind while the suite still says PASS is exactly what this is for. +`--verbose` tags every test `[rolled back]`, `[committed]` or +`[ROLLBACK FAILED]`. A misspelled strategy is a parse error, not a silent +commit. + **Docker — the after-startup runner.** The whole suite is compiled into the project's after-startup microflow, the container is restarted, and results are parsed out of its log. `--legacy-runner` selects this on a local run too. diff --git a/mdl-examples/doctype-tests/cleanup-rollback.test.mdl b/mdl-examples/doctype-tests/cleanup-rollback.test.mdl new file mode 100644 index 000000000..204177837 --- /dev/null +++ b/mdl-examples/doctype-tests/cleanup-rollback.test.mdl @@ -0,0 +1,56 @@ +-- ============================================================================ +-- @cleanup rollback — worked example +-- ============================================================================ +-- Demonstrates what happens to a test's database writes. +-- +-- @cleanup rollback (the default) the writes are rolled back +-- @cleanup none the writes commit and persist +-- +-- Rollback needs the test endpoint, which owns the context each test runs in, +-- so it applies to `--local` and `--attach`. The Docker / --legacy-runner path +-- runs tests inside the after-startup action and always commits. +-- +-- Setup — these are the microflows under test: +-- +-- create persistent entity App.Person (FirstName: string(100)); +-- +-- create microflow App.CreatePerson (FirstName: string) +-- returns string as $Stored +-- begin +-- declare $Stored String = ''; +-- $P = create App.Person (FirstName = $FirstName); +-- commit $P; +-- set $Stored = $P/FirstName; +-- return $Stored; +-- end; +-- / +-- +-- Run: mxcli test cleanup-rollback.test.mdl -p app.mpr --local --verbose +-- +-- --verbose tags each result [rolled back] / [committed], and afterwards only +-- the PersistedProbe row is in the database. +-- ============================================================================ + +/** + * @test the default is rollback — this Person does not survive the run + * @expect $result = 'RollbackProbe' + */ +$result = CALL MICROFLOW App.CreatePerson(FirstName = 'RollbackProbe'); +/ + +/** + * @test stating rollback explicitly does the same thing + * @expect $result = 'ExplicitRollbackProbe' + * @cleanup rollback + */ +$result = CALL MICROFLOW App.CreatePerson(FirstName = 'ExplicitRollbackProbe'); +/ + +/** + * @test @cleanup none commits — the control that proves rollback is doing the + * work above, and the way to seed a fixture you want to keep + * @expect $result = 'PersistedProbe' + * @cleanup none + */ +$result = CALL MICROFLOW App.CreatePerson(FirstName = 'PersistedProbe'); +/ From 00a6f51663cb4e874b07ec00997a56fae08be663 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 00:42:13 +0000 Subject: [PATCH 02/31] fix(test): run the app's own after-startup microflow during --local MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reported from a real project (mxcli-formula1 findings #19): a suite passed under --attach and failed under --local, with cached-service assertions seeing zero rows. The app loads its cache from an after-startup microflow, and the --local runner displaced that microflow with its own. That was a deliberate choice — a test run wants a known baseline — but it was invisible. The run printed only "After-startup set to MxTest.RegisterEndpoint", never that the user's startup logic had been displaced, so the failure looked like a bug in the code under test. It was also inconsistent: the hosted --test-endpoint path already chained the project's own microflow, which is precisely why the two modes disagreed. --local now chains it too, so a suite behaves the same either way and tests see the app in the state it really boots into. --skip-app-startup opts out for a deterministic empty baseline, and the run always prints which of the two it did: … (registers the endpoint; runs no tests, then runs your MyModule.ASU_Startup) … (registers the endpoint; runs no tests; --skip-app-startup, so … will NOT run) Verified live with a seeding after-startup microflow and a test asserting on its row: PASS chained, FAIL under --skip-app-startup, from an emptied table. Note the startup microflow's writes are not covered by @cleanup rollback — they happen at boot, outside any test's transaction. Also from the same report (#15): mxcli test --list bypassed resolveTestPaths, so a project-relative path resolved for execution but not for listing. Confirmed against the pre-fix binary, which fails with "stat tests/: no such file or directory" on the command that now works. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .claude/skills/fix-issue.md | 2 + .claude/skills/mendix/test-microflows.md | 33 +++++++++- CLAUDE.md | 2 +- cmd/mxcli/cmd_test_run.go | 42 ++++++++----- cmd/mxcli/main.go | 1 + cmd/mxcli/syntax/features_misc.go | 7 +++ cmd/mxcli/testrunner/cleanup_strategy.go | 21 +++++++ cmd/mxcli/testrunner/cleanup_strategy_test.go | 61 +++++++++++++++++++ cmd/mxcli/testrunner/runner.go | 36 +++++++---- docs-site/src/tools/running-tests.md | 11 ++++ 10 files changed, 186 insertions(+), 30 deletions(-) diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 5427fbebf..06fd7f511 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -402,3 +402,5 @@ extracting `OffsetExpression`/`LimitExpression`. | `$Total = 5;` does not parse — `no viable alternative at input '$Total=5'` — while `DECLARE $Total Integer = 0;` does, and so do `$X = HEAD($List)`, `$X = create M.E (…)` and `$X = execute database query …`. The error names the token, not the missing keyword | Assignment existed only as a **prefix** on specific activity statements (`(VARIABLE EQUALS)?` on CALL/CREATE/RETRIEVE/…), plus a `SET $Var = expression` statement. A plain value therefore required `SET`, which nothing in the error or the surrounding syntax suggested | `mdl/grammar/domains/MDLMicroflow.g4` (`setStatement : SET? …`), `cmd/mxcli/syntax/features_microflow.go` | Make the guessable form work rather than improve the error: `SET` is now optional and both spellings produce the same `MfSetStmt`/`ChangeVariableAction`. **Prove a grammar relaxation causes no regressions with a control binary, not by reading**: `git stash` the `.g4`, `make grammar`, build `bin/mxcli-control`, sweep every `mdl-examples/**/*.mdl` with both — 13 scripts fail, the *same* 13, all pre-existing. ANTLR's adaptive prediction picks the activity-prefixed alternatives over `setStatement` on its own; no ordering change was needed. Executed against a real .mpr, mxbuild reports 0 errors. Tests `mdl/visitor/visitor_microflow_bare_assign_test.go` (bare and keyword forms must agree on the AST, not merely both parse). mxcli-formula1 #13 | | `mxcli test tests/ -p app/App.mpr` fails with "no such file or directory" for a `tests/` that sits right next to the `.mpr` | Test paths resolved against the process CWD only. Defensible in isolation, but mxcli otherwise encourages naming the project (`-p`) rather than standing in its directory, and project auto-discovery searches outward — so the two conventions collide and the failure looks like a missing directory | `cmd/mxcli/cmd_test_run.go` (`resolveTestPaths`) | Fall back to project-relative **only when the CWD-relative path does not exist**: a `tests/` in both places must resolve to the one the user is standing in, since silently preferring the project's copy would run the wrong suite. A path that exists in neither is passed through unchanged so the error names what was typed, not a rewritten path the user never mentioned. Tests `cmd/mxcli/cmd_test_run_paths_test.go`. mxcli-formula1 #13 | | A test annotated `@cleanup rollback` (or with no `@cleanup` at all — rollback is the documented default) still leaves its rows in the database; a misspelled strategy like `@cleanup rollbak` does the same, silently, while the run reports PASS | `TestCase.Cleanup` was parsed and then used nowhere. The after-startup runner had no seam to implement it — tests run inside the startup action, so there is no context the runner owns. The test endpoint creates that seam: it builds the `IContext` each test runs on | `cmd/mxcli/testrunner/endpoint.go` (the handler's execute block), `cmd/mxcli/testrunner/cleanup_strategy.go` | Wrap the call in `ctx.startTransaction()` … `ctx.rollbackTransaction()` in a **finally** (a throwing test is the one most likely to leave half-written data), gated on a `rollback=1` query parameter the client sends per test. Report `rolledBack`/`rollbackError` in the response and warn per test — a rollback that fails silently is worse than none. Reject an unknown `@cleanup` value at **parse** time so `--list` catches it too. Verify against the database, not the endpoint's own claim: run one test with rollback and one with `@cleanup none` in the same suite and query Postgres — the `none` row must be the only survivor | +| A suite passes under `mxcli test --attach` and fails under `--local`, with assertions that depend on startup state (a loaded cache, seeded reference data) seeing zero rows | The `--local` runner pointed after-startup at its own registration microflow and did **not** chain the project's own, so the app's startup logic never ran. It was a deliberate choice (a known baseline) but was invisible: the run printed only `After-startup set to MxTest.RegisterEndpoint`, never that the user's microflow had been displaced | `cmd/mxcli/testrunner/runner.go` (`runEndpoint`), `cmd/mxcli/testrunner/cleanup_strategy.go` (`describeStartup`) | Capture project state **before** generating the endpoint MDL, and pass `state.afterStartup` to `GenerateEndpointMDL` so the generated flow chains it — the hosted `--test-endpoint` path already did this, and the mismatch between the two was the bug. Add `--skip-app-startup` for a deterministic empty baseline, and always print which of the two happened. Note the startup microflow's writes run at boot, outside any test transaction, so `@cleanup rollback` does not undo them. mxcli-formula1 findings #19 | +| `mxcli test tests/ -p app/App.mpr --list` fails with `stat tests/: no such file or directory` while the same command without `--list` runs fine | The `--list` branch passed raw `args` to `ListTests`, bypassing `resolveTestPaths` — so a path relative to the project (rather than the working directory) resolved for execution but not for listing | `cmd/mxcli/cmd_test_run.go` (the `if list` branch) | Pass `resolveTestPaths(args, projectPath)` there too. When a command has two entry points into the same input, check both go through the same path resolution. mxcli-formula1 findings #15 | diff --git a/.claude/skills/mendix/test-microflows.md b/.claude/skills/mendix/test-microflows.md index 6133b2c85..2a2f90ff8 100644 --- a/.claude/skills/mendix/test-microflows.md +++ b/.claude/skills/mendix/test-microflows.md @@ -172,8 +172,9 @@ older **after-startup microflow** pattern. 2. Records the project's current after-startup microflow, and whether an `MxTest` module already exists 3. Generates **one `MxTest.Test_` microflow per test**, plus a Java action - that registers an HTTP endpoint, and points after-startup at a microflow whose - only job is to call it — **no test runs during startup** + that registers an HTTP endpoint, and points after-startup at a microflow that + registers it and then **chains your own after-startup microflow** — + **no test runs during startup** 4. Builds and boots the app once 5. Invokes each test by name over HTTP; each returns its own verdict in the response @@ -192,6 +193,34 @@ Two consequences worth knowing when reading a failing run: Each test is a separate microflow with its own variable scope, so `$result` in one test never collides with `$result` in another. +#### Your app's after-startup microflow still runs + +The generated startup flow registers the endpoint and then calls the project's +own after-startup microflow, so tests see the app in the state it actually boots +into — a loaded cache, seeded reference data, whatever your app does. The run +says which happened: + +``` +After-startup set to MxTest.RegisterEndpoint (registers the endpoint; runs no tests, then runs your MyModule.ASU_Startup) +``` + +Pass `--skip-app-startup` when you want an empty, deterministic baseline +instead — the app seeds demo data and your tests assert on counts, say: + +``` +After-startup set to MxTest.RegisterEndpoint (… --skip-app-startup, so MyModule.ASU_Startup will NOT run) +``` + +This is why a suite behaves the same under `--local` and `--attach`. Before it +chained, `--local` ran with the app's startup logic suppressed, and a suite that +depended on startup state passed under `--attach` and failed under `--local` for +reasons unrelated to the code. + +One thing rollback does **not** cover: whatever the startup microflow writes +happens at boot, outside any test's transaction, so `@cleanup rollback` does not +undo it. Under `--local` that lands in the scratch `_test` database; +under `--attach` your app wrote it at its own boot regardless. + #### `--watch`: keep the runtime warm ```bash diff --git a/CLAUDE.md b/CLAUDE.md index cea1e09ee..f39bf63e3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -610,7 +610,7 @@ Full syntax tables for all MDL statements (microflows, pages, security, navigati - LSP server with hover, go-to-definition, completion, diagnostics, symbols, folding - VS Code extension (`vscode-mdl`) with context menu commands (Run/Check/Selection) - Docker build integration (`mxcli docker build`) with PAD patching (Phase 1) -- Warm test loop (`mxcli test --local [--watch]`, `--attach`, `run --local --test-endpoint`): local test runs go through a **token-guarded HTTP endpoint** registered by a generated Java custom request handler, instead of compiling the suite into the project's after-startup microflow. One microflow per test, resolved by name at request time from `Core.getMicroflowNames()` and invoked with `Core.microflowCall(...).execute(...)` — so a throwing test fails only itself (not the boot), results are returned rather than scraped from the runtime log, and each test has its own variable scope. Owning the `IContext` is also what finally makes **`@cleanup rollback`** (the annotation's documented default, previously parsed and ignored) real: the handler wraps the call in `startTransaction()`/`rollbackTransaction()`, so a test's writes do not survive it — verified against Postgres, with `@cleanup none` as the in-run control. A rollback that fails is reported per test and summarised, never silent; an unknown strategy is a parse error. The handler **survives `reload_model`** (after-startup does not re-run, the JVM is unchanged), which is what makes `--watch` possible: ~30s first run, then ~2s from an edit — to a test *or* to the microflow under test — to a verdict. `--attach` skips even that boot by running against an app already up under `run --local --test-endpoint`, driving that process's serve + admin APIs over loopback; it uses **that app's database**, only ever adds/removes its own test microflows, and refuses a change needing a restart. Security: the handler is **not registered at all** without `MXCLI_TEST_TOKEN` in the runtime env (so a project that kept the `MxTest` module through a failed cleanup is inert in production), the token is constant-time compared, non-loopback callers are refused, `/list` is clamped to the test namespace, and only `MxTest.Test_*` may be invoked. The token reaches the runtime via its environment and is never written into the project. Docker keeps the after-startup runner (`--legacy-runner` selects it locally). Packages: `cmd/mxcli/testrunner/` (`endpoint.go`, `client.go`, `watch.go`, `host.go`, `handshake.go`). See `docs/15-testing/SPIKE_test_endpoint_request_handler.md` +- Warm test loop (`mxcli test --local [--watch]`, `--attach`, `run --local --test-endpoint`): local test runs go through a **token-guarded HTTP endpoint** registered by a generated Java custom request handler, instead of compiling the suite into the project's after-startup microflow. Boot registers the endpoint and then **chains the project's own after-startup microflow**, so tests see the app as it really boots (`--skip-app-startup` opts out) — without that, a suite depending on startup state passed under `--attach` and failed under `--local`. One microflow per test, resolved by name at request time from `Core.getMicroflowNames()` and invoked with `Core.microflowCall(...).execute(...)` — so a throwing test fails only itself (not the boot), results are returned rather than scraped from the runtime log, and each test has its own variable scope. Owning the `IContext` is also what finally makes **`@cleanup rollback`** (the annotation's documented default, previously parsed and ignored) real: the handler wraps the call in `startTransaction()`/`rollbackTransaction()`, so a test's writes do not survive it — verified against Postgres, with `@cleanup none` as the in-run control. A rollback that fails is reported per test and summarised, never silent; an unknown strategy is a parse error. The handler **survives `reload_model`** (after-startup does not re-run, the JVM is unchanged), which is what makes `--watch` possible: ~30s first run, then ~2s from an edit — to a test *or* to the microflow under test — to a verdict. `--attach` skips even that boot by running against an app already up under `run --local --test-endpoint`, driving that process's serve + admin APIs over loopback; it uses **that app's database**, only ever adds/removes its own test microflows, and refuses a change needing a restart. Security: the handler is **not registered at all** without `MXCLI_TEST_TOKEN` in the runtime env (so a project that kept the `MxTest` module through a failed cleanup is inert in production), the token is constant-time compared, non-loopback callers are refused, `/list` is clamped to the test namespace, and only `MxTest.Test_*` may be invoked. The token reaches the runtime via its environment and is never written into the project. Docker keeps the after-startup runner (`--legacy-runner` selects it locally). Packages: `cmd/mxcli/testrunner/` (`endpoint.go`, `client.go`, `watch.go`, `host.go`, `handshake.go`). See `docs/15-testing/SPIKE_test_endpoint_request_handler.md` - Warm local dev loop (`mxcli run --local [--watch] [--screenshot]`): Docker-free `mxbuild --serve` + standalone runtime, hot `reload_model` for behavioural changes and restart+DDL for structural ones (chosen from the serve build's `restartRequired`). Bundles the browser client (`web/dist/` via mxbuild's rollup runner, which the serve Deploy target skips) so Mendix 11.x apps render in a browser. `--watch` keeps an incremental rollup bundler hot (CHOKIDAR_USEPOLLING for container fs; ~3-4s page re-bundle, skipped for model-only edits) and watches only model source (`.mpr`+`mprcontents/`). `--ensure-db` provisions the local Postgres + app database if missing; `--setup` does the non-blocking prerequisites (cache mxbuild+runtime, ensure DB) and exits — `mxcli init` wires it into a Claude Code SessionStart hook so a fresh/reaped web session self-bootstraps, and `docs-site/src/tools/bootstrap-prompt.md` is the empty-repo seed prompt. `--screenshot` captures a Playwright PNG each change (pixel-perfect page loop), with `--screenshot-url` deep links (repeatable for multi-page sets, one PNG per page) and `--screenshot-user`/`--screenshot-password` form login (session saved as Playwright storage state, reused via `screenshot --load-storage`). See `docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md` - External browser preview (`mxcli run --hub ` + `mxcli tunnel-hub`): the app stays local and reverse-tunnels out over a single 443 connection (embedded chisel) to a static relay, so it is reachable in a browser at a public URL — works from egress-only environments (Claude Code web), verified live through the session's MITM egress proxy. `run --hub` implies `--local`, boots the runtime with `ApplicationRootUrl` set to the assigned URL (so the SPA/`originURI` work under the public origin), resolves the control proxy honouring `NO_PROXY`, and retries forever. `mxcli tunnel-hub --domain ` is the **multi-tenant** relay: a registry keyed by prefix/project/solution/branch/worktree (stable URLs on reconnect) fronts many previews at per-subdomain hosts (`[prefix-]project[-branch].`; main collapses to the project) over one 443 with per-subdomain autocert, a registration API (`/api/register|status|deregister|backends|sessions`), and an availability overview at `hub./` **grouped by Claude Code session** (`/api/sessions`): each session lists the endpoints it exposed and links back to its `claude.ai/code` conversation. Client identity flags: `--hub-prefix`/`--hub-project`/`--hub-solution`/`--hub-branch`/`--hub-worktree` (project + branch auto-detected); `--hub-session` groups a session's endpoints (auto-detected from `CLAUDE_CODE_REMOTE_SESSION_ID`). Past sessions are retained: a durable per-session endpoint history (`--sessions-file`, default `~/.mxcli/hub-sessions.json`) survives restarts and reaping, and is pruned after `--session-retention` (default 30d) — so the overview shows offline sessions too (`SessionLog` in `cmd/mxcli/tunnelhub/sessions.go`). Package: `cmd/mxcli/tunnelhub/`. See `docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md` (slices 3–4) - Tunnel-hub GitHub authentication (opt-in, gated on `--github-oauth-client-id`; absent = today's open hub): **viewer plane** — GitHub OAuth web flow + HMAC-signed SSO session cookie (`Domain=.`), owner-checked previews (`--require-auth` default on → 302 to login / 403 non-owner; soft mode filters the listing only), `/api/backends` filtered to the viewer (unauthenticated → 401), admin "signed in as" via `/api/whoami`. **Registration plane** — durable, hashed hub API keys (`--keys-file`, default `~/.mxcli/hub-keys.json`, survive restarts) presented as `X-Hub-Key` → stamps `Backend.Owner`; shared `X-Hub-Secret` still works as an owner-less fallback. **Key issuance** — the hub's `/cli` browser page mints a key from the session cookie (no PAT; the device flow was removed as Claude Code containers block GitHub's device endpoints), rotate-by-default + count + revoke-all; `mxcli auth hub login --token ` is the headless path; `run --hub` reads `MXCLI_HUB_KEY` (env → `~/.mxcli/auth.json`) and degrades to local-only if registration fails. Append-only JSONL audit trail (`--audit-log`, no secrets). Packages: `cmd/mxcli/tunnelhub/` (+`audit/`), `cmd/mxcli/hubauth/`. See `docs/11-proposals/PROPOSAL_hub_authentication.md` diff --git a/cmd/mxcli/cmd_test_run.go b/cmd/mxcli/cmd_test_run.go index f5e5eb45a..46b7abcdd 100644 --- a/cmd/mxcli/cmd_test_run.go +++ b/cmd/mxcli/cmd_test_run.go @@ -38,10 +38,16 @@ loop can keep serving the same project while tests run. 1. Parses test files and extracts test blocks with @test/@expect annotations 2. Generates one microflow per test, plus a Java action that registers a token-guarded HTTP endpoint - 3. Boots the app once — startup only registers the endpoint, it runs no tests + 3. Boots the app once — startup registers the endpoint and then runs your own + after-startup microflow, so tests see the app as it really boots. No test + runs during startup 4. Invokes each test by name over HTTP; the verdict comes back in the response 5. Restores original project settings +Your after-startup microflow running is what makes a suite behave the same under +--local and --attach. Pass --skip-app-startup for an empty, deterministic +baseline instead — the run always prints which of the two it did. + Because each test is its own microflow invoked on its own, a test that throws fails only itself instead of ending the run, and results are returned rather than recovered from the runtime log. @@ -127,6 +133,7 @@ Examples: legacyRunner, _ := cmd.Flags().GetBool("legacy-runner") watch, _ := cmd.Flags().GetBool("watch") attach, _ := cmd.Flags().GetBool("attach") + skipAppStartup, _ := cmd.Flags().GetBool("skip-app-startup") verbose, _ := cmd.Flags().GetBool("verbose") color, _ := cmd.Flags().GetBool("color") timeoutStr, _ := cmd.Flags().GetString("timeout") @@ -138,8 +145,10 @@ Examples: } if list { - // Just list tests, no execution needed - if err := testrunner.ListTests(args, os.Stdout); err != nil { + // resolveTestPaths here too: listing that cannot find a path execution + // finds is a confusing split, and `mxcli test tests/ -p app/App.mpr + // --list` hit exactly that (mxcli-formula1 findings #15). + if err := testrunner.ListTests(resolveTestPaths(args, projectPath), os.Stdout); err != nil { fmt.Fprintf(os.Stderr, "Error: %v\n", err) os.Exit(1) } @@ -153,19 +162,20 @@ Examples: } opts := testrunner.RunOptions{ - ProjectPath: projectPath, - TestFiles: resolveTestPaths(args, projectPath), - SkipBuild: skipBuild, - Local: local, - LegacyRunner: legacyRunner, - Watch: watch, - Attach: attach, - Timeout: timeout, - JUnitOutput: junitOutput, - Verbose: verbose, - Color: color, - Stdout: os.Stdout, - Stderr: os.Stderr, + ProjectPath: projectPath, + TestFiles: resolveTestPaths(args, projectPath), + SkipBuild: skipBuild, + Local: local, + LegacyRunner: legacyRunner, + Watch: watch, + Attach: attach, + SkipAppStartup: skipAppStartup, + Timeout: timeout, + JUnitOutput: junitOutput, + Verbose: verbose, + Color: color, + Stdout: os.Stdout, + Stderr: os.Stderr, } result, err := testrunner.Run(opts) diff --git a/cmd/mxcli/main.go b/cmd/mxcli/main.go index be87beadd..c6fc84421 100644 --- a/cmd/mxcli/main.go +++ b/cmd/mxcli/main.go @@ -373,6 +373,7 @@ func init() { testRunCmd.Flags().Bool("local", false, "Run on mxcli's local runtime instead of Docker (no daemon needed)") testRunCmd.Flags().Bool("legacy-runner", false, "With --local, run tests from the after-startup microflow and parse the log, instead of over the test endpoint") testRunCmd.Flags().BoolP("watch", "w", false, "With --local, keep the runtime warm and re-run the suite on every test or model change (Ctrl-C to stop)") + testRunCmd.Flags().Bool("skip-app-startup", false, "With --local, do not run the project's own after-startup microflow during the test run (it runs by default, so tests see the app as it really boots)") testRunCmd.Flags().Bool("attach", false, "Run against an app already started with 'mxcli run --local --test-endpoint' instead of booting one (tests hit that app's database)") testRunCmd.Flags().BoolP("verbose", "v", false, "Show all runtime log output") testRunCmd.Flags().BoolP("color", "", false, "Use colored output") diff --git a/cmd/mxcli/syntax/features_misc.go b/cmd/mxcli/syntax/features_misc.go index fd5bd761c..5bbab9c68 100644 --- a/cmd/mxcli/syntax/features_misc.go +++ b/cmd/mxcli/syntax/features_misc.go @@ -283,6 +283,9 @@ Flags: every test or model change (Ctrl-C to stop) --attach Run against an app already started with 'mxcli run --local --test-endpoint' — no boot at all + --skip-app-startup + With --local, do not run the project's own + after-startup microflow (it runs by default) --legacy-runner With --local: use the old after-startup runner -v, --verbose Show runtime log lines -t, --timeout DUR Runtime startup timeout (default: 5m) @@ -305,6 +308,10 @@ token-guarded HTTP endpoint the app registers at boot. A test that throws fails only itself, and results are returned rather than scraped from the log. Docker still uses the older after-startup runner. +Boot also runs the project's own after-startup microflow, chained after the +endpoint registration, so tests see the app in the state it really boots into +and a suite behaves the same under --local and --attach. + Cost of a run: cold (--local) ~30s boots a runtime on its own ports + DB warm (--local --watch) ~2s runtime stays up between runs diff --git a/cmd/mxcli/testrunner/cleanup_strategy.go b/cmd/mxcli/testrunner/cleanup_strategy.go index f3c981995..66c855ba4 100644 --- a/cmd/mxcli/testrunner/cleanup_strategy.go +++ b/cmd/mxcli/testrunner/cleanup_strategy.go @@ -87,3 +87,24 @@ func rollbackNote(requested bool, rr *runResponse) string { return " [ROLLBACK FAILED]" } } + +// describeStartup says what the generated after-startup microflow will do, +// naming the project's own microflow when there is one. +// +// This line exists because its absence was a reported trap (mxcli-formula1 +// findings #19). The runner printed only that after-startup had been pointed at +// its own microflow; a reader had no way to tell that their app's startup logic +// — a cache load, in that report — was therefore not going to run. The suite +// passed under --attach, where the app boots normally, and failed under --local +// for reasons that had nothing to do with the code under test. +func describeStartup(appAfterStartup string, skipped bool) string { + base := "After-startup set to " + endpointStartupFlow + " (registers the endpoint; runs no tests" + switch { + case appAfterStartup == "": + return base + "; this project has no after-startup microflow of its own)" + case skipped: + return base + "; --skip-app-startup, so " + appAfterStartup + " will NOT run)" + default: + return base + ", then runs your " + appAfterStartup + ")" + } +} diff --git a/cmd/mxcli/testrunner/cleanup_strategy_test.go b/cmd/mxcli/testrunner/cleanup_strategy_test.go index dcdc8146e..ea4e4c35e 100644 --- a/cmd/mxcli/testrunner/cleanup_strategy_test.go +++ b/cmd/mxcli/testrunner/cleanup_strategy_test.go @@ -215,3 +215,64 @@ func TestRollbackNote(t *testing.T) { }) } } + +// TestDescribeStartup pins the line that mxcli-formula1 findings #19 asked for. +// The runner used to say only that after-startup had been repointed, leaving no +// way to tell that the app's own startup logic would not run — which produced a +// suite that passed under --attach and failed under --local for reasons +// unrelated to the code. +func TestDescribeStartup(t *testing.T) { + tests := []struct { + name string + app string + skipped bool + want []string + absent []string + }{ + { + name: "chains the project's own microflow by default", + app: "MyModule.ASU_Startup", + want: []string{"then runs your MyModule.ASU_Startup"}, + // It must not read as though the app's startup is being skipped. + absent: []string{"NOT run"}, + }, + { + name: "says plainly when it is skipped", + app: "MyModule.ASU_Startup", + skipped: true, + want: []string{"MyModule.ASU_Startup", "NOT run", "--skip-app-startup"}, + }, + { + name: "says when there is nothing to chain", + app: "", + want: []string{"no after-startup microflow of its own"}, + absent: []string{"NOT run"}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := describeStartup(tt.app, tt.skipped) + for _, w := range tt.want { + if !strings.Contains(got, w) { + t.Errorf("message %q does not contain %q", got, w) + } + } + for _, a := range tt.absent { + if strings.Contains(got, a) { + t.Errorf("message %q should not contain %q", got, a) + } + } + }) + } +} + +// TestSkippedStartupNamesTheFlagThatCausedIt keeps the skipped message +// actionable: a reader who did not pass the flag themselves (a script did) can +// still tell why their startup logic is missing. +func TestSkippedStartupNamesTheFlagThatCausedIt(t *testing.T) { + got := describeStartup("Mod.Flow", true) + if !strings.Contains(got, "--skip-app-startup") { + t.Errorf("message %q does not name the flag responsible", got) + } +} diff --git a/cmd/mxcli/testrunner/runner.go b/cmd/mxcli/testrunner/runner.go index ffc8bf459..33e7e903e 100644 --- a/cmd/mxcli/testrunner/runner.go +++ b/cmd/mxcli/testrunner/runner.go @@ -58,6 +58,16 @@ type RunOptions struct { // then run against that app's database rather than a scratch one. Attach bool + // SkipAppStartup stops the project's own after-startup microflow from running + // during a --local test run. + // + // It normally does run: the generated startup flow registers the endpoint and + // then chains it, so tests see the app in the state it actually boots into. + // Set this when the suite wants an empty, deterministic baseline instead — + // e.g. the app seeds demo data at startup and the tests are asserting on + // counts. + SkipAppStartup bool + // Timeout for runtime startup and test execution. Timeout time.Duration @@ -178,10 +188,21 @@ func runEndpoint(opts RunOptions, suite *TestSuite, timeout time.Duration, w io. return nil, err } + // Capture what cleanup will need to restore, before touching anything. This + // must succeed: without it cleanup cannot tell an existing MxTest module from + // the one it is about to create, nor restore the original after-startup — and + // the generated startup flow needs to know what to chain. + state, err := captureProjectState(opts.ProjectPath) + if err != nil { + return nil, fmt.Errorf("capturing project state: %w", err) + } + fmt.Fprintln(w, "Generating test endpoint and test microflows...") - // "" : a test run wants a known starting state, so the project's own - // after-startup is not chained here (a hosted endpoint does chain it). - endpointMDL := GenerateEndpointMDL("") + chain := state.afterStartup + if opts.SkipAppStartup { + chain = "" + } + endpointMDL := GenerateEndpointMDL(chain) flowsMDL := GenerateTestFlows(suite) if opts.Verbose { @@ -191,14 +212,7 @@ func runEndpoint(opts RunOptions, suite *TestSuite, timeout time.Duration, w io. fmt.Fprintln(w, "--- End MDL ---") } - // Capture what cleanup will need to restore, before touching anything. This - // must succeed: without it cleanup cannot tell an existing MxTest module from - // the one it is about to create, nor restore the original after-startup. fmt.Fprintln(w, "Injecting test endpoint into project...") - state, err := captureProjectState(opts.ProjectPath) - if err != nil { - return nil, fmt.Errorf("capturing project state: %w", err) - } // From here on the project is modified, so every exit runs cleanup. // @@ -235,7 +249,7 @@ func runEndpoint(opts RunOptions, suite *TestSuite, timeout time.Duration, w io. return finish(nil, fmt.Errorf("preparing project for the test run (%s): %w", cmd, err)) } } - fmt.Fprintf(w, " After-startup set to %s (registers the endpoint; runs no tests)\n", endpointStartupFlow) + fmt.Fprintln(w, " "+describeStartup(state.afterStartup, opts.SkipAppStartup)) // --watch keeps the runtime and the build server up and re-runs on every // change, so it owns the loop — including printing each run's results, which diff --git a/docs-site/src/tools/running-tests.md b/docs-site/src/tools/running-tests.md index 8fd1770f8..e79a9d837 100644 --- a/docs-site/src/tools/running-tests.md +++ b/docs-site/src/tools/running-tests.md @@ -87,6 +87,17 @@ every request must present that token, non-loopback callers are refused, and it will only ever invoke the generated `MxTest.Test_*` microflows. The token is never written into your project. +### The app's own after-startup microflow + +Boot registers the endpoint and then runs the project's own after-startup +microflow, so tests see the app in the state it really boots into. The run +prints which of the two happened, and `--skip-app-startup` opts out when a suite +wants an empty, deterministic baseline. + +This keeps a suite behaving the same under `--local` and `--attach`. Note that +what the startup microflow writes is not covered by `@cleanup rollback` — it +runs at boot, outside any test's transaction. + ### `@cleanup`: what happens to a test's data `rollback` is the **default**, so a test's database writes do not survive it. From cdc90eed2754244cda6c44394a7f9927631a9098 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 00:47:03 +0000 Subject: [PATCH 03/31] feat(check): reject validation rules on non-persistent entities (MDL054) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `create non-persistent entity X ( Name: String(100) not null error '…' )` passed both `mxcli check` and `mxcli exec`, and only a real build caught it: [error] [CE0070] "Validations rules are not allowed on entity 'X', because it is not persistable." `not null` and `unique` ARE validation rules — Studio Pro models "required" and "uniqueness" as rules on the entity rather than as column constraints — so Mendix rejects both on a non-persistable entity. Nothing in mxcli connected the attribute constraint to the entity's persistence kind. The construct matrix was established against mxbuild 11.6.6 rather than taken from the issue text: `not null` with a message, `not null` bare, and `unique` each produce CE0070, while a plain attribute does not. The bare form matters — the report only showed the message form, and treating the message as the trigger would have left half the bug in place. Scoped to the CREATE path, where the persistence kind is known. An `ALTER ENTITY … ADD ATTRIBUTE` does not carry it and cannot be told apart from a persistent entity without a project — the same limitation MDL020 has, and the rule comment says so rather than pretending otherwise. Checked for false positives before committing: no file in mdl-examples/ trips the new rule, and scripts/check-skill-mdl.sh still passes all 189 checkable blocks. The negative test is a .fail.mdl (must fail check, enforced by `make check-mdl`), paired with an -ok.mdl that pins the other edge — the same constraints on a persistent entity, which mxbuild confirms builds clean. Fixes #832 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- .claude/skills/fix-issue.md | 1 + .../bug-tests/832-npe-validation-rules-ok.mdl | 24 ++++++ .../832-npe-validation-rules.fail.mdl | 36 ++++++++ mdl/executor/bugfix_test.go | 86 +++++++++++++++++++ mdl/executor/cmd_enumerations.go | 47 ++++++++++ 5 files changed, 194 insertions(+) create mode 100644 mdl-examples/bug-tests/832-npe-validation-rules-ok.mdl create mode 100644 mdl-examples/bug-tests/832-npe-validation-rules.fail.mdl diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index abcf12831..286b0866c 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -391,3 +391,4 @@ extracting `OffsetExpression`/`LimitExpression`. | A fresh clone of a project created by `mxcli new` goes dirty the first time anyone builds it: ~50 **tracked** files modified that nobody edited — every `javascriptsource/*/actions/*.js` gains a banner, `import { Big } from "big.js"` and `export async function`, plus the matching `javasource` stubs. In a cloud session with a stop-hook git check it reads as "uncommitted changes" at the end of clean work | The template ships the generated action stubs in a slightly older shape and MxBuild rewrites them all on the first build. `mx check` does **not** — only a build does — so nothing before the first `run --local` could reveal it, which is after the user has already committed | `cmd/mxcli/docker/settle.go` (`SettleGeneratedSources`), `cmd/mxcli/cmd_new.go` (step 5/6, `--skip-build`), `cmd/mxcli/init.go` (`/theme-cache/` in the generated ignore list) | Fix the *timing*, not the content: run the build while the project is still being created, so the settled form lands in the first commit. Do **not** reimplement the rewrite — it is mxbuild's generator and version-specific; run the real thing. Best-effort by contract (no JDK, no mxbuild, failed build → warning, never a failed creation), because a settled tree is a nicety and a usable project is the deliverable. The other half is gitignore: `theme-cache/` is a cache and says so. A/B on 11.12.1, both git-init'd then built: `--skip-build` → 50 dirty files, default → **0**. Tests `cmd/mxcli/docker/settle_test.go`. mxcli-todo #7 | | `mxcli check` reports `✓ Syntax OK` / `Check passed!`, then `mxcli exec` on the same script fails partway through with `failed to resolve page: page not found: Module.Page` — a button targeting a page the script creates further down. `exec` is not transactional, so the statements before the failure are already written to the .mpr | Page references are resolved in statement order at exec time. `check --references` already had an ordered pass (`validateForwardPageRefs`), but it needs `-p`; plain `check` had nothing, and plain `check` is what gets run | `mdl/executor/validate_page_order.go` (`ValidateScriptPageOrder`, MDL-PAGE01), wired in `cmd/mxcli/cmd_check.go` beside the other project-free validators | The soundness argument is what makes this work without a project: a **plain** CREATE later in the script would fail if the page already existed, so the script itself asserts the page does not exist yet and the earlier reference cannot resolve against the project either. `CREATE OR MODIFY`/`OR REPLACE` assert nothing, so they stay with `--references`, which can look. **Generalisable**: an ordering rule that seems to need project state often does not, once you find the statement that already asserts what you were going to look up. Two things the diagnostic must say and does: a cycle cannot be fixed by ordering (create one page without the linking widget, add it with `ALTER PAGE … INSERT`), and commit before executing a large script. Verified against all `mdl-examples/**/*.mdl` for false positives (0). Tests `mdl/executor/validate_page_order_test.go`, example `mdl-examples/bug-tests/todo-9-forward-page-reference.fail.mdl`. mxcli-todo #9 | | A sidebar navigation label renders truncated — "All tasks" shows as "All task" — under every theme (`signal`, `ledger`, `console`) and both variants, so switching theme does not help. Measured on the live DOM as `scrollWidth=56` inside `clientWidth=48` | Atlas's **closed** sidebar is an icon rail: `--navsidebar-width-closed: 48px` in Atlas's own `themesource/atlas_core/web/themes/_theme-default.scss`. The label is wider than the rail, and the closed rail expects an *icon*, not text. No mxcli theme sets any navigation width — the themes map colours — which is exactly why every theme reproduces it | `.claude/skills/mendix/theme-styling.md` (documented; no code change) | **A fix was written, verified, and reverted** — record the reason: adding `text-overflow: ellipsis` to the nav item helps only where Atlas also sets `white-space: nowrap`; elsewhere the label wraps to two readable lines, and the rule turns `All / tasks` into `All / t…`. Screenshotted both ways against the real compiled CSS. The answer belongs to the app (give nav items icons — what the rail is for — or keep the sidebar open), not to a theme that would impose it on every app. **Generalisable**: when a reported symptom traces to an upstream layout constant, reproduce the geometry against the real compiled CSS (48px container, real class names, Playwright measurement) — it takes minutes, tells you whose constant it is, and shows when the "obvious" CSS fix is a regression. mxcli-todo #19d | +| `create non-persistent entity X ( A: String not null error '…' )` (or `unique`) passes `mxcli check` AND `mxcli exec`, then the build fails **CE0070** "Validations rules are not allowed on entity 'X', because it is not persistable" | `not null` / `unique` ARE validation rules — Studio Pro models "required" and "uniqueness" as rules on the entity, not as column constraints — so Mendix rejects them on a non-persistable entity. Nothing in mxcli connected the attribute constraint to the entity's persistence kind | `mdl/executor/cmd_enumerations.go` (`validateNPEValidationRules`, called from `ValidateEntity`) | Add **MDL054**, error severity, fired from the CREATE path only. **Establish the construct matrix against mxbuild before writing the rule, not from the issue text** — verified on 11.6.6 that `not null` with a message, `not null` bare, AND `unique` each produce CE0070 while a plain attribute does not, so the bare form (easy to miss, since the reporter only showed the message form) is flagged too. The CREATE path is the only one that can run this: `ALTER ENTITY … ADD ATTRIBUTE` does not carry the persistence kind, the same limitation MDL020 has. Sweep `mdl-examples/` + `scripts/check-skill-mdl.sh` after adding any error-severity rule — a false positive there breaks every user with that shape. Negative test `mdl-examples/bug-tests/832-npe-validation-rules.fail.mdl` (`.fail.mdl` = must fail check, enforced by `make check-mdl`) plus `-ok.mdl` pinning the other edge; tests `TestValidateEntityNPEValidationRules`. Issue #832 | diff --git a/mdl-examples/bug-tests/832-npe-validation-rules-ok.mdl b/mdl-examples/bug-tests/832-npe-validation-rules-ok.mdl new file mode 100644 index 000000000..7e7c088cc --- /dev/null +++ b/mdl-examples/bug-tests/832-npe-validation-rules-ok.mdl @@ -0,0 +1,24 @@ +-- ============================================================================ +-- Issue #832 — the forms MDL054 must NOT reject (positive half) +-- ============================================================================ +-- +-- The negative half is 832-npe-validation-rules.fail.mdl. This file pins the +-- other edge: MDL054 must not fire on a validation rule that is legitimately +-- placed, or on a non-persistent entity that carries none. +-- +-- Verified against mxbuild 11.6.6: this file builds with 0 errors. +-- ============================================================================ + +CREATE MODULE Bug832Ok; + +-- A PERSISTENT entity is exactly where validation rules belong. +CREATE OR MODIFY PERSISTENT ENTITY Bug832Ok.P ( + "Name": String(100) not null error 'Name is required', + "Code": String(50) unique error 'Code must be unique' +); + +-- A non-persistent entity with no validation rule is fine. +CREATE OR MODIFY NON-PERSISTENT ENTITY Bug832Ok.NpPlain ( + "Name": String(100), + "Qty": Integer +); diff --git a/mdl-examples/bug-tests/832-npe-validation-rules.fail.mdl b/mdl-examples/bug-tests/832-npe-validation-rules.fail.mdl new file mode 100644 index 000000000..dba2caf20 --- /dev/null +++ b/mdl-examples/bug-tests/832-npe-validation-rules.fail.mdl @@ -0,0 +1,36 @@ +-- ============================================================================ +-- Issue #832 — validation rules on a non-persistent entity were accepted +-- ============================================================================ +-- +-- NEGATIVE TEST (.fail.mdl): `mxcli check` MUST reject this file. An +-- unexpected pass means MDL054 has regressed. +-- +-- Mendix refuses a validation rule on a non-persistable entity: +-- +-- [error] [CE0070] "Validations rules are not allowed on entity 'X', +-- because it is not persistable." +-- +-- `not null` and `unique` ARE validation rules — Studio Pro models "required" +-- and "uniqueness" as rules on the entity, not as column constraints — so both +-- are rejected on an NPE. `mxcli check` and `mxcli exec` both accepted them and +-- only a real build caught it, which is the worst place to find out. +-- +-- Verified against mxbuild 11.6.6: `not null` with a message, `not null` bare, +-- and `unique` each produce CE0070; a plain attribute does not. The message is +-- optional and does not change the verdict. +-- +-- The accepted counterparts — the same constraints on a PERSISTENT entity, and +-- an NPE with no constraint — are in 832-npe-validation-rules-ok.mdl, which +-- must PASS. Together they pin both edges of the rule. +-- +-- Only the CREATE path can catch this: an `ALTER ENTITY … ADD ATTRIBUTE` does +-- not carry the entity's persistence kind, so it cannot be told apart from a +-- persistent entity without a project. Same limitation as MDL020. +-- ============================================================================ + +CREATE MODULE Bug832; + +CREATE OR MODIFY NON-PERSISTENT ENTITY Bug832.Np ( + "Name": String(100) not null error 'Name is required', + "Code": String(50) unique error 'Code must be unique' +); diff --git a/mdl/executor/bugfix_test.go b/mdl/executor/bugfix_test.go index f29792ee0..a92522418 100644 --- a/mdl/executor/bugfix_test.go +++ b/mdl/executor/bugfix_test.go @@ -906,3 +906,89 @@ func TestExprToStringNoSpaces(t *testing.T) { }) } } + +// TestValidateEntityNPEValidationRules covers issue #832: Mendix refuses +// validation rules on a non-persistable entity with +// +// CE0070 "Validations rules are not allowed on entity 'X', because it is +// not persistable." +// +// `not null` and `unique` ARE validation rules — Studio Pro models "required" +// and "uniqueness" as rules on the entity, not as column constraints — so both +// forms are rejected on an NPE. Verified against mxbuild 11.6.6: `not null` +// with a message, `not null` bare, and `unique` each produce CE0070, while a +// plain attribute does not. Before this rule `mxcli check` and `mxcli exec` +// both accepted them and only a real build caught it. +func TestValidateEntityNPEValidationRules(t *testing.T) { + cases := []struct { + name string + input string + wantFor []string // attribute names expected to be flagged + }{ + { + "not null with message", + `create non-persistent entity Test.NP ( "Name" : String(100) not null error 'req' );`, + []string{"Name"}, + }, + { + // The message is optional; the rule exists either way, so bare + // `not null` is rejected by Mendix just the same. + "not null bare", + `create non-persistent entity Test.NP ( "Name" : String(100) not null );`, + []string{"Name"}, + }, + { + "unique", + `create non-persistent entity Test.NP ( "Code" : String(50) unique error 'dup' );`, + []string{"Code"}, + }, + { + "both constraints on separate attributes", + `create non-persistent entity Test.NP ( "Name" : String(100) not null, "Code" : String(50) unique );`, + []string{"Name", "Code"}, + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + prog, errs := visitor.Build(c.input) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + stmt := prog.Statements[0].(*ast.CreateEntityStmt) + violations := ValidateEntity(stmt) + for _, attrName := range c.wantFor { + found := false + for _, v := range violations { + if v.RuleID == "MDL054" && strings.Contains(v.Message, "'"+attrName+"'") { + found = true + } + } + if !found { + t.Errorf("expected MDL054 for attribute %q (CE0070), got: %v", attrName, violations) + } + } + }) + } +} + +// A PERSISTENT entity may carry exactly the same constraints — that is where +// validation rules belong — so the rule must not fire there. Nor should it fire +// on an NPE attribute that carries no constraint. +func TestValidateEntityValidationRulesAllowedWhenPersistent(t *testing.T) { + cases := []string{ + `create persistent entity Test.P ( "Name" : String(100) not null error 'req', "Code" : String(50) unique );`, + `create non-persistent entity Test.NP ( "Name" : String(100), "Qty" : Integer );`, + } + for _, input := range cases { + prog, errs := visitor.Build(input) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + stmt := prog.Statements[0].(*ast.CreateEntityStmt) + for _, v := range ValidateEntity(stmt) { + if v.RuleID == "MDL054" { + t.Errorf("unexpected MDL054 on %q: %s", input, v.Message) + } + } + } +} diff --git a/mdl/executor/cmd_enumerations.go b/mdl/executor/cmd_enumerations.go index 73050fa7c..43801fa90 100644 --- a/mdl/executor/cmd_enumerations.go +++ b/mdl/executor/cmd_enumerations.go @@ -450,6 +450,53 @@ func ValidateEntity(stmt *ast.CreateEntityStmt) []linter.Violation { entityName := stmt.Name.String() for _, attr := range stmt.Attributes { violations = append(violations, validateEntityAttribute(attr, persistent, entityName)...) + if !persistent { + violations = append(violations, validateNPEValidationRules(attr, entityName)...) + } + } + return violations +} + +// validateNPEValidationRules (MDL054) rejects a validation rule on a +// non-persistable entity, which Mendix refuses with +// +// CE0070 "Validations rules are not allowed on entity 'X', because it is +// not persistable." +// +// `not null` and `unique` ARE validation rules: Studio Pro models "required" +// and "uniqueness" as rules on the entity rather than as column constraints, +// so both are rejected on an NPE. Verified against mxbuild 11.6.6 — `not null` +// with a message, `not null` bare, and `unique` each produce CE0070; a plain +// attribute does not. The message is optional and does not change the verdict, +// so the bare form is flagged too. Issue #832. +// +// Only the CREATE path can run this: an `ALTER ENTITY … ADD ATTRIBUTE` does not +// carry the entity's persistence kind, so ValidateAlterEntity cannot tell an NPE +// from a persistent entity without a project. That is the same limitation +// MDL020 has, for the same reason — see ValidateAlterEntity. +func validateNPEValidationRules(attr ast.Attribute, entityName string) []linter.Violation { + var violations []linter.Violation + flag := func(constraint, mdl string) { + violations = append(violations, linter.Violation{ + RuleID: "MDL054", + Severity: linter.SeverityError, + Message: fmt.Sprintf( + "attribute '%s' declares `%s` on non-persistent entity %s — Mendix does not allow "+ + "validation rules on a non-persistable entity (CE0070), and `%s` is a validation rule", + attr.Name, constraint, entityName, constraint), + Location: linter.Location{DocumentType: "entity", DocumentName: entityName}, + Suggestion: fmt.Sprintf( + "Drop `%s` from the attribute, or make the entity persistent. To keep the check on a "+ + "non-persistent entity, enforce it in the microflow that populates it "+ + "(e.g. `if %s = empty then` … ) rather than declaring `%s`.", + mdl, attr.Name, mdl), + }) + } + if attr.NotNull { + flag("not null", "not null") + } + if attr.Unique { + flag("unique", "unique") } return violations } From 6973a999dd419a6d6708d8aa218f28e4b34afea7 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 00:47:03 +0000 Subject: [PATCH 04/31] fix(exec): don't tell a statement to move before itself MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A statement's own name is "defined in the script but not yet created" at the moment it fails, so annotateForwardRef matched it and appended hint: X is defined later in this script — move its create statement before this one to any validation error whose message named its own subject. The advice is impossible to follow: the statement it points at is the one that failed. Surfaced by MDL054, whose message names the entity being created, but the misfire is general — it applies to any create statement whose error mentions itself. The fix uses the ast.Statement parameter the function already took and deliberately ignored (`_ ast.Statement`): collect the names the failing statement defines and skip them. A genuine forward reference — a name some LATER statement defines — is still annotated, which the test pins alongside the regression. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- mdl/executor/validate.go | 13 +++++-- mdl/executor/validate_forwardref_test.go | 49 ++++++++++++++++++++++++ 2 files changed, 59 insertions(+), 3 deletions(-) create mode 100644 mdl/executor/validate_forwardref_test.go diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index 536282f16..2b81d983b 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -148,12 +148,19 @@ func (sc *scriptContext) allNames() []string { // annotateForwardRef checks if a failed statement's error references an object // that is defined later in the script. If so, it appends a hint to reorder. -func annotateForwardRef(err error, _ ast.Statement, created, allDefined *scriptContext) error { +func annotateForwardRef(err error, stmt ast.Statement, created, allDefined *scriptContext) error { msg := err.Error() + // A statement's OWN name is "defined in the script but not yet created" at + // the moment it fails, so without this any validation error that names its + // own subject picked up the reorder hint — telling the author to move a + // statement before itself. Found via MDL054, whose message names the entity + // being created (#832). + self := newScriptContext() + self.collectSingle(stmt) // Check each name that is defined in the script but not yet created. for _, name := range allDefined.allNames() { - if created.has(name) { - continue // already created before this statement + if created.has(name) || self.has(name) { + continue // already created before this statement, or defined by it } if strings.Contains(msg, name) { return fmt.Errorf("%w\n hint: %s is defined later in this script — move its create statement before this one", err, name) diff --git a/mdl/executor/validate_forwardref_test.go b/mdl/executor/validate_forwardref_test.go new file mode 100644 index 000000000..13d6f2318 --- /dev/null +++ b/mdl/executor/validate_forwardref_test.go @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "errors" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// TestAnnotateForwardRef_SkipsOwnName covers a misfire found while adding +// MDL054: a statement's OWN name is "defined in the script but not yet +// created" at the moment it fails, so any validation error whose message names +// its own subject picked up the reorder hint — advising the author to move a +// statement before itself. +// +// A genuine forward reference (a name some LATER statement defines) must still +// be annotated. +func TestAnnotateForwardRef_SkipsOwnName(t *testing.T) { + script := `create non-persistent entity Test.NpEntity ( "Name" : String(100) not null ); +create microflow Test.Later () returns Boolean begin return true; end;` + prog, errs := visitor.Build(script) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + allDefined := newScriptContext() + for _, s := range prog.Statements { + allDefined.collectSingle(s) + } + created := newScriptContext() // nothing created yet + + t.Run("own name is not a forward reference", func(t *testing.T) { + err := errors.New("attribute 'Name' declares `not null` on non-persistent entity Test.NpEntity") + got := annotateForwardRef(err, prog.Statements[0], created, allDefined) + if strings.Contains(got.Error(), "defined later in this script") { + t.Errorf("statement was annotated as referring forward to itself:\n%s", got.Error()) + } + }) + + t.Run("a genuine forward reference is still annotated", func(t *testing.T) { + err := errors.New("microflow not found: Test.Later") + got := annotateForwardRef(err, prog.Statements[0], created, allDefined) + if !strings.Contains(got.Error(), "defined later in this script") { + t.Errorf("expected a reorder hint for Test.Later, got:\n%s", got.Error()) + } + }) +} From cc471f2cfae3ce2963bca7131b48027234af910c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 00:55:29 +0000 Subject: [PATCH 05/31] feat(check): reject XPath association traversal from a variable (MDL055) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `retrieve $L from Mod.Entity where [Name = $RefProduct/Mod.Product_Category/Name]` passed both `mxcli check` and `mxcli exec`, and the build then failed with CE0161 "Error(s) in XPath constraint". Mendix XPath reaches at most one hop off a variable, and nothing checked the hop count. The valid/invalid boundary was established against mxbuild 11.6.6 rather than inferred, and it is narrower than it first appears: $Var/Attr VALID the parameter's own attribute $Var/Mod.Assoc VALID one hop, the associated object $Var/Mod.Assoc/Attr CE0161 two or more hops so the rule keys on the number of segments. The obvious formulation — flag a module-qualified segment following a variable — would have rejected the middle form, which builds clean. Confirmed by building all three and then dropping the offender to verify the remaining two report 0 errors. This is a rejection rather than a smarter serializer because there is no valid XPath for the two-hop form: the constraint has to be restructured, and only the author knows which of the two shapes they meant. Both rewrites the message recommends were built and confirmed at 0 errors before the text claimed they work — retrieving the associated object first (one hop is a legal retrieve SOURCE) and constraining on that variable's own attribute, or inverting so the traversal starts at the entity being retrieved. No positive example in mdl-examples/ trips the rule, and scripts/check-skill-mdl.sh still passes all 189 checkable blocks. Negative test is a .fail.mdl paired with an -ok.mdl carrying both rewrites plus the two one-hop forms that must not be flagged. Fixes #831 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- .claude/skills/fix-issue.md | 1 + .../831-xpath-variable-traversal-ok.mdl | 56 ++++++++++++++ .../831-xpath-variable-traversal.fail.mdl | 44 +++++++++++ mdl/executor/validate_microflow.go | 38 ++++++++++ .../validate_xpath_vartraversal_test.go | 73 +++++++++++++++++++ 5 files changed, 212 insertions(+) create mode 100644 mdl-examples/bug-tests/831-xpath-variable-traversal-ok.mdl create mode 100644 mdl-examples/bug-tests/831-xpath-variable-traversal.fail.mdl create mode 100644 mdl/executor/validate_xpath_vartraversal_test.go diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 286b0866c..1928c8590 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -392,3 +392,4 @@ extracting `OffsetExpression`/`LimitExpression`. | `mxcli check` reports `✓ Syntax OK` / `Check passed!`, then `mxcli exec` on the same script fails partway through with `failed to resolve page: page not found: Module.Page` — a button targeting a page the script creates further down. `exec` is not transactional, so the statements before the failure are already written to the .mpr | Page references are resolved in statement order at exec time. `check --references` already had an ordered pass (`validateForwardPageRefs`), but it needs `-p`; plain `check` had nothing, and plain `check` is what gets run | `mdl/executor/validate_page_order.go` (`ValidateScriptPageOrder`, MDL-PAGE01), wired in `cmd/mxcli/cmd_check.go` beside the other project-free validators | The soundness argument is what makes this work without a project: a **plain** CREATE later in the script would fail if the page already existed, so the script itself asserts the page does not exist yet and the earlier reference cannot resolve against the project either. `CREATE OR MODIFY`/`OR REPLACE` assert nothing, so they stay with `--references`, which can look. **Generalisable**: an ordering rule that seems to need project state often does not, once you find the statement that already asserts what you were going to look up. Two things the diagnostic must say and does: a cycle cannot be fixed by ordering (create one page without the linking widget, add it with `ALTER PAGE … INSERT`), and commit before executing a large script. Verified against all `mdl-examples/**/*.mdl` for false positives (0). Tests `mdl/executor/validate_page_order_test.go`, example `mdl-examples/bug-tests/todo-9-forward-page-reference.fail.mdl`. mxcli-todo #9 | | A sidebar navigation label renders truncated — "All tasks" shows as "All task" — under every theme (`signal`, `ledger`, `console`) and both variants, so switching theme does not help. Measured on the live DOM as `scrollWidth=56` inside `clientWidth=48` | Atlas's **closed** sidebar is an icon rail: `--navsidebar-width-closed: 48px` in Atlas's own `themesource/atlas_core/web/themes/_theme-default.scss`. The label is wider than the rail, and the closed rail expects an *icon*, not text. No mxcli theme sets any navigation width — the themes map colours — which is exactly why every theme reproduces it | `.claude/skills/mendix/theme-styling.md` (documented; no code change) | **A fix was written, verified, and reverted** — record the reason: adding `text-overflow: ellipsis` to the nav item helps only where Atlas also sets `white-space: nowrap`; elsewhere the label wraps to two readable lines, and the rule turns `All / tasks` into `All / t…`. Screenshotted both ways against the real compiled CSS. The answer belongs to the app (give nav items icons — what the rail is for — or keep the sidebar open), not to a theme that would impose it on every app. **Generalisable**: when a reported symptom traces to an upstream layout constant, reproduce the geometry against the real compiled CSS (48px container, real class names, Playwright measurement) — it takes minutes, tells you whose constant it is, and shows when the "obvious" CSS fix is a regression. mxcli-todo #19d | | `create non-persistent entity X ( A: String not null error '…' )` (or `unique`) passes `mxcli check` AND `mxcli exec`, then the build fails **CE0070** "Validations rules are not allowed on entity 'X', because it is not persistable" | `not null` / `unique` ARE validation rules — Studio Pro models "required" and "uniqueness" as rules on the entity, not as column constraints — so Mendix rejects them on a non-persistable entity. Nothing in mxcli connected the attribute constraint to the entity's persistence kind | `mdl/executor/cmd_enumerations.go` (`validateNPEValidationRules`, called from `ValidateEntity`) | Add **MDL054**, error severity, fired from the CREATE path only. **Establish the construct matrix against mxbuild before writing the rule, not from the issue text** — verified on 11.6.6 that `not null` with a message, `not null` bare, AND `unique` each produce CE0070 while a plain attribute does not, so the bare form (easy to miss, since the reporter only showed the message form) is flagged too. The CREATE path is the only one that can run this: `ALTER ENTITY … ADD ATTRIBUTE` does not carry the persistence kind, the same limitation MDL020 has. Sweep `mdl-examples/` + `scripts/check-skill-mdl.sh` after adding any error-severity rule — a false positive there breaks every user with that shape. Negative test `mdl-examples/bug-tests/832-npe-validation-rules.fail.mdl` (`.fail.mdl` = must fail check, enforced by `make check-mdl`) plus `-ok.mdl` pinning the other edge; tests `TestValidateEntityNPEValidationRules`. Issue #832 | +| `retrieve $L from Mod.Entity where [Attr = $Var/Mod.Assoc/Attr]` passes `mxcli check` AND `mxcli exec`, then the build fails **CE0161** "Error(s) in XPath constraint" | Mendix XPath reaches at most ONE hop off a variable, and nothing checked the hop count | `mdl/executor/validate_microflow.go` (`checkXPathVariableTraversal`, `xpathVarTraversalRe`), called from the `*ast.RetrieveStmt` arm beside MDL047/MDL048 | Add **MDL055**, error severity, matching a `$var`-rooted path with **2+ segments**. **Establish the boundary against mxbuild first — it is narrower than it looks**: `$Var/Attr` VALID, `$Var/Mod.Assoc` VALID (one hop to the associated object), `$Var/Mod.Assoc/Attr` CE0161. A rule keying on "a module-qualified segment follows a variable" would reject the middle form, which is legal; key on hop count instead. Verified on 11.6.6 by building all three and dropping the offender to confirm the other two are clean. **Reject, don't try to serialize** — there is no valid XPath for the two-hop form, so the constraint must be restructured and only the author knows which way. **Verify the suggestion you emit**: both recommended rewrites (`retrieve $Related from $Var/Mod.Assoc;` then constrain on `$Related/Attr`; or invert to `[Mod.Assoc/Mod.Entity = $Var]`) were built and confirmed at 0 errors before the message claimed "both forms build clean". Negative test `mdl-examples/bug-tests/831-xpath-variable-traversal.fail.mdl` + `-ok.mdl`; test `TestXPathVariableTraversal`. Issue #831 | diff --git a/mdl-examples/bug-tests/831-xpath-variable-traversal-ok.mdl b/mdl-examples/bug-tests/831-xpath-variable-traversal-ok.mdl new file mode 100644 index 000000000..59285c468 --- /dev/null +++ b/mdl-examples/bug-tests/831-xpath-variable-traversal-ok.mdl @@ -0,0 +1,56 @@ +-- ============================================================================ +-- Issue #831 — the forms MDL055 must NOT reject (positive half) +-- ============================================================================ +-- +-- The negative half is 831-xpath-variable-traversal.fail.mdl. This file pins +-- the other edge: the two restructurings MDL055's message recommends, plus the +-- one-hop forms that are valid XPath and must not be flagged. +-- +-- Verified against mxbuild 11.6.6: this file builds with 0 errors. +-- ============================================================================ + +CREATE MODULE Bug831Ok; + +CREATE OR MODIFY PERSISTENT ENTITY Bug831Ok.Category ( Name: String(50) ); + +CREATE OR MODIFY PERSISTENT ENTITY Bug831Ok.Product ( Code: String(50) ); + +CREATE OR MODIFY ASSOCIATION Bug831Ok.Product_Category + FROM Bug831Ok.Product TO Bug831Ok.Category TYPE Reference; + +-- Recommended form 1: retrieve the associated object first (one hop is a legal +-- retrieve SOURCE), then constrain on that variable's own attribute. +CREATE OR MODIFY MICROFLOW Bug831Ok.Form1 ($RefProduct: Bug831Ok.Product) +RETURNS list of Bug831Ok.Category +BEGIN + retrieve $Related from $RefProduct/Bug831Ok.Product_Category; + retrieve $Categories from Bug831Ok.Category where [Name = $Related/Name]; + return $Categories; +END; + +-- Recommended form 2: invert the constraint so the traversal starts at the +-- entity being retrieved, which XPath does support. +CREATE OR MODIFY MICROFLOW Bug831Ok.Form2 ($RefProduct: Bug831Ok.Product) +RETURNS list of Bug831Ok.Category +BEGIN + retrieve $Categories from Bug831Ok.Category + where [Bug831Ok.Product_Category/Bug831Ok.Product = $RefProduct]; + return $Categories; +END; + +-- One hop off a variable is valid and must not be flagged: an attribute… +CREATE OR MODIFY MICROFLOW Bug831Ok.OneHopAttribute ($RefProduct: Bug831Ok.Product) +RETURNS list of Bug831Ok.Category +BEGIN + retrieve $Categories from Bug831Ok.Category where [Name = $RefProduct/Code]; + return $Categories; +END; + +-- …and the associated object itself. +CREATE OR MODIFY MICROFLOW Bug831Ok.OneHopAssociation ($RefProduct: Bug831Ok.Product) +RETURNS list of Bug831Ok.Product +BEGIN + retrieve $Products from Bug831Ok.Product + where [Bug831Ok.Product_Category = $RefProduct/Bug831Ok.Product_Category]; + return $Products; +END; diff --git a/mdl-examples/bug-tests/831-xpath-variable-traversal.fail.mdl b/mdl-examples/bug-tests/831-xpath-variable-traversal.fail.mdl new file mode 100644 index 000000000..efebccd60 --- /dev/null +++ b/mdl-examples/bug-tests/831-xpath-variable-traversal.fail.mdl @@ -0,0 +1,44 @@ +-- ============================================================================ +-- Issue #831 — RETRIEVE WHERE traversing an association from a variable +-- ============================================================================ +-- +-- NEGATIVE TEST (.fail.mdl): `mxcli check` MUST reject this file. An +-- unexpected pass means MDL055 has regressed. +-- +-- `where [Name = $RefProduct/ZKT39.Product_Category/Name]` passed `mxcli check` +-- and `mxcli exec`, then the build failed: +-- +-- [error] [CE0161] "Error(s) in XPath constraint." +-- +-- Mendix XPath reaches at most ONE hop off a variable. The boundary is narrower +-- than "a qualified name after a variable" — verified against mxbuild 11.6.6: +-- +-- $Var/Attr VALID the parameter's own attribute +-- $Var/Mod.Assoc VALID one hop, the associated object +-- $Var/Mod.Assoc/Attr CE0161 two or more hops +-- +-- so the rule keys on hop count. A rule that flagged any qualified segment +-- would reject the middle form, which is valid. +-- +-- There is no valid serialization of the two-hop form, which is why this is a +-- rejection and not a writer fix: the constraint has to be restructured, and +-- only the author knows which shape they meant. Both restructurings are in +-- 831-xpath-variable-traversal-ok.mdl, which must PASS. +-- ============================================================================ + +CREATE MODULE Bug831; + +CREATE OR MODIFY PERSISTENT ENTITY Bug831.Category ( Name: String(50) ); + +CREATE OR MODIFY PERSISTENT ENTITY Bug831.Product ( Code: String(50) ); + +CREATE OR MODIFY ASSOCIATION Bug831.Product_Category + FROM Bug831.Product TO Bug831.Category TYPE Reference; + +CREATE OR MODIFY MICROFLOW Bug831.ACT_Find ($RefProduct: Bug831.Product) +RETURNS list of Bug831.Category +BEGIN + retrieve $Categories from Bug831.Category + where [Name = $RefProduct/Bug831.Product_Category/Name]; + return $Categories; +END; diff --git a/mdl/executor/validate_microflow.go b/mdl/executor/validate_microflow.go index c05dee69a..86cb2e8d2 100644 --- a/mdl/executor/validate_microflow.go +++ b/mdl/executor/validate_microflow.go @@ -265,6 +265,7 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { xp := expressionToXPath(stmt.Where) v.checkXPathAssociationEmpty(stmt.Variable, xp) v.checkXPathIdConstraint(stmt.Variable, xp) + v.checkXPathVariableTraversal(stmt.Variable, xp) } case *ast.CallMicroflowStmt: v.checkAssociationObjectArgs("microflow "+stmt.MicroflowName.String(), stmt.Arguments) @@ -561,6 +562,43 @@ func (v *microflowValidator) checkXPathAssociationEmpty(variable, xpath string) } } +// xpathVarTraversalRe matches a path rooted at a $variable with TWO OR MORE +// segments (`$P/Mod.Assoc/Name`). One segment is deliberately not matched: both +// `$P/Code` (the parameter's own attribute) and `$P/Mod.Assoc` (one hop to the +// associated object) are valid XPath. The boundary is the hop count, not whether +// a segment is module-qualified — see checkXPathVariableTraversal. +var xpathVarTraversalRe = regexp.MustCompile(`\$(\w+)((?:/[A-Za-z_][\w.]*){2,})`) + +// checkXPathVariableTraversal flags a retrieve constraint that traverses an +// association FROM a variable (`[Name = $RefProduct/Mod.Product_Category/Name]`). +// Mendix XPath reaches at most one hop off a variable, so this fails the build +// with CE0161 while mxcli accepted it silently (issue #831). +// +// Verified against mxbuild 11.6.6 — the boundary is narrower than it looks: +// +// $Var/Attr VALID a parameter's own attribute +// $Var/Mod.Assoc VALID one hop, the associated object +// $Var/Mod.Assoc/Attr CE0161 two or more hops +// +// There is no valid serialization of the two-hop form, which is why this is a +// rejection rather than a writer fix: the constraint has to be restructured, and +// only the author knows which of the two shapes they meant. +func (v *microflowValidator) checkXPathVariableTraversal(variable, xpath string) { + for _, m := range xpathVarTraversalRe.FindAllStringSubmatch(xpath, -1) { + root, path := m[1], "$"+m[1]+m[2] + segs := strings.Split(strings.TrimPrefix(m[2], "/"), "/") + firstHop, leaf := segs[0], segs[len(segs)-1] + v.addViolation("MDL055", linter.SeverityError, + fmt.Sprintf("retrieve '$%s' constraint traverses an association from a variable (`%s`), which Mendix XPath "+ + "does not support (CE0161 \"Error(s) in XPath constraint\") — a constraint reaches at most one hop off a variable", + variable, path), + fmt.Sprintf("Retrieve the associated object first, then constrain on its own attribute: "+ + "`retrieve $Related from $%s/%s;` and use `[%s = $Related/%s]`. Or invert the constraint so the "+ + "traversal starts at the entity being retrieved: `[%s/ = $%s]`. Both forms build clean.", + root, firstHop, leaf, leaf, firstHop, root)) + } +} + // xpathIdConstraintRe matches a constraint comparing the object id against a VALUE // (`id = $strVar`, `id = '123'`, `id != 5`). It captures the right-hand operand. // Comparing `id` against an OBJECT variable (`[id != $ExistingOrder]` — the valid diff --git a/mdl/executor/validate_xpath_vartraversal_test.go b/mdl/executor/validate_xpath_vartraversal_test.go new file mode 100644 index 000000000..01d0aed1a --- /dev/null +++ b/mdl/executor/validate_xpath_vartraversal_test.go @@ -0,0 +1,73 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// TestXPathVariableTraversal covers issue #831: a retrieve constraint whose +// right-hand side traverses an association FROM a variable +// +// where [Name = $RefProduct/Mod.Product_Category/Name] +// +// passed `mxcli check` and `mxcli exec`, and the build then failed CE0161. +// +// The valid/invalid boundary was established against mxbuild 11.6.6, and it is +// narrower than "a qualified name after a variable": +// +// $Var/Attr VALID — a parameter's own attribute +// $Var/Mod.Assoc VALID — one hop to the associated object +// $Var/Mod.Assoc/Attr CE0161 — two or more hops +// +// so the rule must key on the number of segments, not on the presence of a +// module-qualified one. Flagging the middle form would reject valid MDL. +func TestXPathVariableTraversal(t *testing.T) { + cases := []struct { + name string + where string + flag bool + }{ + {"attribute of a parameter", `[Name = $P/Code]`, false}, + {"one hop to the associated object", `[BX.Product_Category = $P/BX.Product_Category]`, false}, + {"two hops — the reported form", `[Name = $P/BX.Product_Category/Name]`, true}, + {"three hops", `[Name = $P/BX.A_B/BX.B_C/Name]`, true}, + // An entity-rooted traversal is not variable-rooted and is valid XPath. + {"entity-rooted traversal", `[BX.Product_Category/BX.Product = $P]`, false}, + {"bare attribute compare", `[Name = 'x']`, false}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + src := `create microflow BX.M ($P: BX.Product) returns list of BX.Category +begin + retrieve $L from BX.Category where ` + c.where + `; + return $L; +end;` + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + stmt, ok := prog.Statements[0].(*ast.CreateMicroflowStmt) + if !ok { + t.Fatalf("got %T", prog.Statements[0]) + } + got := false + var msg string + for _, v := range ValidateMicroflow(stmt) { + if v.RuleID == "MDL055" { + got, msg = true, v.Message + } + } + if got != c.flag { + t.Errorf("MDL055 fired = %v, want %v (where %s)\n message: %s", got, c.flag, c.where, msg) + } + if c.flag && got && !strings.Contains(msg, "$P") { + t.Errorf("message should name the offending variable path: %s", msg) + } + }) + } +} From b2c4f20e2a54e3220cdbfcf34f8628df642d8749 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:03:29 +0000 Subject: [PATCH 06/31] fix(test): resolve a project-relative path for --list too `mxcli test tests/ -p app/App.mpr` ran from the solution root and `--list` on the same command line did not: Error: stat tests/: no such file or directory resolveTestPaths was called below the --list branch, which returned first. Listing and running now accept the same paths. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- cmd/mxcli/cmd_test_run.go | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/cmd/mxcli/cmd_test_run.go b/cmd/mxcli/cmd_test_run.go index 56f059b1c..0c99361f8 100644 --- a/cmd/mxcli/cmd_test_run.go +++ b/cmd/mxcli/cmd_test_run.go @@ -131,9 +131,15 @@ Examples: os.Exit(1) } + // Resolve before the --list branch, not after: listing and running must + // accept the same paths, or `mxcli test tests/ -p app/App.mpr` runs from + // the solution root and `--list` on the same command line does not. + // With no project this returns the paths untouched. + testPaths := resolveTestPaths(args, projectPath) + if list { // Just list tests, no execution needed - if err := testrunner.ListTests(args, os.Stdout); err != nil { + if err := testrunner.ListTests(testPaths, os.Stdout); err != nil { fmt.Fprintf(os.Stderr, "Error: %v\n", err) os.Exit(1) } @@ -148,7 +154,7 @@ Examples: opts := testrunner.RunOptions{ ProjectPath: projectPath, - TestFiles: resolveTestPaths(args, projectPath), + TestFiles: testPaths, SkipBuild: skipBuild, Local: local, LegacyRunner: legacyRunner, From bab4d4251b2c9477a0010012e0b93f07acb55586 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:03:29 +0000 Subject: [PATCH 07/31] fix(odata): publish Integer as Int64, and say when an enum is a string MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every whole-number attribute in a published service failed the build, one CE5016 each: Attribute …Stg_Season.Year is has type Integer, but is published as Edm.Int32. Mendix publishes Integer as Int64, same as Long. The mapping's own comment flagged Integer as an unverified guess, and the existing test pinned the guess. Publishing every attribute type on 11.12.1 and reading the errors off the build also caught a second wrong pair the report had only suspected: an enumeration was written as Edm.String with EnumerationAsString hardcoded false, which is the one combination Mendix rejects — CE5016 plus CE4583 "Enumeration 'Edm.Colour' is not published in this service". The type and the flag are one setting, so the flag now travels with the attribute. Verified: the same all-types service builds 0 errors. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/backend/modelsdk/odata_write.go | 2 +- mdl/executor/cmd_contract_test.go | 14 +++++-- mdl/executor/cmd_odata.go | 53 +++++++++++++++++++------ mdl/executor/cmd_odata_edm_type_test.go | 29 ++++++++++++++ model/types.go | 7 ++++ sdk/mpr/writer_odata.go | 2 +- 6 files changed, 89 insertions(+), 18 deletions(-) create mode 100644 mdl/executor/cmd_odata_edm_type_test.go diff --git a/mdl/backend/modelsdk/odata_write.go b/mdl/backend/modelsdk/odata_write.go index 7a153da21..a02b7d92b 100644 --- a/mdl/backend/modelsdk/odata_write.go +++ b/mdl/backend/modelsdk/odata_write.go @@ -354,7 +354,7 @@ func publishedMemberToGen(m *model.PublishedMember, ownerQN string) element.Elem addBool(g, "Filterable", m.Filterable) addBool(g, "Sortable", m.Sortable) addBool(g, "IsPartOfKey", m.IsPartOfKey) - addBool(g, "EnumerationAsString", false) + addBool(g, "EnumerationAsString", m.EnumerationAsString) addBool(g, "StringAsGuid", false) return g } diff --git a/mdl/executor/cmd_contract_test.go b/mdl/executor/cmd_contract_test.go index 696142eb9..8ca06d2b1 100644 --- a/mdl/executor/cmd_contract_test.go +++ b/mdl/executor/cmd_contract_test.go @@ -135,8 +135,13 @@ func TestCreateNavigationAssociations_NoDuplicateOnReimport(t *testing.T) { // TestMendixAttrTypeToEdm guards the Mendix→EDM type mapping used to populate a // published attribute's EdmType. Without it Studio Pro reports CE5016 -// ("published as ."). String/Decimal/Boolean/DateTimeOffset are verified against -// Studio Pro's corrected BSON. +// ("published as ."). +// +// Every row is now adjudicated by mxbuild rather than assumed: one attribute of +// each type published in one service on 11.12.1, then the CE5016s read off the +// build. This test previously pinned Integer to Edm.Int32, which was an +// unverified guess and wrong — Mendix publishes Integer as Int64, so every whole +// number in a published service failed the build (mxcli-formula1 #16). func TestMendixAttrTypeToEdm(t *testing.T) { cases := []struct { typ domainmodel.AttributeType @@ -144,13 +149,14 @@ func TestMendixAttrTypeToEdm(t *testing.T) { }{ {&domainmodel.StringAttributeType{}, "Edm.String"}, {&domainmodel.HashedStringAttributeType{}, "Edm.String"}, - {&domainmodel.IntegerAttributeType{}, "Edm.Int32"}, + {&domainmodel.IntegerAttributeType{}, "Edm.Int64"}, {&domainmodel.LongAttributeType{}, "Edm.Int64"}, {&domainmodel.AutoNumberAttributeType{}, "Edm.Int64"}, {&domainmodel.DecimalAttributeType{}, "Edm.Decimal"}, {&domainmodel.BooleanAttributeType{}, "Edm.Boolean"}, {&domainmodel.DateTimeAttributeType{}, "Edm.DateTimeOffset"}, - {&domainmodel.BinaryAttributeType{}, "Edm.Binary"}, + {&domainmodel.BinaryAttributeType{}, "Edm.Binary"}, // never reachable: CE5013 forbids exposing Binary at all + {&domainmodel.EnumerationAttributeType{}, "Edm.String"}, {nil, ""}, } for _, c := range cases { diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index 28a0a1d89..047be7d1f 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -1620,8 +1620,12 @@ type assocMembership struct { // return empty collections rather than failing the whole publish. // mendixAttrTypeToEdm maps a Mendix attribute type to the OData EDM type Studio // Pro publishes it as (the PublishedAttribute.EdmType field). Inverse of -// edmToDomainModelAttrType. String/Decimal/Boolean/DateTimeOffset verified -// against Studio Pro output; the rest follow the standard OData mapping. +// edmToDomainModelAttrType. +// +// Every case here has been adjudicated by mxbuild rather than assumed: an +// attribute published as the wrong EDM type is CE5016 ("has type Integer, but is +// published as Edm.Int32"), one error per attribute. Verified on 11.12.1 by +// publishing one attribute of each type and reading the errors. func mendixAttrTypeToEdm(t domainmodel.AttributeType) string { if t == nil { return "" @@ -1629,9 +1633,10 @@ func mendixAttrTypeToEdm(t domainmodel.AttributeType) string { switch t.GetTypeName() { case "String", "HashedString": return "Edm.String" - case "Integer": - return "Edm.Int32" - case "Long", "AutoNumber": + case "Integer", "Long", "AutoNumber": + // Mendix publishes Integer as Int64 too, not Int32 — an Integer is + // 64-bit in the Mendix type system, and Int32 is CE5016 on every whole + // number in the service. return "Edm.Int64" case "Decimal": return "Edm.Decimal" @@ -1640,18 +1645,38 @@ func mendixAttrTypeToEdm(t domainmodel.AttributeType) string { case "DateTime", "Date": return "Edm.DateTimeOffset" case "Binary": + // Kept so the mapping is total, but a Binary attribute cannot actually + // be exposed over OData — Mendix rejects it outright with CE5013 + // regardless of the published type. return "Edm.Binary" case "Enumeration": - // Enums exposed as string (EnumerationAsString path). A non-string enum - // exposure would use the enum's own type — not yet modelled. + // Paired with EnumerationAsString=true (see enumPublishedAsString). + // Edm.String with that flag false is rejected: Mendix then wants the + // enumeration published in the service and typed as its own EDM enum. return "Edm.String" default: return "Edm.String" } } -func lookupEntityMembers(ctx *ExecContext, entityQN ast.QualifiedName) (map[string]string, map[string]*assocMembership) { - attrs := make(map[string]string) // attr name -> OData EDM type (for the published EdmType) +// publishedAttrType is how one Mendix attribute publishes over OData: the EDM +// type, plus whether it is an enumeration flattened to a string. The two travel +// together because Edm.String alone is ambiguous — a String and an +// EnumerationAsString enum both carry it, and only the flag tells them apart. +type publishedAttrType struct { + Edm string + AsString bool +} + +// enumPublishedAsString reports whether a published attribute needs the +// EnumerationAsString flag — i.e. whether it is an enumeration at all. The flag +// and the Edm.String type are a matched pair; see mendixAttrTypeToEdm. +func enumPublishedAsString(t domainmodel.AttributeType) bool { + return t != nil && t.GetTypeName() == "Enumeration" +} + +func lookupEntityMembers(ctx *ExecContext, entityQN ast.QualifiedName) (map[string]publishedAttrType, map[string]*assocMembership) { + attrs := make(map[string]publishedAttrType) // attr name -> how it publishes assocs := make(map[string]*assocMembership) if ctx == nil || ctx.Backend == nil { return attrs, assocs @@ -1676,7 +1701,10 @@ func lookupEntityMembers(ctx *ExecContext, entityQN ast.QualifiedName) (map[stri } if thisEntity != nil { for _, a := range thisEntity.Attributes { - attrs[a.Name] = mendixAttrTypeToEdm(a.Type) + attrs[a.Name] = publishedAttrType{ + Edm: mendixAttrTypeToEdm(a.Type), + AsString: enumPublishedAsString(a.Type), + } } } for _, a := range dm.Associations { @@ -1741,9 +1769,10 @@ func astEntityDefToModel(ctx *ExecContext, def *ast.PublishedEntityDef) (*model. member.ExposedName = member.Name } // Auto-detect kind: attribute first, association as fallback. - if edmType, ok := entityAttrs[m.Name]; ok { + if pub, ok := entityAttrs[m.Name]; ok { member.Kind = "attribute" - member.EdmType = edmType + member.EdmType = pub.Edm + member.EnumerationAsString = pub.AsString } else if assoc := moduleAssocs[m.Name]; assoc != nil { member.Kind = "association" member.ExposedAssociationName = m.Name diff --git a/mdl/executor/cmd_odata_edm_type_test.go b/mdl/executor/cmd_odata_edm_type_test.go new file mode 100644 index 000000000..1781183f3 --- /dev/null +++ b/mdl/executor/cmd_odata_edm_type_test.go @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// An enumeration published as Edm.String must also carry EnumerationAsString. +// The pair is one setting: with the flag false, Mendix wants the enumeration +// published in the service as its own EDM enum type and rejects the string — +// CE5016 plus CE4583 "Enumeration 'Edm.Colour' is not published in this +// service". mxcli wrote Edm.String with the flag hardcoded false, which is the +// one combination that cannot build. +func TestEnumerationPublishesAsString(t *testing.T) { + if !enumPublishedAsString(&domainmodel.EnumerationAttributeType{}) { + t.Error("an enumeration attribute must set EnumerationAsString") + } + // A plain String also publishes as Edm.String but is not an enumeration — + // the flag is what tells the two apart, so it must not be set here. + if enumPublishedAsString(&domainmodel.StringAttributeType{}) { + t.Error("a String attribute must not set EnumerationAsString") + } + if enumPublishedAsString(nil) { + t.Error("a nil type must not set EnumerationAsString") + } +} diff --git a/model/types.go b/model/types.go index 7ee133361..6435c5127 100644 --- a/model/types.go +++ b/model/types.go @@ -496,6 +496,13 @@ type PublishedMember struct { // without it `mx check` reports CE5016 ("published as ."). Attribute members only. EdmType string `json:"edmType,omitempty"` + // EnumerationAsString publishes an enumeration attribute as Edm.String + // rather than as its own EDM enum type. The two are one setting, not two: + // with this false, Mendix expects the enumeration itself to be published in + // the service and rejects Edm.String (CE5016 + CE4583 "Enumeration is not + // published in this service"). Attribute members only. + EnumerationAsString bool `json:"enumerationAsString,omitempty"` + // Association-specific fields (Kind == "association"). Studio Pro's // ODataPublish$PublishedAssociationEnd records both the association // target entity (qualified name) and the bare association name diff --git a/sdk/mpr/writer_odata.go b/sdk/mpr/writer_odata.go index 4b8d3ee60..e79557311 100644 --- a/sdk/mpr/writer_odata.go +++ b/sdk/mpr/writer_odata.go @@ -411,7 +411,7 @@ func serializePublishedMember(m *model.PublishedMember, ownerQN string) bson.D { doc = append(doc, bson.E{Key: "Filterable", Value: m.Filterable}) doc = append(doc, bson.E{Key: "Sortable", Value: m.Sortable}) doc = append(doc, bson.E{Key: "IsPartOfKey", Value: m.IsPartOfKey}) - doc = append(doc, bson.E{Key: "EnumerationAsString", Value: false}) + doc = append(doc, bson.E{Key: "EnumerationAsString", Value: m.EnumerationAsString}) doc = append(doc, bson.E{Key: "StringAsGuid", Value: false}) case "association": doc = append(doc, bson.E{Key: "$Type", Value: "ODataPublish$PublishedAssociationEnd"}) From 29481bc7f3c0016316e22e505c688d1c999d3c09 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:03:29 +0000 Subject: [PATCH 08/31] fix(external-entities): read an attribute's OData mapping back `create or modify external entity` touching only an entity-level property detonated every attribute of the entity: [CE6612] "Attribute 'circuitId' of external entity 'Stg_Circuit' is not supported." one per attribute, leaving a project that cannot build. The executor already preserves attributes it was not asked to change, so the loss was a layer down: attributeFromGen handled StoredValue and OqlViewValue but not Rest$ODataMappedValue. Every attribute of an external entity therefore came back with no RemoteName, and the writer's `isExternal && a.RemoteName != ""` arm fell through to a plain StoredValue on the next read-modify-write. The per-attribute Filterable/Sortable/Creatable/Updatable flags live on the same value and were lost with it. This is the attribute-level half of #782, which fixed the entity level only. Verified on 11.12.1 against a real contract import: three CE6612 before, none after, and the modify's own change still lands. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/backend/modelsdk/domainmodel.go | 18 ++++++++ .../modelsdk/external_entity_read_test.go | 46 +++++++++++++++++++ 2 files changed, 64 insertions(+) diff --git a/mdl/backend/modelsdk/domainmodel.go b/mdl/backend/modelsdk/domainmodel.go index a8d61fa80..23260e7e7 100644 --- a/mdl/backend/modelsdk/domainmodel.go +++ b/mdl/backend/modelsdk/domainmodel.go @@ -266,6 +266,24 @@ func attributeFromGen(a *genDm.Attribute) *domainmodel.Attribute { // View-entity attribute: the OQL column reference must survive a // read-modify-write (e.g. MOVE ENTITY) or the view goes out of sync (CE6770). attr.Value = &domainmodel.AttributeValue{ViewReference: v.Reference()} + case *genRest.ODataMappedValue: + // External-entity attribute: the mapping to the remote OData property. + // Reading it back is what makes a read-modify-write safe — without it + // every attribute of an external entity comes back unmapped, and the + // writer's `isExternal && a.RemoteName != ""` arm falls through to a + // plain StoredValue. The entity then no longer matches the contract: + // "Attribute 'year' of external entity 'Stg_Season' is not supported." + attr.RemoteName = v.RemoteName() + attr.RemoteType = v.RemoteType() + attr.Filterable = v.Filterable() + attr.Sortable = v.Sortable() + attr.Creatable = v.Creatable() + attr.Updatable = v.Updatable() + case *genRest.ODataMappedPrimitiveCollectionValue: + // The single attribute of a primitive-collection NPE (issue #718). + attr.RemoteName = v.RemoteName() + attr.RemoteType = v.RemoteType() + attr.IsPrimitiveCollection = true } return attr } diff --git a/mdl/backend/modelsdk/external_entity_read_test.go b/mdl/backend/modelsdk/external_entity_read_test.go index 556773014..3c737c9fb 100644 --- a/mdl/backend/modelsdk/external_entity_read_test.go +++ b/mdl/backend/modelsdk/external_entity_read_test.go @@ -227,3 +227,49 @@ func TestExternalEntity_PrimitiveCollectionSourceRoundTrip(t *testing.T) { t.Errorf("RemoteServiceName = %q", got.RemoteServiceName) } } + +// TestExternalEntity_AttributeRemoteMappingRoundTrip is the attribute-level half +// of #782, found by mxcli-formula1 #25: entityFromGen learned to read the +// entity's own remote fields, but attributeFromGen still handled only +// StoredValue and OqlViewValue. A Rest$ODataMappedValue therefore came back with +// no RemoteName, and the write path's `isExternal && a.RemoteName != ""` arm fell +// through to a plain StoredValue on the next read-modify-write. +// +// The visible failure is a `create or modify external entity` that touches only +// an entity-level property and detonates every attribute: +// +// [CE6612] "Attribute 'circuitId' of external entity 'Stg_Circuit' is not supported." +// +// one per attribute. Confirmed on 11.12.1 against a real contract import: three +// CE6612 before the fix, none after. +func TestExternalEntity_AttributeRemoteMappingRoundTrip(t *testing.T) { + proj, modID := externalEntityFixture(t, func(e *domainmodel.Entity) { + e.Attributes = []*domainmodel.Attribute{{ + Name: "ProductName", + Type: &domainmodel.StringAttributeType{Length: 120}, + RemoteName: "Name", + RemoteType: "Edm.String", + Filterable: true, + Sortable: true, + Updatable: true, + }} + }) + got := readEntity(t, proj, modID, "Products") + + if len(got.Attributes) != 1 { + t.Fatalf("got %d attributes, want 1", len(got.Attributes)) + } + a := got.Attributes[0] + if a.RemoteName != "Name" { + t.Errorf("RemoteName = %q, want Name — the OData mapping did not survive the read", a.RemoteName) + } + if a.RemoteType != "Edm.String" { + t.Errorf("RemoteType = %q, want Edm.String", a.RemoteType) + } + // The per-attribute capability flags live on the same ODataMappedValue and + // are equally lost if the case arm is missing. + if !a.Filterable || !a.Sortable || !a.Updatable { + t.Errorf("capability flags lost: filterable=%v sortable=%v updatable=%v", + a.Filterable, a.Sortable, a.Updatable) + } +} From 3198948bc9c8758c2a2c220577fffc4df315ae4d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:06:32 +0000 Subject: [PATCH 09/31] fix(exec): refuse the XPath constraints check already rejects (#833) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mxcli check` reported MDL048 for `where [id = $GuidText]` and `mxcli exec` wrote the microflow anyway, so a script that skips check produced a project the build fails with CE0161. The cause is two validators: the exec path ran ValidateMicroflowBody (semantic errors), while the MDL0xx rule set lives in ValidateMicroflow, wired only into cmd_check.go and the LSP. Same shape as #836, where a guard existed on every exec path but was never reached from validate. Enforced at exec via an explicit allowlist of rules whose claims were verified against mxbuild 11.6.6 — MDL047, MDL048 and MDL055, all XPath-constraint rules whose constructs were built and confirmed to fail CE0161. Blanket promotion of all 17 error-severity rules was implemented first and then reverted, because it makes every rule a write barrier and at least one rule is wrong: MDL009 ("enumeration splits require exactly one value per branch") is a FALSE POSITIVE — a multi-value branch covering every enum value builds at 0 errors, and the shipped write-microflows skill documents exactly that form. It also broke an existing test whose fixture uses `else` on an enum split. MDL008 is by contrast correct (mxbuild reports CE0079 per uncovered value plus CE0773), which is the point: the two rules look alike and only a real build tells them apart. A test now fails if the allowlist is widened without that check. Placed in the create handler rather than validateWithContext, so `check --references` does not report each violation twice. Warnings are never promoted — check itself passes with them. Fixes #833 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- .claude/skills/fix-issue.md | 1 + mdl/executor/cmd_microflows_create.go | 9 ++ mdl/executor/validate.go | 56 +++++++ .../validate_microflow_rules_exec_test.go | 148 ++++++++++++++++++ 4 files changed, 214 insertions(+) create mode 100644 mdl/executor/validate_microflow_rules_exec_test.go diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 1928c8590..15f6d0aa1 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -393,3 +393,4 @@ extracting `OffsetExpression`/`LimitExpression`. | A sidebar navigation label renders truncated — "All tasks" shows as "All task" — under every theme (`signal`, `ledger`, `console`) and both variants, so switching theme does not help. Measured on the live DOM as `scrollWidth=56` inside `clientWidth=48` | Atlas's **closed** sidebar is an icon rail: `--navsidebar-width-closed: 48px` in Atlas's own `themesource/atlas_core/web/themes/_theme-default.scss`. The label is wider than the rail, and the closed rail expects an *icon*, not text. No mxcli theme sets any navigation width — the themes map colours — which is exactly why every theme reproduces it | `.claude/skills/mendix/theme-styling.md` (documented; no code change) | **A fix was written, verified, and reverted** — record the reason: adding `text-overflow: ellipsis` to the nav item helps only where Atlas also sets `white-space: nowrap`; elsewhere the label wraps to two readable lines, and the rule turns `All / tasks` into `All / t…`. Screenshotted both ways against the real compiled CSS. The answer belongs to the app (give nav items icons — what the rail is for — or keep the sidebar open), not to a theme that would impose it on every app. **Generalisable**: when a reported symptom traces to an upstream layout constant, reproduce the geometry against the real compiled CSS (48px container, real class names, Playwright measurement) — it takes minutes, tells you whose constant it is, and shows when the "obvious" CSS fix is a regression. mxcli-todo #19d | | `create non-persistent entity X ( A: String not null error '…' )` (or `unique`) passes `mxcli check` AND `mxcli exec`, then the build fails **CE0070** "Validations rules are not allowed on entity 'X', because it is not persistable" | `not null` / `unique` ARE validation rules — Studio Pro models "required" and "uniqueness" as rules on the entity, not as column constraints — so Mendix rejects them on a non-persistable entity. Nothing in mxcli connected the attribute constraint to the entity's persistence kind | `mdl/executor/cmd_enumerations.go` (`validateNPEValidationRules`, called from `ValidateEntity`) | Add **MDL054**, error severity, fired from the CREATE path only. **Establish the construct matrix against mxbuild before writing the rule, not from the issue text** — verified on 11.6.6 that `not null` with a message, `not null` bare, AND `unique` each produce CE0070 while a plain attribute does not, so the bare form (easy to miss, since the reporter only showed the message form) is flagged too. The CREATE path is the only one that can run this: `ALTER ENTITY … ADD ATTRIBUTE` does not carry the persistence kind, the same limitation MDL020 has. Sweep `mdl-examples/` + `scripts/check-skill-mdl.sh` after adding any error-severity rule — a false positive there breaks every user with that shape. Negative test `mdl-examples/bug-tests/832-npe-validation-rules.fail.mdl` (`.fail.mdl` = must fail check, enforced by `make check-mdl`) plus `-ok.mdl` pinning the other edge; tests `TestValidateEntityNPEValidationRules`. Issue #832 | | `retrieve $L from Mod.Entity where [Attr = $Var/Mod.Assoc/Attr]` passes `mxcli check` AND `mxcli exec`, then the build fails **CE0161** "Error(s) in XPath constraint" | Mendix XPath reaches at most ONE hop off a variable, and nothing checked the hop count | `mdl/executor/validate_microflow.go` (`checkXPathVariableTraversal`, `xpathVarTraversalRe`), called from the `*ast.RetrieveStmt` arm beside MDL047/MDL048 | Add **MDL055**, error severity, matching a `$var`-rooted path with **2+ segments**. **Establish the boundary against mxbuild first — it is narrower than it looks**: `$Var/Attr` VALID, `$Var/Mod.Assoc` VALID (one hop to the associated object), `$Var/Mod.Assoc/Attr` CE0161. A rule keying on "a module-qualified segment follows a variable" would reject the middle form, which is legal; key on hop count instead. Verified on 11.6.6 by building all three and dropping the offender to confirm the other two are clean. **Reject, don't try to serialize** — there is no valid XPath for the two-hop form, so the constraint must be restructured and only the author knows which way. **Verify the suggestion you emit**: both recommended rewrites (`retrieve $Related from $Var/Mod.Assoc;` then constrain on `$Related/Attr`; or invert to `[Mod.Assoc/Mod.Entity = $Var]`) were built and confirmed at 0 errors before the message claimed "both forms build clean". Negative test `mdl-examples/bug-tests/831-xpath-variable-traversal.fail.mdl` + `-ok.mdl`; test `TestXPathVariableTraversal`. Issue #831 | +| `mxcli check` reports a microflow rule (e.g. MDL048 `[id = $StringVar]`) but `mxcli exec` writes the same script silently — a script that skips `check` produces a project the build rejects | Two different validators. The exec path called `ValidateMicroflowBody` (semantic errors); the MDL0xx rule set lives in `ValidateMicroflow`, which was wired only into `cmd/mxcli/cmd_check.go` and the LSP. Same shape as #836, where a guard existed on every exec path but was never reached from validate | `mdl/executor/validate.go` (`validateMicroflowRules`, `execEnforcedMicroflowRules`) called from `mdl/executor/cmd_microflows_create.go` — the handler, mirroring `ValidateEntity` in `cmd_entities.go` (NOT `validateWithContext`, which would double-report under `check --references`) | Promote an **explicit allowlist** of rules verified against a real mxbuild, never the whole set. **Blanket promotion was tried and reverted**: it turns every error-severity rule into a write barrier, and `MDL009` ("enumeration splits require exactly one value per branch") is a **false positive** — a multi-value branch covering every enum value builds at 0 errors on 11.6.6, and the shipped `write-microflows` skill documents that very form. It also broke an existing repo test whose fixture uses `else` on an enum split. `MDL008` by contrast IS correct (mxbuild: CE0079 per uncovered value + CE0773) — so before promoting any rule, build its construct and read the verdict. Warnings are never promoted. Tests `TestValidateMicroflowRules_ReachedFromExec` and `…_UnverifiedRulesNotPromoted` (the latter fails if the allowlist is widened carelessly). Issue #833 | diff --git a/mdl/executor/cmd_microflows_create.go b/mdl/executor/cmd_microflows_create.go index 227b787dc..d48a134bc 100644 --- a/mdl/executor/cmd_microflows_create.go +++ b/mdl/executor/cmd_microflows_create.go @@ -42,6 +42,15 @@ func execCreateMicroflow(ctx *ExecContext, s *ast.CreateMicroflowStmt) error { return mdlerrors.NewValidation("microflow name must not be empty") } + // Refuse the XPath constraints Mendix rejects, before writing anything. + // `mxcli check` already reported these, but exec ran a different validator + // and wrote them anyway, so a script that skipped check produced a project + // the build fails on (issue #833). Same placement as the entity handler's + // ValidateEntity call. + if err := validateMicroflowRules(s); err != nil { + return err + } + // Find or auto-create module module, err := findOrCreateModule(ctx, s.Name.Module) if err != nil { diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index 2b81d983b..e58b7d281 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -12,6 +12,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/linter" "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/domainmodel" ) @@ -955,3 +956,58 @@ func getErrorHandlerBody(stmt ast.MicroflowStatement) []ast.MicroflowStatement { } return nil } + +// execEnforcedMicroflowRules are the MDL rules `mxcli exec` refuses to write, +// not just report. Membership requires that the rule's claim has been verified +// against a real mxbuild — a rule that is merely plausible must not become a +// hard write barrier. +// +// All three are XPath-constraint rules whose constructs were built and confirmed +// to fail CE0161: +// +// MDL047 [Mod.Assoc = empty] — no `= empty` for an association +// MDL048 [id = $StringVar] — no id operator from an expression +// MDL055 [Attr = $Var/Mod.Assoc/Attr] — at most one hop off a variable +// +// The rest of the MDL0xx set stays check-only deliberately. Promoting all 17 +// error-severity rules was tried and rejected: MDL009 ("enumeration splits +// require exactly one value per branch") is a FALSE POSITIVE — a multi-value +// branch covering every enum value builds at 0 errors on 11.6.6, and the +// shipped write-microflows skill documents that form — so promoting the set +// wholesale would have made exec refuse valid MDL. Verify a rule before adding +// it here. +var execEnforcedMicroflowRules = map[string]bool{ + "MDL047": true, + "MDL048": true, + "MDL055": true, +} + +// validateMicroflowRules runs the MDL0xx microflow rule set (ValidateMicroflow) +// on the exec path and turns the verified subset's ERROR-severity violations +// into a failure, so `mxcli exec` refuses to write what those rules reject. +// +// Before this, ValidateMicroflow was wired only into cmd_check.go and the LSP; +// the exec path ran ValidateMicroflowBody, a different validator with a +// different rule set, so a script that skipped `check` wrote microflows the +// build would reject (issue #833, reported via MDL048). +// +// Warnings are never promoted: they are advisory and `check` itself passes with +// them. The rule ID is included so an exec failure matches what `check` prints. +func validateMicroflowRules(stmt *ast.CreateMicroflowStmt) error { + var msgs []string + for _, v := range ValidateMicroflow(stmt) { + if v.Severity != linter.SeverityError || !execEnforcedMicroflowRules[v.RuleID] { + continue + } + msg := fmt.Sprintf("[%s] %s", v.RuleID, v.Message) + if v.Suggestion != "" { + msg += "\n " + v.Suggestion + } + msgs = append(msgs, msg) + } + if len(msgs) == 0 { + return nil + } + return mdlerrors.NewValidationf("microflow '%s' has validation errors:\n - %s", + stmt.Name.String(), strings.Join(msgs, "\n - ")) +} diff --git a/mdl/executor/validate_microflow_rules_exec_test.go b/mdl/executor/validate_microflow_rules_exec_test.go new file mode 100644 index 000000000..a31fa7ef4 --- /dev/null +++ b/mdl/executor/validate_microflow_rules_exec_test.go @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// TestValidateMicroflowRules_ReachedFromExec guards issue #833, which is the +// same shape as #836: a rule that exists and fires in `mxcli check` but is +// never reached from the exec path, so `exec` writes the very construct +// `check` rejects. +// +// ValidateMicroflow (the MDL0xx rule set) was wired only into cmd_check.go and +// the LSP. The exec path called ValidateMicroflowBody, a different function +// with a different rule set, so all 17 error-severity microflow rules were +// check-only. #833 reported it through MDL048 (`[id = $StringVar]`), but the +// gap was never specific to that rule. +// +// Only a VERIFIED subset is promoted (execEnforcedMicroflowRules). Blanket +// promotion was tried and rejected — MDL009 is a false positive, so making the +// whole set a write barrier would refuse valid MDL. Warnings are never promoted. +func TestValidateMicroflowRules_ReachedFromExec(t *testing.T) { + cases := []struct { + name string + src string + wantErr string // substring; "" means exec-validation must accept + }{ + { + // MDL048: comparing the object id against a String value. + name: "id compared to a string variable is rejected", + src: `create microflow M.ACT ($GuidText: String) returns M.Item +begin + retrieve $Found from M.Item where [id = $GuidText] limit 1; + return $Found; +end;`, + wantErr: "MDL048", + }, + { + // MDL055: two-hop traversal off a variable. + name: "variable association traversal is rejected", + src: `create microflow M.ACT ($P: M.Product) returns list of M.Category +begin + retrieve $L from M.Category where [Name = $P/M.Product_Category/Name]; + return $L; +end;`, + wantErr: "MDL055", + }, + { + // The valid counterpart of the same statement must still pass. + name: "one-hop traversal is accepted", + src: `create microflow M.ACT ($P: M.Product) returns list of M.Category +begin + retrieve $L from M.Category where [Name = $P/Code]; + return $L; +end;`, + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + prog, errs := visitor.Build(c.src) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + stmt := prog.Statements[0].(*ast.CreateMicroflowStmt) + err := validateMicroflowRules(stmt) + if c.wantErr == "" { + if err != nil { + t.Errorf("valid microflow rejected by exec validation: %v", err) + } + return + } + if err == nil { + t.Fatalf("expected exec validation to reject this (%s); `check` already does", c.wantErr) + } + if !strings.Contains(err.Error(), c.wantErr) { + t.Errorf("error should name the rule %s so it matches what check prints, got: %v", c.wantErr, err) + } + }) + } +} + +// A warning-severity rule must not block exec — only errors do. MDL001/MDL002 +// and friends are advisory, and turning them into hard exec failures would +// break scripts that check reports as passing. +func TestValidateMicroflowRules_WarningsDoNotBlockExec(t *testing.T) { + // MDL006 (warning): a loop with no body statements. + src := `create microflow M.ACT () returns Boolean +begin + declare $x integer = 1; + return true; +end;` + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + stmt := prog.Statements[0].(*ast.CreateMicroflowStmt) + // Whatever warnings this trips, none may become an exec error. + if err := validateMicroflowRules(stmt); err != nil { + t.Errorf("warning-only microflow must not fail exec validation, got: %v", err) + } +} + +// TestValidateMicroflowRules_UnverifiedRulesNotPromoted pins the deliberate +// narrowness of execEnforcedMicroflowRules. +// +// MDL009 ("enumeration splits require exactly one value per branch") fires in +// `check` but is a FALSE POSITIVE: verified on mxbuild 11.6.6, a multi-value +// branch that covers every enum value builds at 0 errors, and the shipped +// write-microflows skill documents exactly that form. If it were promoted, +// `exec` would refuse valid MDL — so this test fails the moment someone widens +// the allowlist without checking the rule against a real build. +func TestValidateMicroflowRules_UnverifiedRulesNotPromoted(t *testing.T) { + src := `create microflow M.ACT ($S: Enumeration(M.Status)) returns String +begin + case $S + when Open, Pending then + return 'a'; + when Closed then + return 'b'; + end case; +end;` + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + stmt := prog.Statements[0].(*ast.CreateMicroflowStmt) + + // check still reports it... + sawInCheck := false + for _, v := range ValidateMicroflow(stmt) { + if v.RuleID == "MDL009" { + sawInCheck = true + } + } + if !sawInCheck { + t.Skip("MDL009 no longer fires; if the rule was fixed or removed, drop this test") + } + // ...but exec must not refuse to write it. + if err := validateMicroflowRules(stmt); err != nil { + t.Errorf("MDL009 is a false positive (a multi-value branch covering every enum value "+ + "builds at 0 errors) and must not block exec, got: %v", err) + } +} From b9827f1dedeec0445d852c780b35d8e050fabba1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:15:34 +0000 Subject: [PATCH 10/31] fix(external-entities): honour the contract's capability annotations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CREATE EXTERNAL ENTITIES read names, types and navigation properties out of the contract correctly, then defaulted every capability to true regardless of what the contract said. Mendix compares the two at build time and refuses: 'Seasons' is marked Countable=False in the OData service, but True in the app. 'latitude' is marked Filterable=False in the OData service, but True in the app. Eight errors from an eight-resource import — on the one command whose whole job is fidelity to the contract. Insert/Update/Delete restrictions were already parsed; Count/Filter/Sort were not, so there was nothing for the import to honour. An unannotated set still means countable/filterable/sortable, which is OData's own default — silence is not a restriction. Verified on 11.12.1: a contract declaring CountRestrictions/Countable=false and NonFilterableProperties produced two CE6630 before, none after. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/executor/cmd_contract.go | 24 +++++++-- mdl/types/edmx.go | 27 ++++++++++ mdl/types/edmx_test.go | 95 ++++++++++++++++++++++++++++++++++++ 3 files changed, 143 insertions(+), 3 deletions(-) diff --git a/mdl/executor/cmd_contract.go b/mdl/executor/cmd_contract.go index 0d6357cd7..51fc15745 100644 --- a/mdl/executor/cmd_contract.go +++ b/mdl/executor/cmd_contract.go @@ -598,6 +598,13 @@ func createExternalEntities(ctx *ExecContext, s *ast.CreateExternalEntitiesStmt) } nonInsertable := make(map[string]bool) nonUpdatable := make(map[string]bool) + // Filter/Sort restrictions name the properties the service refuses to + // filter or sort on. Marking them filterable anyway is CE6630 + // ("'latitude' is marked Filterable=False in the OData service, but + // True in the app") — one per property, on an import whose whole job + // is to match the contract. + nonFilterable := make(map[string]bool) + nonSortable := make(map[string]bool) if entitySet != nil { for _, name := range entitySet.NonInsertableProperties { nonInsertable[name] = true @@ -605,6 +612,12 @@ func createExternalEntities(ctx *ExecContext, s *ast.CreateExternalEntitiesStmt) for _, name := range entitySet.NonUpdatableProperties { nonUpdatable[name] = true } + for _, name := range entitySet.NonFilterableProperties { + nonFilterable[name] = true + } + for _, name := range entitySet.NonSortableProperties { + nonSortable[name] = true + } } // Build attributes from merged properties @@ -641,8 +654,8 @@ func createExternalEntities(ctx *ExecContext, s *ast.CreateExternalEntitiesStmt) Type: edmToDomainModelAttrType(p, keyPropSet[p.Name]), RemoteName: p.Name, RemoteType: p.Type, - Filterable: true, - Sortable: true, + Filterable: !nonFilterable[p.Name], + Sortable: !nonSortable[p.Name], Creatable: creatable, Updatable: updatable, } @@ -1157,7 +1170,12 @@ func applyExternalEntityFields( ent.Source = "Rest$ODataRemoteEntitySource" ent.Persistable = true ent.RemoteEntitySet = entitySet.Name - ent.Countable = true + // Countable follows the contract when it says so. OData's own default is + // countable, so an unannotated set stays true — but a service that + // declares CountRestrictions/Countable=false and an app that says true is + // CE6630 ("marked Countable=False in the OData service, but True in the + // app"), which is the whole point of generating from $metadata. + ent.Countable = entitySet.Countable == nil || *entitySet.Countable // Capabilities default to false (Mendix's conservative read-only default) // when the entity set has no Insert/Delete restriction annotation — an // unannotated service is treated as read-only, and the app must match or diff --git a/mdl/types/edmx.go b/mdl/types/edmx.go index 15ac367d1..c04ed35e6 100644 --- a/mdl/types/edmx.go +++ b/mdl/types/edmx.go @@ -77,6 +77,14 @@ type EdmEntitySet struct { Insertable *bool // InsertRestrictions/Insertable Updatable *bool // UpdateRestrictions/Updatable Deletable *bool // DeleteRestrictions/Deletable + Countable *bool // CountRestrictions/Countable + + // Property names the service says cannot be filtered or sorted on, from + // FilterRestrictions/NonFilterableProperties and + // SortRestrictions/NonSortableProperties. Mendix compares these against the + // app's per-attribute flags and reports CE6630 on a mismatch. + NonFilterableProperties []string + NonSortableProperties []string // Navigation property names listed under // Org.OData.Capabilities.V1.{Insert,Update}Restrictions/Non*NavigationProperties. @@ -408,6 +416,25 @@ func applyCapabilityAnnotations(es *EdmEntitySet, annotations []xmlCapabilitiesA es.Deletable = &v } } + case "Org.OData.Capabilities.V1.CountRestrictions": + for _, pv := range ann.Record.PropertyValues { + if pv.Property == "Countable" && pv.Bool != "" { + v := pv.Bool == "true" + es.Countable = &v + } + } + case "Org.OData.Capabilities.V1.FilterRestrictions": + for _, pv := range ann.Record.PropertyValues { + if pv.Property == "NonFilterableProperties" && pv.Collection != nil { + es.NonFilterableProperties = pv.Collection.PropertyPaths + } + } + case "Org.OData.Capabilities.V1.SortRestrictions": + for _, pv := range ann.Record.PropertyValues { + if pv.Property == "NonSortableProperties" && pv.Collection != nil { + es.NonSortableProperties = pv.Collection.PropertyPaths + } + } } } } diff --git a/mdl/types/edmx_test.go b/mdl/types/edmx_test.go index 4814d7209..a0e63178b 100644 --- a/mdl/types/edmx_test.go +++ b/mdl/types/edmx_test.go @@ -398,3 +398,98 @@ func TestParseEdmx_ConcurrencyModeFixed(t *testing.T) { t.Error("ConcurrencyMode='Fixed' must set Computed=true so the attribute is not marked Creatable (issue #525)") } } + +// mxcli-formula1 findings #24: CREATE EXTERNAL ENTITIES read names, types and +// navigation properties out of the contract correctly, then defaulted every +// capability to true regardless of what the contract said. Mendix compares the +// two at build time and refuses: +// +// 'Seasons' is marked Countable=False in the OData service, but True in the app. +// 'latitude' is marked Filterable=False in the OData service, but True in the app. +// +// Insert/Update/Delete restrictions were already parsed; Count/Filter/Sort were +// not, so there was nothing for the import to honour. +func TestParseEdmx_CountFilterSortRestrictions(t *testing.T) { + const md = ` + + + + + + + + + + + + + + + + + latitude + + + + + altitude + + + + + + +` + + doc, err := ParseEdmx(md) + if err != nil { + t.Fatalf("ParseEdmx: %v", err) + } + if len(doc.EntitySets) != 1 { + t.Fatalf("got %d entity sets, want 1", len(doc.EntitySets)) + } + es := doc.EntitySets[0] + + if es.Countable == nil || *es.Countable { + t.Errorf("Countable = %v, want an explicit false", es.Countable) + } + if len(es.NonFilterableProperties) != 1 || es.NonFilterableProperties[0] != "latitude" { + t.Errorf("NonFilterableProperties = %v, want [latitude]", es.NonFilterableProperties) + } + if len(es.NonSortableProperties) != 1 || es.NonSortableProperties[0] != "altitude" { + t.Errorf("NonSortableProperties = %v, want [altitude]", es.NonSortableProperties) + } +} + +// A contract that says nothing must leave the capabilities unspecified, so the +// import keeps OData's own default (countable, filterable, sortable) rather than +// reading silence as a restriction. +func TestParseEdmx_NoRestrictionsLeavesCapabilitiesUnset(t *testing.T) { + const md = ` + + + + + + + + + + + + +` + + doc, err := ParseEdmx(md) + if err != nil { + t.Fatalf("ParseEdmx: %v", err) + } + es := doc.EntitySets[0] + if es.Countable != nil { + t.Errorf("Countable = %v, want nil (unspecified)", *es.Countable) + } + if len(es.NonFilterableProperties) != 0 || len(es.NonSortableProperties) != 0 { + t.Errorf("restrictions invented from an unannotated set: filter=%v sort=%v", + es.NonFilterableProperties, es.NonSortableProperties) + } +} From df1e52b08754a15fa1b53c9f0ed5933e7db4c8f2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:15:34 +0000 Subject: [PATCH 11/31] fix(odata): use the client's own credentials to fetch $metadata MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CREATE ODATA CLIENT accepts UseAuthentication / HttpUsername / HttpPassword and stores them for the runtime, but the design-time fetch was a bare client.Get. Against a service behind `authentication basic` that is a 401 — and since the failure is only a warning, the client is created with no cached entity types, so the CREATE EXTERNAL ENTITIES that follows imports nothing from a script that looks like it succeeded. The credentials and any HEADERS now go out with the fetch. Only literals can be used. The visitor strips a quoted literal's quotes, so 'f1api' and Module.ApiUser both arrive as bare strings; the AST now records which was written. A constant is resolved by the runtime, and sending its *name* as the password would be worse than sending nothing — so unresolved names are reported instead, alongside a note that the client was left empty and that pointing MetadataUrl at a committed contract file avoids the problem entirely. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/ast/ast_odata.go | 13 ++- mdl/executor/cmd_odata.go | 96 ++++++++++++++++++- mdl/executor/cmd_odata_metadata_auth_test.go | 97 ++++++++++++++++++++ mdl/executor/cmd_odata_test.go | 6 +- mdl/visitor/visitor_odata.go | 21 ++++- 5 files changed, 224 insertions(+), 9 deletions(-) create mode 100644 mdl/executor/cmd_odata_metadata_auth_test.go diff --git a/mdl/ast/ast_odata.go b/mdl/ast/ast_odata.go index 729a8a24a..158129ab7 100644 --- a/mdl/ast/ast_odata.go +++ b/mdl/ast/ast_odata.go @@ -24,7 +24,15 @@ type CreateODataClientStmt struct { UseAuthentication bool HttpUsername string // Mendix expression for username HttpPassword string // Mendix expression for password - ClientCertificate string + + // Whether the credential above was written as a quoted literal rather than + // a constant reference. The visitor strips a literal's quotes, so by the + // time it reaches the executor `'f1api'` and `Module.ApiUser` are both bare + // strings — and only the first is a value mxcli can use for the design-time + // $metadata fetch. A constant is resolved by the runtime, not by us. + HttpUsernameIsLiteral bool + HttpPasswordIsLiteral bool + ClientCertificate string // Microflow references. `ConfigurationMicroflow` (returns // System.ConsumedODataConfiguration) and `HeadersMicroflow` (returns a list @@ -52,6 +60,9 @@ type CreateODataClientStmt struct { type HeaderDef struct { Key string Value string // Mendix expression + // ValueIsLiteral mirrors HttpUsernameIsLiteral: a quoted literal can be sent + // on the design-time fetch, a constant reference cannot. + ValueIsLiteral bool } func (s *CreateODataClientStmt) isStatement() {} diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index 047be7d1f..b72bc5ef9 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -1127,9 +1127,15 @@ Got: %s`, stmt.ServiceUrl) } newSvc.MetadataUrl = normalizedUrl - metadata, hash, err := fetchODataMetadata(normalizedUrl) + auth := metadataAuthFromStmt(stmt) + metadata, hash, err := fetchODataMetadata(normalizedUrl, auth) if err != nil { fmt.Fprintf(ctx.Output, "Warning: could not fetch $metadata: %v\n", err) + for _, hint := range auth.hints() { + fmt.Fprintf(ctx.Output, " %s\n", hint) + } + fmt.Fprintf(ctx.Output, " The client is created with no cached entity types, so a following\n") + fmt.Fprintf(ctx.Output, " 'create external entities from %s.%s' will import nothing.\n", stmt.Name.Module, stmt.Name.Name) } else if metadata != "" { newSvc.Metadata = metadata newSvc.MetadataHash = hash @@ -1825,7 +1831,82 @@ func astEntityDefToModel(ctx *ExecContext, def *ast.PublishedEntityDef) (*model. // Returns the metadata XML and its SHA-256 hash, or empty strings if the fetch fails. // Note: metadataUrl is expected to be already normalized by NormalizeURL() in createODataClient, // so all relative paths have been converted to absolute file:// URLs. -func fetchODataMetadata(metadataUrl string) (metadata string, hash string, err error) { +// metadataFetchAuth carries the credentials and headers used for the +// design-time $metadata fetch. +// +// Only *literal* values are usable here. MDL stores a quoted literal with its +// quotes ('f1api') and a constant reference bare (Module.ApiUser); at design +// time mxcli has no runtime to resolve a constant against, so a reference is +// reported rather than sent as if it were the value itself. +type metadataFetchAuth struct { + Username string // literal, quotes already stripped + Password string // literal, quotes already stripped + Headers map[string]string // literal values only + // Unresolved names the caller referenced by constant, for the message. + Unresolved []string +} + +// apply sets basic auth and headers on the request. A nil receiver is a no-op, +// so an unauthenticated fetch needs no special case at the call site. +func (a *metadataFetchAuth) apply(req *http.Request) { + if a == nil { + return + } + if a.Username != "" || a.Password != "" { + req.SetBasicAuth(a.Username, a.Password) + } + for k, v := range a.Headers { + req.Header.Set(k, v) + } +} + +// metadataAuthFromStmt collects the statement's own credentials and headers for +// the design-time fetch, keeping only the literals. +func metadataAuthFromStmt(stmt *ast.CreateODataClientStmt) *metadataFetchAuth { + auth := &metadataFetchAuth{Headers: map[string]string{}} + switch { + case stmt.HttpUsernameIsLiteral: + auth.Username = stmt.HttpUsername + case stmt.HttpUsername != "": + auth.Unresolved = append(auth.Unresolved, "HttpUsername ("+stmt.HttpUsername+")") + } + switch { + case stmt.HttpPasswordIsLiteral: + auth.Password = stmt.HttpPassword + case stmt.HttpPassword != "": + // Named, not printed: a constant reference is a name, but the value it + // resolves to is a secret and this line goes to the console. + auth.Unresolved = append(auth.Unresolved, "HttpPassword ("+stmt.HttpPassword+")") + } + for _, h := range stmt.Headers { + switch { + case h.ValueIsLiteral: + auth.Headers[h.Key] = h.Value + case h.Value != "": + auth.Unresolved = append(auth.Unresolved, "header "+h.Key+" ("+h.Value+")") + } + } + sort.Strings(auth.Unresolved) + return auth +} + +// hints explains a failed fetch when the reason is credentials mxcli could not +// resolve, and points at the workaround that also happens to be better practice. +func (a *metadataFetchAuth) hints() []string { + var out []string + if a == nil { + return out + } + if len(a.Unresolved) > 0 { + out = append(out, "These are constant references, which only the runtime can resolve, so the fetch went out without them: "+strings.Join(a.Unresolved, ", ")) + } + out = append(out, + "Fetch the contract once and point MetadataUrl at the file — it commits, so the", + "model rebuilds without the service running and a contract change is a reviewable diff.") + return out +} + +func fetchODataMetadata(metadataUrl string, auth *metadataFetchAuth) (metadata string, hash string, err error) { if metadataUrl == "" { return "", "", nil } @@ -1847,7 +1928,16 @@ func fetchODataMetadata(metadataUrl string) (metadata string, hash string, err e } else { // HTTP(S) fetch client := &http.Client{Timeout: 30 * time.Second} - resp, err := client.Get(metadataUrl) + req, reqErr := http.NewRequest(http.MethodGet, metadataUrl, nil) + if reqErr != nil { + return "", "", mdlerrors.NewBackend(fmt.Sprintf("build $metadata request for %s", metadataUrl), reqErr) + } + // The credentials and headers already on the statement apply to this + // fetch too. Without them a service behind `authentication basic` + // answers 401, the client is created with no cached entity types, and + // the CREATE EXTERNAL ENTITIES that follows silently imports nothing. + auth.apply(req) + resp, err := client.Do(req) if err != nil { return "", "", mdlerrors.NewBackend(fmt.Sprintf("fetch $metadata from %s", metadataUrl), err) } diff --git a/mdl/executor/cmd_odata_metadata_auth_test.go b/mdl/executor/cmd_odata_metadata_auth_test.go new file mode 100644 index 000000000..455286d29 --- /dev/null +++ b/mdl/executor/cmd_odata_metadata_auth_test.go @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "net/http" + "net/http/httptest" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// mxcli-formula1 findings #23: CREATE ODATA CLIENT accepts UseAuthentication / +// HttpUsername / HttpPassword and stores them for the runtime, but the +// design-time $metadata fetch was a bare client.Get. Against a service behind +// `authentication basic` that is a 401, and because the fetch failure is only a +// warning the client is created with no cached entity types — so the +// CREATE EXTERNAL ENTITIES that follows imports nothing and the script looks +// like it succeeded. +func TestFetchODataMetadata_SendsCredentialsAndHeaders(t *testing.T) { + const body = `` + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + user, pass, ok := r.BasicAuth() + if !ok || user != "f1api" || pass != "s3cret" { + w.WriteHeader(http.StatusUnauthorized) + return + } + if r.Header.Get("X-Probe") != "yes" { + w.WriteHeader(http.StatusForbidden) + return + } + _, _ = w.Write([]byte(body)) + })) + defer srv.Close() + + auth := &metadataFetchAuth{ + Username: "f1api", + Password: "s3cret", + Headers: map[string]string{"X-Probe": "yes"}, + } + got, hash, err := fetchODataMetadata(srv.URL, auth) + if err != nil { + t.Fatalf("fetch with credentials failed: %v", err) + } + if got != body { + t.Errorf("body = %q, want the served metadata", got) + } + if hash == "" { + t.Error("no hash computed for a successful fetch") + } + + // Without them, the same service is a 401 — which is what shipped. + if _, _, err := fetchODataMetadata(srv.URL, nil); err == nil { + t.Error("unauthenticated fetch succeeded, so the test server proves nothing") + } +} + +// A literal is usable at design time; a constant reference is not — the runtime +// resolves those, mxcli has nothing to resolve them against. Sending the +// constant's *name* as the password would be worse than sending nothing, so the +// name is reported instead. +func TestMetadataAuthFromStmt_LiteralsOnly(t *testing.T) { + stmt := &ast.CreateODataClientStmt{ + HttpUsername: "f1api", + HttpUsernameIsLiteral: true, + HttpPassword: "Module.ApiPassword", // a constant reference + Headers: []ast.HeaderDef{ + {Key: "X-Probe", Value: "yes", ValueIsLiteral: true}, + {Key: "X-Token", Value: "Module.Token"}, + }, + } + auth := metadataAuthFromStmt(stmt) + + if auth.Username != "f1api" { + t.Errorf("Username = %q, want the literal f1api", auth.Username) + } + if auth.Password != "" { + t.Errorf("Password = %q, want empty — a constant reference is not a value", auth.Password) + } + if auth.Headers["X-Probe"] != "yes" { + t.Errorf("literal header dropped: %v", auth.Headers) + } + if _, ok := auth.Headers["X-Token"]; ok { + t.Error("a constant-reference header was sent as its own name") + } + // Both unresolved names are reported, sorted, so the user learns why the + // fetch went out unauthenticated. + want := []string{"HttpPassword (Module.ApiPassword)", "header X-Token (Module.Token)"} + if len(auth.Unresolved) != len(want) { + t.Fatalf("Unresolved = %v, want %v", auth.Unresolved, want) + } + for i := range want { + if auth.Unresolved[i] != want[i] { + t.Errorf("Unresolved[%d] = %q, want %q", i, auth.Unresolved[i], want[i]) + } + } +} diff --git a/mdl/executor/cmd_odata_test.go b/mdl/executor/cmd_odata_test.go index 9df6b8f00..166dde9e8 100644 --- a/mdl/executor/cmd_odata_test.go +++ b/mdl/executor/cmd_odata_test.go @@ -47,7 +47,7 @@ func TestFetchODataMetadata_LocalFile(t *testing.T) { for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - metadata, hash, err := fetchODataMetadata(tt.url) + metadata, hash, err := fetchODataMetadata(tt.url, nil) if tt.wantErr { if err == nil { @@ -72,7 +72,7 @@ func TestFetchODataMetadata_LocalFile(t *testing.T) { } // Hash should be consistent - _, hash2, _ := fetchODataMetadata(tt.url) + _, hash2, _ := fetchODataMetadata(tt.url, nil) if hash != hash2 { t.Errorf("Hash inconsistent between calls: %q vs %q", hash, hash2) } @@ -182,7 +182,7 @@ func TestFetchODataMetadata_LocalFileAbsolute(t *testing.T) { fileURL = "file:///" + filepath.ToSlash(filePath) } - metadata, hash, err := fetchODataMetadata(fileURL) + metadata, hash, err := fetchODataMetadata(fileURL, nil) if err != nil { t.Errorf("Unexpected error: %v", err) } diff --git a/mdl/visitor/visitor_odata.go b/mdl/visitor/visitor_odata.go index b45380676..519dac706 100644 --- a/mdl/visitor/visitor_odata.go +++ b/mdl/visitor/visitor_odata.go @@ -45,8 +45,10 @@ func (b *Builder) ExitCreateODataClientStatement(ctx *parser.CreateODataClientSt stmt.UseAuthentication = strings.EqualFold(value, "true") || strings.EqualFold(value, "yes") case "httpusername": stmt.HttpUsername = value + stmt.HttpUsernameIsLiteral = odataValueIsLiteral(prop) case "httppassword": stmt.HttpPassword = value + stmt.HttpPasswordIsLiteral = odataValueIsLiteral(prop) case "clientcertificate": stmt.ClientCertificate = value case "configurationmicroflow": @@ -296,6 +298,18 @@ func odataValueText(val *parser.OdataPropertyValueContext) string { return "" } +// odataValueIsLiteral reports whether an OData property value was written as a +// quoted string rather than a constant reference. odataValueText strips a +// literal's quotes, so this is the only thing that still tells the two apart — +// and mxcli can only use a literal for the design-time $metadata fetch. +func odataValueIsLiteral(prop *parser.OdataPropertyAssignmentContext) bool { + valCtx := prop.OdataPropertyValue() + if valCtx == nil { + return false + } + return valCtx.(*parser.OdataPropertyValueContext).STRING_LITERAL() != nil +} + // odataAssignmentValueText extracts the string value from an OData property assignment. func odataAssignmentValueText(prop *parser.OdataPropertyAssignmentContext) string { valCtx := prop.OdataPropertyValue() @@ -390,10 +404,13 @@ func parseODataHeaders(ctx parser.IOdataHeadersClauseContext) []ast.HeaderDef { entry := entryCtx.(*parser.OdataHeaderEntryContext) key := unquoteString(entry.STRING_LITERAL().GetText()) value := "" + isLiteral := false if valCtx := entry.OdataPropertyValue(); valCtx != nil { - value = odataValueText(valCtx.(*parser.OdataPropertyValueContext)) + vc := valCtx.(*parser.OdataPropertyValueContext) + value = odataValueText(vc) + isLiteral = vc.STRING_LITERAL() != nil } - headers = append(headers, ast.HeaderDef{Key: key, Value: value}) + headers = append(headers, ast.HeaderDef{Key: key, Value: value, ValueIsLiteral: isLiteral}) } return headers From a32fde9119427710d95912f6993c7245d57099eb Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:15:35 +0000 Subject: [PATCH 12/31] fix(microflows): pass a dynamic query expression through unquoted MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `execute database query … dynamic $Sql` reached the runtime as the string literal '$Sql', so the database was asked to execute four characters: ERROR - ExternalDatabaseConnector: Parser Error: syntax error at or near "$" The builder quoted anything not already starting with a quote — correct for `dynamic 'SELECT …'`, wrong for an expression — and the AST kept no literal-vs-expression flag, so it could not tell them apart. That blocked runtime-built SQL, and therefore query pushdown, outright. Verified by reading the stored BSON: DynamicQuery now holds $Sql, not '$Sql'. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/ast/ast_microflow.go | 19 +++--- mdl/executor/cmd_microflows_builder_calls.go | 25 ++++++-- .../cmd_microflows_dynamic_query_test.go | 60 +++++++++++++++++++ mdl/visitor/visitor_microflow_actions.go | 1 + 4 files changed, 93 insertions(+), 12 deletions(-) create mode 100644 mdl/executor/cmd_microflows_dynamic_query_test.go diff --git a/mdl/ast/ast_microflow.go b/mdl/ast/ast_microflow.go index ac4f48b4d..3f7af75e2 100644 --- a/mdl/ast/ast_microflow.go +++ b/mdl/ast/ast_microflow.go @@ -452,13 +452,18 @@ func (s *CallWebServiceStmt) isMicroflowStatement() {} // ExecuteDatabaseQueryStmt represents: EXECUTE DATABASE QUERY Module.Connection.QueryName ... type ExecuteDatabaseQueryStmt struct { - OutputVariable string // Optional output variable - QueryName string // Full 3-part identifier: Module.Connection.QueryName - DynamicQuery string // Optional dynamic SQL override - Arguments []CallArgument // Parameter mappings (query parameters) - ConnectionArguments []CallArgument // Connection parameter mappings (runtime connection override) - ErrorHandling *ErrorHandlingClause // Optional ON ERROR clause - Annotations *ActivityAnnotations // Optional @position, @caption, @color, @annotation + OutputVariable string // Optional output variable + QueryName string // Full 3-part identifier: Module.Connection.QueryName + DynamicQuery string // Optional dynamic SQL override + // DynamicQueryIsExpression distinguishes `dynamic $Sql` from `dynamic 'SELECT …'`. + // Both reach the executor as a bare string, and the builder has to quote one + // and not the other: quoting an expression sends the literal text `$Sql` to + // the database, which is a syntax error at the far end, not a Mendix one. + DynamicQueryIsExpression bool + Arguments []CallArgument // Parameter mappings (query parameters) + ConnectionArguments []CallArgument // Connection parameter mappings (runtime connection override) + ErrorHandling *ErrorHandlingClause // Optional ON ERROR clause + Annotations *ActivityAnnotations // Optional @position, @caption, @color, @annotation } func (s *ExecuteDatabaseQueryStmt) isMicroflowStatement() {} diff --git a/mdl/executor/cmd_microflows_builder_calls.go b/mdl/executor/cmd_microflows_builder_calls.go index 380378585..cc67c7240 100644 --- a/mdl/executor/cmd_microflows_builder_calls.go +++ b/mdl/executor/cmd_microflows_builder_calls.go @@ -1344,13 +1344,28 @@ func buildRestParameterMappings( return pathMappings, queryMappings } +// dynamicQueryExpression renders the statement's dynamic query as the Mendix +// expression the action stores. +// +// A literal SQL string has to be quoted, because the field holds an expression — +// but an expression must be passed through untouched. Quoting `$Sql` sends the +// four characters "$Sql" to the database: +// +// ERROR - ExternalDatabaseConnector: Parser Error: syntax error at or near "$" +// +// which blocks runtime-built SQL entirely. The two spellings are only +// distinguishable at parse time, hence DynamicQueryIsExpression. +func dynamicQueryExpression(s *ast.ExecuteDatabaseQueryStmt) string { + q := s.DynamicQuery + if q == "" || s.DynamicQueryIsExpression || strings.HasPrefix(q, "'") { + return q + } + return "'" + strings.ReplaceAll(q, "'", "''") + "'" +} + // addExecuteDatabaseQueryAction creates an EXECUTE DATABASE QUERY statement. func (fb *flowBuilder) addExecuteDatabaseQueryAction(s *ast.ExecuteDatabaseQueryStmt) model.ID { - // DynamicQuery is a Mendix expression — string literals need single quotes - dynamicQuery := s.DynamicQuery - if dynamicQuery != "" && !strings.HasPrefix(dynamicQuery, "'") { - dynamicQuery = "'" + strings.ReplaceAll(dynamicQuery, "'", "''") + "'" - } + dynamicQuery := dynamicQueryExpression(s) action := µflows.ExecuteDatabaseQueryAction{ BaseElement: model.BaseElement{ID: model.ID(types.GenerateID())}, diff --git a/mdl/executor/cmd_microflows_dynamic_query_test.go b/mdl/executor/cmd_microflows_dynamic_query_test.go new file mode 100644 index 000000000..f3d2723d6 --- /dev/null +++ b/mdl/executor/cmd_microflows_dynamic_query_test.go @@ -0,0 +1,60 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// mxcli-formula1 findings #21: `execute database query … dynamic $Sql` reached +// the runtime as the string literal '$Sql', so DuckDB was asked to execute the +// four characters: +// +// ERROR - ExternalDatabaseConnector: Parser Error: syntax error at or near "$" +// +// The builder quoted anything not already starting with a quote, and the AST +// kept no literal-vs-expression flag, so it could not tell them apart. This +// blocked runtime-built SQL — query pushdown — outright. +func TestDynamicQueryExpressionIsNotQuoted(t *testing.T) { + cases := []struct { + name string + stmt *ast.ExecuteDatabaseQueryStmt + want string + }{ + { + "a variable passes through untouched", + &ast.ExecuteDatabaseQueryStmt{DynamicQuery: "$Sql", DynamicQueryIsExpression: true}, + "$Sql", + }, + { + "so does a built expression", + &ast.ExecuteDatabaseQueryStmt{DynamicQuery: "'SELECT * FROM t LIMIT ' + toString($Limit)", DynamicQueryIsExpression: true}, + "'SELECT * FROM t LIMIT ' + toString($Limit)", + }, + { + "a literal is still quoted, because the field holds an expression", + &ast.ExecuteDatabaseQueryStmt{DynamicQuery: "SELECT * FROM t"}, + "'SELECT * FROM t'", + }, + { + "a literal's own quotes are doubled, Mendix-style", + &ast.ExecuteDatabaseQueryStmt{DynamicQuery: "SELECT 'a'"}, + "'SELECT ''a'''", + }, + { + "no dynamic query stays empty rather than becoming two quotes", + &ast.ExecuteDatabaseQueryStmt{}, + "", + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := dynamicQueryExpression(tc.stmt); got != tc.want { + t.Errorf("got %q, want %q", got, tc.want) + } + }) + } +} diff --git a/mdl/visitor/visitor_microflow_actions.go b/mdl/visitor/visitor_microflow_actions.go index fa47a2217..fba6bedff 100644 --- a/mdl/visitor/visitor_microflow_actions.go +++ b/mdl/visitor/visitor_microflow_actions.go @@ -574,6 +574,7 @@ func buildExecuteDatabaseQueryStatement(ctx parser.IExecuteDatabaseQueryStatemen stmt.DynamicQuery = unquoteDollarString(ds.GetText()) } else if expr := execCtx.Expression(); expr != nil { stmt.DynamicQuery = expr.GetText() + stmt.DynamicQueryIsExpression = true } } From 63f72e02ba11c8f5d03fb8d896b1c69f71a8b450 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:18:52 +0000 Subject: [PATCH 13/31] fix(check): advertise the three OData properties MDL-ODATA01 forgot MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The hint said "Known properties here: ReadMode, InsertMode, UpdateMode, DeleteMode, UsePaging, PageSize" long after the visitor learned Countable, SkipSupported and TopSupported — so a user typing an accepted property was told it was unknown. The lists are separate by design (the visitor decides, the hint displays), but nothing kept them in step. The AST struct is now the source: every field of PublishedEntityDef and CreateExternalEntityStmt must be advertised or explicitly listed as structural, so adding a property and forgetting the hint is a test failure instead of a wrong message. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/executor/validate_odata_properties.go | 6 + .../validate_odata_properties_drift_test.go | 143 ++++++++++++++++++ 2 files changed, 149 insertions(+) create mode 100644 mdl/executor/validate_odata_properties_drift_test.go diff --git a/mdl/executor/validate_odata_properties.go b/mdl/executor/validate_odata_properties.go index eb666627a..9119ce961 100644 --- a/mdl/executor/validate_odata_properties.go +++ b/mdl/executor/validate_odata_properties.go @@ -21,6 +21,11 @@ import ( // Known property names, in the spelling the syntax help uses. These are for the // error message only — the visitor is the authority on what is accepted, and it // matches case-insensitively. +// +// The two drifted once already: Countable/SkipSupported/TopSupported were added +// to the visitor and the hint went on advertising six properties, so a user +// reading it would think three accepted properties were not. TestKnownODataProps +// keeps them in step by running every name below through the visitor. var ( knownODataServiceProps = []string{ "Path", "Version", "ODataVersion", "Namespace", "ServiceName", @@ -28,6 +33,7 @@ var ( } knownPublishEntityProps = []string{ "ReadMode", "InsertMode", "UpdateMode", "DeleteMode", "UsePaging", "PageSize", + "Countable", "SkipSupported", "TopSupported", } knownODataClientProps = []string{ "Version", "ODataVersion", "MetadataUrl", "Timeout", "ProxyType", diff --git a/mdl/executor/validate_odata_properties_drift_test.go b/mdl/executor/validate_odata_properties_drift_test.go new file mode 100644 index 000000000..08cef446a --- /dev/null +++ b/mdl/executor/validate_odata_properties_drift_test.go @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "reflect" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// MDL-ODATA01's hint listed six publish-entity properties long after the visitor +// grew three more (Countable / SkipSupported / TopSupported), so the message told +// users that accepted properties were unknown. The lists are separate by design — +// the visitor decides, the hint only displays — but they must not drift, and +// nothing was checking (mxcli-formula1 #15). +// +// The visitor is the authority: every name the hint advertises is fed through it +// and must not come back as unknown. +func TestKnownODataProps_MatchTheVisitor(t *testing.T) { + cases := []struct { + what string + props []string + build func(prop string) string + }{ + {"odata service", knownODataServiceProps, func(p string) string { + return fmt.Sprintf("create odata service M.S (%s: 'x');", p) + }}, + {"publish entity", knownPublishEntityProps, func(p string) string { + return fmt.Sprintf("create odata service M.S (Path: 'p/')\n{\n publish entity M.E as 'Es' (%s: 'x')\n expose (A);\n};", p) + }}, + {"odata client", knownODataClientProps, func(p string) string { + return fmt.Sprintf("create odata client M.C (%s: 'x');", p) + }}, + {"external entity", knownExternalEntityProps, func(p string) string { + return fmt.Sprintf("create external entity M.E from odata client M.C (%s: 'x');", p) + }}, + } + + for _, tc := range cases { + for _, prop := range tc.props { + t.Run(tc.what+"/"+prop, func(t *testing.T) { + prog, errs := visitor.Build(tc.build(prop)) + if len(errs) > 0 { + t.Fatalf("%q does not parse in a %s: %v", prop, tc.what, errs) + } + if unknown := collectUnknownProps(prog); len(unknown) > 0 { + t.Errorf("the hint advertises %q but the visitor discards it (unknown: %v)", prop, unknown) + } + }) + } + } +} + +// The direction that actually broke: a property the AST carries but the hint does +// not advertise. Countable/SkipSupported/TopSupported were added as fields, the +// visitor learned to set them, and the hint was never updated — so the message +// told users three accepted properties were unknown. +// +// The AST struct is the single source here: every field is a property unless it +// is listed as structural, so adding one and forgetting the hint fails this test +// rather than shipping a wrong message. +func TestKnownODataProps_CoverEveryASTField(t *testing.T) { + cases := []struct { + what string + typ reflect.Type + advertised []string + structural map[string]bool + }{ + { + "publish entity", reflect.TypeOf(ast.PublishedEntityDef{}), knownPublishEntityProps, + map[string]bool{"Entity": true, "ExposedName": true, "Members": true, "UnknownProperties": true}, + }, + { + "external entity", reflect.TypeOf(ast.CreateExternalEntityStmt{}), knownExternalEntityProps, + map[string]bool{ + "Name": true, "ServiceRef": true, "Attributes": true, "Documentation": true, + "CreateOrModify": true, "UnknownProperties": true, + }, + }, + } + + for _, tc := range cases { + t.Run(tc.what, func(t *testing.T) { + have := map[string]bool{} + for _, p := range tc.advertised { + have[strings.ToLower(p)] = true + } + for i := 0; i < tc.typ.NumField(); i++ { + f := tc.typ.Field(i) + if tc.structural[f.Name] || !f.IsExported() { + continue + } + // Flags that only record whether a sibling was set are not + // properties in their own right. + if strings.HasSuffix(f.Name, "IsLiteral") || strings.HasSuffix(f.Name, "Set") || + strings.HasSuffix(f.Name, "IsExpression") { + continue + } + if !have[strings.ToLower(f.Name)] { + t.Errorf("%s carries %s but MDL-ODATA01 does not advertise it — "+ + "a user typing that property is told it is unknown", tc.what, f.Name) + } + } + }) + } +} + +// The converse: a name nothing accepts must still be reported, or the test above +// would pass against a visitor that silently swallowed everything. +func TestKnownODataProps_UnknownIsStillFlagged(t *testing.T) { + prog, errs := visitor.Build("create odata client M.C (NotAProperty: 'x');") + if len(errs) > 0 { + t.Fatalf("parse: %v", errs) + } + unknown := collectUnknownProps(prog) + if len(unknown) != 1 || !strings.EqualFold(unknown[0], "NotAProperty") { + t.Errorf("unknown = %v, want [NotAProperty]", unknown) + } +} + +func collectUnknownProps(prog *ast.Program) []string { + var out []string + for _, stmt := range prog.Statements { + switch s := stmt.(type) { + case *ast.CreateODataServiceStmt: + out = append(out, s.UnknownProperties...) + for _, e := range s.Entities { + if e != nil { + out = append(out, e.UnknownProperties...) + } + } + case *ast.CreateODataClientStmt: + out = append(out, s.UnknownProperties...) + case *ast.CreateExternalEntityStmt: + out = append(out, s.UnknownProperties...) + } + } + return out +} From 9377fbbcea40104950550214a3e2a234b49742be Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:18:52 +0000 Subject: [PATCH 14/31] feat(test): say which after-startup microflow a local run displaces MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mxcli test --local` sets the after-startup microflow to its own endpoint registrar and restores it afterwards — deliberate, because a test run wants a known starting state. But it said only: After-startup set to MxTest.RegisterEndpoint (registers the endpoint; runs no tests) so a suite that needs startup state passes under --attach and fails under --local against an empty scratch database, with nothing in the failure pointing at the cause. The tests were asking for state the runner had prevented. The run now names the displaced microflow and says --attach is the way to test against an app that has actually started up. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- cmd/mxcli/testrunner/runner.go | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/cmd/mxcli/testrunner/runner.go b/cmd/mxcli/testrunner/runner.go index ffc8bf459..f5c97c0d1 100644 --- a/cmd/mxcli/testrunner/runner.go +++ b/cmd/mxcli/testrunner/runner.go @@ -236,6 +236,16 @@ func runEndpoint(opts RunOptions, suite *TestSuite, timeout time.Duration, w io. } } fmt.Fprintf(w, " After-startup set to %s (registers the endpoint; runs no tests)\n", endpointStartupFlow) + if state.afterStartup != "" { + // Naming what was displaced is the whole point. A suite that needs + // startup state passes under --attach and fails here against an empty + // database, and nothing in the failure points at the cause: the tests + // asked for state the runner deliberately prevented. + fmt.Fprintf(w, " Note: %s does NOT run during this test run — the suite starts from an empty\n", state.afterStartup) + fmt.Fprintf(w, " %s database. For tests that need what your startup microflow sets up,\n", localTestDBSuffix) + fmt.Fprintf(w, " run them with --attach against an app already up under\n") + fmt.Fprintf(w, " 'mxcli run --local --test-endpoint', which uses that app's database.\n") + } // --watch keeps the runtime and the build server up and re-runs on every // change, so it owns the loop — including printing each run's results, which From 6b6932e154a32fb7a2c073746ce8ee021988c4f8 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:21:43 +0000 Subject: [PATCH 15/31] fix(check): retire the false-positive MDL009, add MDL056 for (empty) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MDL009 errored on `when Open, Pending then` with "Mendix enumeration splits require exactly one value per branch". Mendix does no such thing: verified on mxbuild 11.6.6, a multi-value branch covering every value plus `(empty)` builds with 0 errors. So `mxcli check` was rejecting valid MDL — and contradicting the shipped write-microflows skill, which documents that very form. The rule was found while deciding what #833 could safely promote to a hard exec failure. What actually fails the build is a MISSING branch. An enum split is an exclusive split needing one outgoing flow per condition value, and an uncovered one is CE0079 "The 'X' condition value should be configured in properties for an outgoing flow." MDL056 checks the `(empty)` branch specifically. That half of CE0079 is universal and needs no knowledge of the enumeration's members — confirmed it fires even when the split is on a `not null` enum attribute — so it works from the statement alone. Full value coverage is deliberately left out: it requires resolving the split variable's type to an enum member list, which ValidateMicroflow cannot see, and guessing would trade one false positive for another. A new rule ID rather than a repurposed MDL009, so anything still citing the old number keeps meaning the old, wrong thing. MDL008 (no `else` branch) is correct and stays — mxbuild reports CE0079 for each uncovered value AND CE0773 on the else flow, so an `else` does not stand in for the missing flows. The skill's CASE example used `else` and is corrected here in the same change, since leaving it would teach MDL that fails the build. No positive example in mdl-examples/ trips MDL056; check-mdl and check-skill-mdl both pass. The -ok.mdl repro builds at 0 errors in mxbuild, including the multi-value branch MDL009 used to reject. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- .claude/skills/fix-issue.md | 3 +- .claude/skills/mendix/write-microflows.md | 16 ++++- .../mdl009-enum-split-empty-branch-ok.mdl | 59 +++++++++++++++++ .../mdl009-enum-split-empty-branch.fail.mdl | 43 +++++++++++++ mdl/executor/validate_microflow.go | 57 ++++++++++++++--- .../validate_microflow_enum_split_test.go | 64 +++++++++++++++++-- .../validate_microflow_rules_exec_test.go | 26 ++++---- 7 files changed, 236 insertions(+), 32 deletions(-) create mode 100644 mdl-examples/bug-tests/mdl009-enum-split-empty-branch-ok.mdl create mode 100644 mdl-examples/bug-tests/mdl009-enum-split-empty-branch.fail.mdl diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 15f6d0aa1..bd06b564e 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -393,4 +393,5 @@ extracting `OffsetExpression`/`LimitExpression`. | A sidebar navigation label renders truncated — "All tasks" shows as "All task" — under every theme (`signal`, `ledger`, `console`) and both variants, so switching theme does not help. Measured on the live DOM as `scrollWidth=56` inside `clientWidth=48` | Atlas's **closed** sidebar is an icon rail: `--navsidebar-width-closed: 48px` in Atlas's own `themesource/atlas_core/web/themes/_theme-default.scss`. The label is wider than the rail, and the closed rail expects an *icon*, not text. No mxcli theme sets any navigation width — the themes map colours — which is exactly why every theme reproduces it | `.claude/skills/mendix/theme-styling.md` (documented; no code change) | **A fix was written, verified, and reverted** — record the reason: adding `text-overflow: ellipsis` to the nav item helps only where Atlas also sets `white-space: nowrap`; elsewhere the label wraps to two readable lines, and the rule turns `All / tasks` into `All / t…`. Screenshotted both ways against the real compiled CSS. The answer belongs to the app (give nav items icons — what the rail is for — or keep the sidebar open), not to a theme that would impose it on every app. **Generalisable**: when a reported symptom traces to an upstream layout constant, reproduce the geometry against the real compiled CSS (48px container, real class names, Playwright measurement) — it takes minutes, tells you whose constant it is, and shows when the "obvious" CSS fix is a regression. mxcli-todo #19d | | `create non-persistent entity X ( A: String not null error '…' )` (or `unique`) passes `mxcli check` AND `mxcli exec`, then the build fails **CE0070** "Validations rules are not allowed on entity 'X', because it is not persistable" | `not null` / `unique` ARE validation rules — Studio Pro models "required" and "uniqueness" as rules on the entity, not as column constraints — so Mendix rejects them on a non-persistable entity. Nothing in mxcli connected the attribute constraint to the entity's persistence kind | `mdl/executor/cmd_enumerations.go` (`validateNPEValidationRules`, called from `ValidateEntity`) | Add **MDL054**, error severity, fired from the CREATE path only. **Establish the construct matrix against mxbuild before writing the rule, not from the issue text** — verified on 11.6.6 that `not null` with a message, `not null` bare, AND `unique` each produce CE0070 while a plain attribute does not, so the bare form (easy to miss, since the reporter only showed the message form) is flagged too. The CREATE path is the only one that can run this: `ALTER ENTITY … ADD ATTRIBUTE` does not carry the persistence kind, the same limitation MDL020 has. Sweep `mdl-examples/` + `scripts/check-skill-mdl.sh` after adding any error-severity rule — a false positive there breaks every user with that shape. Negative test `mdl-examples/bug-tests/832-npe-validation-rules.fail.mdl` (`.fail.mdl` = must fail check, enforced by `make check-mdl`) plus `-ok.mdl` pinning the other edge; tests `TestValidateEntityNPEValidationRules`. Issue #832 | | `retrieve $L from Mod.Entity where [Attr = $Var/Mod.Assoc/Attr]` passes `mxcli check` AND `mxcli exec`, then the build fails **CE0161** "Error(s) in XPath constraint" | Mendix XPath reaches at most ONE hop off a variable, and nothing checked the hop count | `mdl/executor/validate_microflow.go` (`checkXPathVariableTraversal`, `xpathVarTraversalRe`), called from the `*ast.RetrieveStmt` arm beside MDL047/MDL048 | Add **MDL055**, error severity, matching a `$var`-rooted path with **2+ segments**. **Establish the boundary against mxbuild first — it is narrower than it looks**: `$Var/Attr` VALID, `$Var/Mod.Assoc` VALID (one hop to the associated object), `$Var/Mod.Assoc/Attr` CE0161. A rule keying on "a module-qualified segment follows a variable" would reject the middle form, which is legal; key on hop count instead. Verified on 11.6.6 by building all three and dropping the offender to confirm the other two are clean. **Reject, don't try to serialize** — there is no valid XPath for the two-hop form, so the constraint must be restructured and only the author knows which way. **Verify the suggestion you emit**: both recommended rewrites (`retrieve $Related from $Var/Mod.Assoc;` then constrain on `$Related/Attr`; or invert to `[Mod.Assoc/Mod.Entity = $Var]`) were built and confirmed at 0 errors before the message claimed "both forms build clean". Negative test `mdl-examples/bug-tests/831-xpath-variable-traversal.fail.mdl` + `-ok.mdl`; test `TestXPathVariableTraversal`. Issue #831 | -| `mxcli check` reports a microflow rule (e.g. MDL048 `[id = $StringVar]`) but `mxcli exec` writes the same script silently — a script that skips `check` produces a project the build rejects | Two different validators. The exec path called `ValidateMicroflowBody` (semantic errors); the MDL0xx rule set lives in `ValidateMicroflow`, which was wired only into `cmd/mxcli/cmd_check.go` and the LSP. Same shape as #836, where a guard existed on every exec path but was never reached from validate | `mdl/executor/validate.go` (`validateMicroflowRules`, `execEnforcedMicroflowRules`) called from `mdl/executor/cmd_microflows_create.go` — the handler, mirroring `ValidateEntity` in `cmd_entities.go` (NOT `validateWithContext`, which would double-report under `check --references`) | Promote an **explicit allowlist** of rules verified against a real mxbuild, never the whole set. **Blanket promotion was tried and reverted**: it turns every error-severity rule into a write barrier, and `MDL009` ("enumeration splits require exactly one value per branch") is a **false positive** — a multi-value branch covering every enum value builds at 0 errors on 11.6.6, and the shipped `write-microflows` skill documents that very form. It also broke an existing repo test whose fixture uses `else` on an enum split. `MDL008` by contrast IS correct (mxbuild: CE0079 per uncovered value + CE0773) — so before promoting any rule, build its construct and read the verdict. Warnings are never promoted. Tests `TestValidateMicroflowRules_ReachedFromExec` and `…_UnverifiedRulesNotPromoted` (the latter fails if the allowlist is widened carelessly). Issue #833 | +| `mxcli check` reports a microflow rule (e.g. MDL048 `[id = $StringVar]`) but `mxcli exec` writes the same script silently — a script that skips `check` produces a project the build rejects | Two different validators. The exec path called `ValidateMicroflowBody` (semantic errors); the MDL0xx rule set lives in `ValidateMicroflow`, which was wired only into `cmd/mxcli/cmd_check.go` and the LSP. Same shape as #836, where a guard existed on every exec path but was never reached from validate | `mdl/executor/validate.go` (`validateMicroflowRules`, `execEnforcedMicroflowRules`) called from `mdl/executor/cmd_microflows_create.go` — the handler, mirroring `ValidateEntity` in `cmd_entities.go` (NOT `validateWithContext`, which would double-report under `check --references`) | Promote an **explicit allowlist** of rules verified against a real mxbuild, never the whole set. **Blanket promotion was tried and reverted**: it turns every error-severity rule into a write barrier, and `MDL009` ("enumeration splits require exactly one value per branch") was a **false positive** — a multi-value branch covering every enum value builds at 0 errors on 11.6.6, and the shipped `write-microflows` skill documents that very form (since retired, see the MDL056 row). It also broke an existing repo test whose fixture uses `else` on an enum split. `MDL008` by contrast IS correct (mxbuild: CE0079 per uncovered value + CE0773) — so before promoting any rule, build its construct and read the verdict. Warnings are never promoted. Tests `TestValidateMicroflowRules_ReachedFromExec` and `…_UnverifiedRulesNotPromoted` (the latter fails if the allowlist is widened carelessly). Issue #833 | +| `mxcli check` errors **MDL009** "enumeration splits require exactly one value per branch" on `when Open, Pending then` — but Mendix accepts it, so check rejects valid MDL and contradicts the shipped `write-microflows` skill | The rule asserted the opposite of the platform's behaviour. Nobody had built the construct: a multi-value branch covering every value **plus `(empty)`** builds at 0 errors on 11.6.6 | `mdl/executor/validate_microflow.go` (the `*ast.EnumSplitStmt` arm; `checkEnumSplitEmptyBranch`) | Retire the assertion and replace it with what actually fails: an enum split needs an outgoing flow per condition value, so a missing branch is **CE0079**. **MDL056** checks the `(empty)` branch — universal, verified to hold even on a `not null` enum attribute, so it needs no enumeration lookup and works from the statement alone. Full value coverage (the other half of CE0079) is deliberately NOT implemented: it needs the enum's member list, i.e. resolving the split variable's type against script or project, which `ValidateMicroflow` cannot see — guessing would trade one false positive for another. **Use a NEW rule ID rather than repurposing**, so anything citing the old number still means the old, wrong thing. `MDL008` (no `else`) is correct and stays — mxbuild gives CE0079 per uncovered value **plus** CE0773 on the else flow. Fix the skill in the same change: it documented the invalid `else` form. Tests `TestValidateMicroflow_EnumSplitMultipleValuesAllowed` / `…RequiresEmptyBranch`; repro `mdl-examples/bug-tests/mdl009-enum-split-empty-branch.fail.mdl` + `-ok.mdl` | diff --git a/.claude/skills/mendix/write-microflows.md b/.claude/skills/mendix/write-microflows.md index efdc57f08..f316c25fa 100644 --- a/.claude/skills/mendix/write-microflows.md +++ b/.claude/skills/mendix/write-microflows.md @@ -421,15 +421,27 @@ Use `case` when a microflow branches on an enumeration value. case $Status when Open, Pending then return true; - when (empty) then + when Closed then return false; - else + when (empty) then return false; end case; ``` `(empty)` represents an unset enumeration value. Multiple values can share one `when` branch by separating them with commas. Case values are bare identifiers — do **not** quote them. +> **Every value needs a branch, including `(empty)` — and there is no `else`.** +> A Mendix enum split is an exclusive split with one outgoing flow per condition +> value, so an uncovered value fails the build with **CE0079** *"The 'X' condition +> value should be configured in properties for an outgoing flow."* `mxcli check` +> reports a missing `(empty)` branch as **MDL056**, and an `else` branch as +> **MDL008** (an `else` does not stand in for the missing flows: mxbuild reports +> CE0079 for each uncovered value *and* CE0773 on the else flow itself). +> +> The `(empty)` branch is required **even when the attribute is `not null`** — +> verified on Mendix 11.6.6. If several values share a path, put them in one +> branch (`when Open, Pending then`) rather than reaching for `else`. + ### Type Split And Cast Statements Use `split type` when a microflow branches on an object's runtime specialization. diff --git a/mdl-examples/bug-tests/mdl009-enum-split-empty-branch-ok.mdl b/mdl-examples/bug-tests/mdl009-enum-split-empty-branch-ok.mdl new file mode 100644 index 000000000..0a759a309 --- /dev/null +++ b/mdl-examples/bug-tests/mdl009-enum-split-empty-branch-ok.mdl @@ -0,0 +1,59 @@ +-- ============================================================================ +-- MDL009 retired / MDL056 — the enum-split forms that must NOT be rejected +-- ============================================================================ +-- +-- The negative half is mdl009-enum-split-empty-branch.fail.mdl. +-- +-- Verified against mxbuild 11.6.6: this file builds with 0 errors, including +-- the multi-value branch that the retired MDL009 used to reject. +-- ============================================================================ + +CREATE MODULE BugM9Ok; + +CREATE ENUMERATION BugM9Ok.Status ( + Open caption 'Open', + Pending caption 'Pending', + Closed caption 'Closed' +); + +-- A multi-value branch is valid — this is what MDL009 wrongly rejected. +CREATE OR MODIFY MICROFLOW BugM9Ok.MultiValue ($Status: Enumeration(BugM9Ok.Status)) +RETURNS Boolean +BEGIN + case $Status + when Open, Pending then + return true; + when Closed then + return false; + when (empty) then + return false; + end case; +END; + +-- One value per branch is equally valid. +CREATE OR MODIFY MICROFLOW BugM9Ok.OnePerBranch ($Status: Enumeration(BugM9Ok.Status)) +RETURNS Boolean +BEGIN + case $Status + when Open then + return true; + when Pending then + return true; + when Closed then + return false; + when (empty) then + return false; + end case; +END; + +-- `(empty)` may share a branch with real values. +CREATE OR MODIFY MICROFLOW BugM9Ok.EmptySharesBranch ($Status: Enumeration(BugM9Ok.Status)) +RETURNS Boolean +BEGIN + case $Status + when Open, Pending then + return true; + when Closed, (empty) then + return false; + end case; +END; diff --git a/mdl-examples/bug-tests/mdl009-enum-split-empty-branch.fail.mdl b/mdl-examples/bug-tests/mdl009-enum-split-empty-branch.fail.mdl new file mode 100644 index 000000000..2cc6140ed --- /dev/null +++ b/mdl-examples/bug-tests/mdl009-enum-split-empty-branch.fail.mdl @@ -0,0 +1,43 @@ +-- ============================================================================ +-- MDL009 retired, MDL056 added — enum split branch rules +-- ============================================================================ +-- +-- NEGATIVE TEST (.fail.mdl): `mxcli check` MUST reject this file. +-- +-- MDL009 used to error on `when Open, Pending then`, claiming Mendix required +-- exactly one value per branch. That was wrong — verified on mxbuild 11.6.6, a +-- multi-value branch covering every value builds with 0 errors, and the shipped +-- write-microflows skill documents that form. The rule rejected valid MDL, so +-- it is retired. +-- +-- What actually fails the build is a MISSING branch. An enum split is an +-- exclusive split needing one outgoing flow per condition value: +-- +-- [error] [CE0079] "The '(empty)' condition value should be configured in +-- properties for an outgoing flow." +-- +-- MDL056 catches the `(empty)` case, which is universal and needs no knowledge +-- of the enumeration's members — it holds even on a `not null` attribute. +-- +-- The valid forms are in mdl009-enum-split-empty-branch-ok.mdl, which must PASS. +-- ============================================================================ + +CREATE MODULE BugM9; + +CREATE ENUMERATION BugM9.Status ( + Open caption 'Open', + Pending caption 'Pending', + Closed caption 'Closed' +); + +-- REJECTED (MDL056): every value is covered, but `(empty)` is not. +CREATE OR MODIFY MICROFLOW BugM9.NoEmptyBranch ($Status: Enumeration(BugM9.Status)) +RETURNS Boolean +BEGIN + case $Status + when Open, Pending then + return true; + when Closed then + return false; + end case; +END; diff --git a/mdl/executor/validate_microflow.go b/mdl/executor/validate_microflow.go index 86cb2e8d2..a198ce7b8 100644 --- a/mdl/executor/validate_microflow.go +++ b/mdl/executor/validate_microflow.go @@ -162,9 +162,9 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { v.walkBody(stmt.ThenBody) v.walkBody(stmt.ElseBody) case *ast.EnumSplitStmt: - // Mendix enumeration splits map to exclusive splits with one outgoing - // flow per enum value. Multiple values per branch and a default (else) - // flow are not supported — Studio Pro will reject both with CE errors. + // A Mendix enumeration split is an exclusive split that needs an + // outgoing flow for every enum value AND for (empty); a default flow + // is not offered. Verified on mxbuild 11.6.6. if len(stmt.ElseBody) > 0 { v.addViolation("MDL008", linter.SeverityError, fmt.Sprintf("case statement on '$%s' has an else branch; "+ @@ -173,14 +173,14 @@ func (v *microflowValidator) walkBody(body []ast.MicroflowStatement) { stmt.Variable), "Add an explicit when branch for every enum value instead of using else") } + // MDL009 used to error here on a branch listing more than one value, + // claiming Mendix required exactly one per branch. That was wrong — + // `when Open, Pending then` covering every value builds with 0 errors, + // and the write-microflows skill documents that form — so the rule + // rejected valid MDL. It is retired rather than repurposed; MDL056 + // below checks what actually fails the build. + v.checkEnumSplitEmptyBranch(stmt) for _, c := range stmt.Cases { - if len(c.Values) > 1 { - v.addViolation("MDL009", linter.SeverityError, - fmt.Sprintf("case statement on '$%s': when branch lists %d values (%s); "+ - "Mendix enumeration splits require exactly one value per branch.", - stmt.Variable, len(c.Values), strings.Join(c.Values, ", ")), - "Split into separate when branches, one per enum value") - } v.walkBody(c.Body) } v.walkBody(stmt.ElseBody) @@ -1244,3 +1244,40 @@ func isEmptyMessage(expr ast.Expression) bool { } return false } + +// checkEnumSplitEmptyBranch (MDL056) flags an enumeration split with no +// `(empty)` branch. A Mendix enum split needs an outgoing flow for every value +// AND for the unset case; without one the build fails with +// +// CE0079 "The '(empty)' condition value should be configured in properties +// for an outgoing flow." +// +// Verified on mxbuild 11.6.6, and the requirement is universal — it holds even +// when the split is on a `not null` enum attribute, so no nullability analysis +// is needed and the check works from the statement alone. +// +// This replaces the retired MDL009, which asserted the opposite of what Mendix +// does (see the EnumSplitStmt arm). A new ID was used rather than repurposing +// MDL009 so that anything referring to the old number still refers to the old, +// wrong meaning. +// +// Value coverage — every enum member having a branch, the other half of CE0079 — +// is deliberately NOT checked here: it needs the enumeration's member list, +// which means resolving the split variable's type against the script or the +// project. ValidateMicroflow sees only one statement. Worth adding where that +// context exists; guessing it here would trade one false positive for another. +func (v *microflowValidator) checkEnumSplitEmptyBranch(stmt *ast.EnumSplitStmt) { + for _, c := range stmt.Cases { + for _, val := range c.Values { + if strings.EqualFold(strings.TrimSpace(val), "(empty)") { + return + } + } + } + v.addViolation("MDL056", linter.SeverityError, + fmt.Sprintf("case statement on '$%s' has no `(empty)` branch; a Mendix enumeration split needs an "+ + "outgoing flow for the unset value too, so this builds as CE0079 \"The '(empty)' condition value "+ + "should be configured in properties for an outgoing flow\"", stmt.Variable), + "Add a `when (empty) then …` branch. It is required even when the attribute is `not null`. "+ + "A branch may list several values (`when Open, (empty) then …`) if they share a path.") +} diff --git a/mdl/executor/validate_microflow_enum_split_test.go b/mdl/executor/validate_microflow_enum_split_test.go index 7533cda89..7a368dbc4 100644 --- a/mdl/executor/validate_microflow_enum_split_test.go +++ b/mdl/executor/validate_microflow_enum_split_test.go @@ -65,7 +65,13 @@ func TestValidateMicroflow_EnumSplitElseForbidden(t *testing.T) { t.Fatalf("expected MDL008 for enum split with else branch, got %#v", violations) } -func TestValidateMicroflow_EnumSplitMultipleValuesForbidden(t *testing.T) { +// TestValidateMicroflow_EnumSplitMultipleValuesAllowed inverts what MDL009 used +// to assert. The old rule claimed "Mendix enumeration splits require exactly one +// value per branch" and errored on `when Open, Pending then`. That is wrong: +// verified on mxbuild 11.6.6, a multi-value branch covering every enum value +// (plus `(empty)`) builds with 0 errors, and the shipped write-microflows skill +// documents exactly that form. The rule rejected valid MDL. +func TestValidateMicroflow_EnumSplitMultipleValuesAllowed(t *testing.T) { stmt := &ast.CreateMicroflowStmt{ Name: ast.QualifiedName{Module: "Sample", Name: "Route"}, Body: []ast.MicroflowStatement{ @@ -75,18 +81,66 @@ func TestValidateMicroflow_EnumSplitMultipleValuesForbidden(t *testing.T) { {Values: []string{"Open", "Pending"}, Body: []ast.MicroflowStatement{ &ast.ReturnStmt{Value: &ast.LiteralExpr{Kind: ast.LiteralBoolean, Value: true}}, }}, + {Values: []string{"(empty)"}, Body: []ast.MicroflowStatement{ + &ast.ReturnStmt{Value: &ast.LiteralExpr{Kind: ast.LiteralBoolean, Value: false}}, + }}, }, }, }, } - violations := ValidateMicroflow(stmt) - for _, v := range violations { + for _, v := range ValidateMicroflow(stmt) { if v.RuleID == "MDL009" { - return + t.Fatalf("MDL009 rejected a multi-value branch, which Mendix accepts: %s", v.Message) + } + } +} + +// TestValidateMicroflow_EnumSplitRequiresEmptyBranch pins what MDL009 SHOULD +// have been checking. An enumeration split needs an outgoing flow for `(empty)` +// as well as for each value; without one the build fails +// +// CE0079 "The '(empty)' condition value should be configured in properties +// for an outgoing flow." +// +// Verified on mxbuild 11.6.6, and it is universal: it holds even when the split +// is on a `not null` enum attribute, so no nullability analysis is needed. +func TestValidateMicroflow_EnumSplitRequiresEmptyBranch(t *testing.T) { + mk := func(values ...[]string) *ast.CreateMicroflowStmt { + var cases []ast.EnumSplitCase + for _, vals := range values { + cases = append(cases, ast.EnumSplitCase{Values: vals, Body: []ast.MicroflowStatement{ + &ast.ReturnStmt{Value: &ast.LiteralExpr{Kind: ast.LiteralBoolean, Value: true}}, + }}) } + return &ast.CreateMicroflowStmt{ + Name: ast.QualifiedName{Module: "Sample", Name: "Route"}, + Body: []ast.MicroflowStatement{&ast.EnumSplitStmt{Variable: "Status", Cases: cases}}, + } + } + fires := func(stmt *ast.CreateMicroflowStmt) bool { + for _, v := range ValidateMicroflow(stmt) { + if v.RuleID == "MDL056" { + return true + } + } + return false + } + + if !fires(mk([]string{"Open"}, []string{"Closed"})) { + t.Error("expected MDL056 when no (empty) branch is present (CE0079)") + } + if fires(mk([]string{"Open"}, []string{"Closed"}, []string{"(empty)"})) { + t.Error("MDL056 must not fire when an (empty) branch is present") + } + // The (empty) marker may share a branch with real values. + if fires(mk([]string{"Open", "(empty)"}, []string{"Closed"})) { + t.Error("MDL056 must not fire when (empty) shares a multi-value branch") + } + // Case is not significant in the marker. + if fires(mk([]string{"Open"}, []string{"(EMPTY)"})) { + t.Error("MDL056 must accept the (empty) marker regardless of case") } - t.Fatalf("expected MDL009 for enum split with multiple values per branch, got %#v", violations) } func TestValidateMicroflow_EnumSplitBranchScopedVariable(t *testing.T) { diff --git a/mdl/executor/validate_microflow_rules_exec_test.go b/mdl/executor/validate_microflow_rules_exec_test.go index a31fa7ef4..8ee753cd6 100644 --- a/mdl/executor/validate_microflow_rules_exec_test.go +++ b/mdl/executor/validate_microflow_rules_exec_test.go @@ -108,20 +108,21 @@ end;` // TestValidateMicroflowRules_UnverifiedRulesNotPromoted pins the deliberate // narrowness of execEnforcedMicroflowRules. // -// MDL009 ("enumeration splits require exactly one value per branch") fires in -// `check` but is a FALSE POSITIVE: verified on mxbuild 11.6.6, a multi-value -// branch that covers every enum value builds at 0 errors, and the shipped -// write-microflows skill documents exactly that form. If it were promoted, -// `exec` would refuse valid MDL — so this test fails the moment someone widens -// the allowlist without checking the rule against a real build. +// MDL008 is a CORRECT rule (mxbuild rejects `else` on an enum split with CE0079 +// per uncovered value plus CE0773) that is nonetheless not on the allowlist: +// membership requires a verified construct, and correctness alone is not the +// bar — every promoted rule becomes a hard write barrier. This test fails the +// moment someone widens the allowlist wholesale. func TestValidateMicroflowRules_UnverifiedRulesNotPromoted(t *testing.T) { src := `create microflow M.ACT ($S: Enumeration(M.Status)) returns String begin case $S - when Open, Pending then + when Open then return 'a'; - when Closed then + when (empty) then return 'b'; + else + return 'c'; end case; end;` prog, errs := visitor.Build(src) @@ -130,19 +131,16 @@ end;` } stmt := prog.Statements[0].(*ast.CreateMicroflowStmt) - // check still reports it... sawInCheck := false for _, v := range ValidateMicroflow(stmt) { - if v.RuleID == "MDL009" { + if v.RuleID == "MDL008" { sawInCheck = true } } if !sawInCheck { - t.Skip("MDL009 no longer fires; if the rule was fixed or removed, drop this test") + t.Fatal("expected check to report MDL008 for an else branch on an enum split") } - // ...but exec must not refuse to write it. if err := validateMicroflowRules(stmt); err != nil { - t.Errorf("MDL009 is a false positive (a multi-value branch covering every enum value "+ - "builds at 0 errors) and must not block exec, got: %v", err) + t.Errorf("MDL008 is not on the verified allowlist and must not block exec, got: %v", err) } } From 7da49c092ded6f64d4a274d94dd58e88e5aeac77 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:25:39 +0000 Subject: [PATCH 16/31] fix(odata): apply published-entity changes on modify MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-running a `create or modify odata service` after editing a `publish entity` block did not apply the change. Marking a member Filterable and re-executing left the served $metadata exactly as it was; only `drop odata service` + create picked it up. The modify branch updated the service's scalar properties and never touched EntityTypes or EntitySets. Supplied entities now replace the stored ones wholesale. Replacing rather than merging is what makes the script the description of the service: a member removed from the script is removed from the service, which merging could not express. The same change carries AllowedModuleRoles across a modify. That is a guard, not a reproduction — the reported grant loss (mxcli-formula1 #26) did not reproduce on 11.12.1, on either the fixed or the previous build — but a modify cannot express grants, so it must not be able to drop them. Verified on 11.12.1: the same script yields `Label as 'label'` before and `Label as 'label' (Filterable, Sortable)` after, and the build stays at 0 errors. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/executor/cmd_odata.go | 34 +++++ mdl/executor/cmd_odata_modify_members_test.go | 143 ++++++++++++++++++ 2 files changed, 177 insertions(+) create mode 100644 mdl/executor/cmd_odata_modify_members_test.go diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index b72bc5ef9..2e98d8cf9 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -1339,6 +1339,8 @@ func createODataService(ctx *ExecContext, stmt *ast.CreateODataServiceStmt) erro modName := h.GetModuleName(modID) if strings.EqualFold(modName, stmt.Name.Module) && strings.EqualFold(svc.Name, stmt.Name.Name) { if stmt.CreateOrModify { + // Snapshot the grants before anything below can clear them. + existingRoles := append([]string(nil), svc.AllowedModuleRoles...) svc.Documentation = stmt.Documentation if stmt.Path != "" { svc.Path = stmt.Path @@ -1371,6 +1373,38 @@ func createODataService(ctx *ExecContext, stmt *ast.CreateODataServiceStmt) erro if len(stmt.AuthenticationTypes) > 0 { svc.AuthenticationTypes = stmt.AuthenticationTypes } + // Published entities are replaced wholesale when the statement + // supplies any. Previously the modify branch ignored them + // entirely: editing a `publish entity` block and re-running + // left the served $metadata unchanged, so `Filterable` or + // `Countable` changes appeared to do nothing and the only + // thing that worked was drop + create. Replacing rather than + // merging is what makes the script the description of the + // service — a member removed from the script is removed from + // the service, which merging could never express. + if len(stmt.Entities) > 0 { + svc.EntityTypes = nil + svc.EntitySets = nil + for _, entityDef := range stmt.Entities { + entityType, entitySet := astEntityDefToModel(ctx, entityDef) + svc.EntityTypes = append(svc.EntityTypes, entityType) + svc.EntitySets = append(svc.EntitySets, entitySet) + } + } + // AllowedModuleRoles is granted by a separate statement + // (`grant access on odata service …`) and cannot be expressed + // here, so a modify must carry it through or the build fails + // with "At least one allowed role must be selected for the + // published OData service to be accessible." + // + // A guard, not a fix for an observed defect: the loss was + // reported (mxcli-formula1 #26) but did not reproduce on + // 11.12.1 — grants survived a modify on both the current and + // the previous build. Kept because the invariant is real and + // the cost is a slice copy; if it never fires, nothing is lost. + if len(svc.AllowedModuleRoles) == 0 && len(existingRoles) > 0 { + svc.AllowedModuleRoles = existingRoles + } if err := ctx.Backend.UpdatePublishedODataService(svc); err != nil { return mdlerrors.NewBackend("update OData service", err) } diff --git a/mdl/executor/cmd_odata_modify_members_test.go b/mdl/executor/cmd_odata_modify_members_test.go new file mode 100644 index 000000000..0e7cf7d55 --- /dev/null +++ b/mdl/executor/cmd_odata_modify_members_test.go @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// mxcli-formula1 findings #26: re-running a `create or modify odata service` +// after editing a `publish entity` block did not apply the change. Marking a +// member Filterable and re-executing left the served $metadata exactly as it +// was; only `drop odata service` + create picked it up. The modify branch +// updated the service's scalar properties and never touched EntityTypes or +// EntitySets. +// +// Confirmed on 11.12.1 against a real build: the same script produced +// `Label as 'label'` before the fix and `Label as 'label' (Filterable, Sortable)` +// after. +func TestModifyODataService_AppliesPublishedEntities(t *testing.T) { + svc, mb, h := existingPublishedService() + var updated *model.PublishedODataService + mb.UpdatePublishedODataServiceFunc = func(s *model.PublishedODataService) error { + updated = s + return nil + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + stmt := &ast.CreateODataServiceStmt{ + Name: ast.QualifiedName{Module: "MyModule", Name: "CatalogService"}, + CreateOrModify: true, + Entities: []*ast.PublishedEntityDef{{ + Entity: ast.QualifiedName{Module: "MyModule", Name: "Order"}, + ExposedName: "Orders", + Members: []*ast.PublishedMemberDef{ + {Name: "Label", ExposedName: "label", Filterable: true, Sortable: true}, + }, + }}, + } + assertNoError(t, createODataService(ctx, stmt)) + + if updated == nil { + t.Fatal("the service was never updated") + } + m := findPublishedMember(t, updated, "Label") + if !m.Filterable || !m.Sortable { + t.Errorf("member Label: filterable=%v sortable=%v, want both true — "+ + "the modify did not apply the edited publish block", m.Filterable, m.Sortable) + } + _ = svc +} + +// A modify cannot express role grants (`grant access on odata service …` is a +// separate statement), so it must carry the existing ones through or the build +// fails with "At least one allowed role must be selected". +// +// A guard rather than a reproduction: the loss was reported but did not +// reproduce on 11.12.1 — grants survived a modify on both the fixed and the +// previous build. The invariant holds regardless. +func TestModifyODataService_KeepsRoleGrants(t *testing.T) { + svc, mb, h := existingPublishedService() + svc.AllowedModuleRoles = []string{"MyModule.ApiUser"} + var updated *model.PublishedODataService + mb.UpdatePublishedODataServiceFunc = func(s *model.PublishedODataService) error { + updated = s + return nil + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + stmt := &ast.CreateODataServiceStmt{ + Name: ast.QualifiedName{Module: "MyModule", Name: "CatalogService"}, + CreateOrModify: true, + Path: "odata/catalog2/", + } + assertNoError(t, createODataService(ctx, stmt)) + + if updated == nil { + t.Fatal("the service was never updated") + } + if len(updated.AllowedModuleRoles) != 1 || updated.AllowedModuleRoles[0] != "MyModule.ApiUser" { + t.Errorf("AllowedModuleRoles = %v, want [MyModule.ApiUser]", updated.AllowedModuleRoles) + } + if updated.Path != "odata/catalog2/" { + t.Errorf("Path = %q, want the modify's own change to land too", updated.Path) + } +} + +// existingPublishedService is a one-entity service already in the model, with +// the backend and hierarchy wired so createODataService takes its modify branch. +func existingPublishedService() (*model.PublishedODataService, *mock.MockBackend, *ContainerHierarchy) { + mod := mkModule("MyModule") + svc := &model.PublishedODataService{ + ContainerID: mod.ID, + Name: "CatalogService", + Path: "odata/catalog/", + ServiceName: "CatalogService", + EntityTypes: []*model.PublishedEntityType{{ + ExposedName: "Order", + Entity: "MyModule.Order", + Members: []*model.PublishedMember{{Kind: "attribute", Name: "Label", ExposedName: "label"}}, + }}, + EntitySets: []*model.PublishedEntitySet{{ + ExposedName: "Orders", + EntityTypeName: "MyModule.Order", + }}, + } + h := mkHierarchy(mod) + withContainer(h, svc.ContainerID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + GetModuleByNameFunc: func(string) (*model.Module, error) { return mod, nil }, + GetDomainModelFunc: func(model.ID) (*domainmodel.DomainModel, error) { + return &domainmodel.DomainModel{ + Entities: []*domainmodel.Entity{{ + Name: "Order", + Attributes: []*domainmodel.Attribute{{Name: "Label", Type: &domainmodel.StringAttributeType{Length: 120}}}, + }}, + }, nil + }, + ListPublishedODataServicesFunc: func() ([]*model.PublishedODataService, error) { + return []*model.PublishedODataService{svc}, nil + }, + } + return svc, mb, h +} + +func findPublishedMember(t *testing.T, svc *model.PublishedODataService, name string) *model.PublishedMember { + t.Helper() + for _, et := range svc.EntityTypes { + for _, m := range et.Members { + if m.Name == name { + return m + } + } + } + t.Fatalf("published member %q not found", name) + return nil +} From 98ddb29bf575cb5a4ee98927d625300ab6cf4cf5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:40:23 +0000 Subject: [PATCH 17/31] feat(security): CREATE OR MODIFY MODULE ROLE MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `create module role` had no `or modify` form, so re-running a security script failed on the first role that already existed and role creation had to live in its own run-once file. `create or modify module role` now updates an existing role's description instead of failing. AddModuleRole already overwrites, so it also adopts the caller's casing — the same path the auto-provisioned-role branch above it uses. `createModuleRoleStatement` carries its own CREATE keyword (it is dispatched from securityStatement, not from the shared createStatement rule), so the optional OR MODIFY goes in that rule and has to stay distinguishable from `create or modify module`. Both spellings are covered by a test, and the full doctype integration gate is green — the lesson from the last grammar change is that a `check`-only sweep proves nothing about what the visitor builds. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .claude/skills/mendix/manage-security.md | 4 ++ cmd/mxcli/syntax/features_security.go | 4 +- docs/01-project/MDL_QUICK_REFERENCE.md | 2 +- mdl/ast/ast_security.go | 4 ++ mdl/executor/cmd_security_write.go | 12 ++++ mdl/grammar/domains/MDLSecurity.g4 | 5 +- mdl/visitor/visitor_security.go | 3 +- .../visitor_security_or_modify_test.go | 71 +++++++++++++++++++ 8 files changed, 100 insertions(+), 5 deletions(-) create mode 100644 mdl/visitor/visitor_security_or_modify_test.go diff --git a/.claude/skills/mendix/manage-security.md b/.claude/skills/mendix/manage-security.md index 7236ffbc3..1fc8c9eae 100644 --- a/.claude/skills/mendix/manage-security.md +++ b/.claude/skills/mendix/manage-security.md @@ -95,6 +95,10 @@ create module role MyModule.Admin description 'Full administrative access'; create module role MyModule.User; create module role MyModule.Viewer description 'Read-only access'; +-- `or modify` updates an existing role's description instead of failing, so the +-- whole security script stays re-runnable rather than needing a run-once file. +create or modify module role MyModule.ApiUser description 'API consumer'; + -- Remove a module role drop module role MyModule.Viewer; ``` diff --git a/cmd/mxcli/syntax/features_security.go b/cmd/mxcli/syntax/features_security.go index 9f6eba6f5..7481baf8b 100644 --- a/cmd/mxcli/syntax/features_security.go +++ b/cmd/mxcli/syntax/features_security.go @@ -21,8 +21,8 @@ func init() { Keywords: []string{ "module role", "create role", "drop role", }, - Syntax: "CREATE MODULE ROLE . [DESCRIPTION ''];\nDROP MODULE ROLE .;", - Example: "CREATE MODULE ROLE Shop.Admin DESCRIPTION 'Full access';\nCREATE MODULE ROLE Shop.User DESCRIPTION 'Read-only access';", + Syntax: "CREATE [OR MODIFY] MODULE ROLE . [DESCRIPTION ''];\nDROP MODULE ROLE .;", + Example: "CREATE MODULE ROLE Shop.Admin DESCRIPTION 'Full access';\n-- OR MODIFY makes a security script re-runnable:\nCREATE OR MODIFY MODULE ROLE Shop.User DESCRIPTION 'Read-only access';", SeeAlso: []string{"security.user-role", "security.entity-access"}, }) diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index ac4e35c32..a4ba27383 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -355,7 +355,7 @@ Nested folders use `/` separator: `'Parent/Child/Grandchild'`. Missing folders a | Show demo users | `show demo users;` | Configured demo users | | Show access on element | `show access on microflow\|nanoflow\|page\|entity Mod.Name;` | Which roles can access | | Show security matrix | `show security matrix [in module];` | Full access overview | -| Create module role | `create module role Mod.Role [description 'text'];` | | +| Create module role | `create [or modify] module role Mod.Role [description 'text'];` | `or modify` updates an existing role instead of failing, so a security script can be re-run | | Drop module role | `drop module role Mod.Role;` | | | Create user role | `create user role Name (Mod.Role, ...) [manage all roles];` | Aggregates module roles | | Alter user role | `alter user role Name add\|remove module roles (Mod.Role, ...);` | | diff --git a/mdl/ast/ast_security.go b/mdl/ast/ast_security.go index 09e06cdf2..9c03d99e7 100644 --- a/mdl/ast/ast_security.go +++ b/mdl/ast/ast_security.go @@ -10,6 +10,10 @@ package ast type CreateModuleRoleStmt struct { Name QualifiedName Description string + // CreateOrModify makes the statement idempotent: an existing role has its + // description updated instead of the statement failing, so a security script + // can be re-run. + CreateOrModify bool } func (s *CreateModuleRoleStmt) isStatement() {} diff --git a/mdl/executor/cmd_security_write.go b/mdl/executor/cmd_security_write.go index 3eb8fdbed..52aa3bb4d 100644 --- a/mdl/executor/cmd_security_write.go +++ b/mdl/executor/cmd_security_write.go @@ -60,6 +60,18 @@ func execCreateModuleRole(ctx *ExecContext, s *ast.CreateModuleRoleStmt) error { } return nil } + if s.CreateOrModify { + // Re-running a security script must not fail on a role that is + // already there. AddModuleRole overwrites, so this also adopts a new + // description and the caller's casing. + if err := ctx.Backend.AddModuleRole(ms.ID, s.Name.Name, s.Description); err != nil { + return mdlerrors.NewBackend("modify module role", err) + } + if !ctx.Quiet { + fmt.Fprintf(ctx.Output, "Modified module role: %s.%s\n", s.Name.Module, s.Name.Name) + } + return nil + } return mdlerrors.NewAlreadyExists("module role", s.Name.Module+"."+s.Name.Name) } diff --git a/mdl/grammar/domains/MDLSecurity.g4 b/mdl/grammar/domains/MDLSecurity.g4 index bcd1c1e45..a49965547 100644 --- a/mdl/grammar/domains/MDLSecurity.g4 +++ b/mdl/grammar/domains/MDLSecurity.g4 @@ -10,8 +10,11 @@ options { tokenVocab = MDLLexer; } // SECURITY STATEMENTS // ============================================================================= +// OR MODIFY makes a security script re-runnable. Without it, re-executing the +// script that sets up roles fails on the first role that already exists, so +// role creation had to live in its own run-once file. createModuleRoleStatement - : CREATE MODULE ROLE qualifiedName (DESCRIPTION STRING_LITERAL)? + : CREATE (OR MODIFY)? MODULE ROLE qualifiedName (DESCRIPTION STRING_LITERAL)? ; dropModuleRoleStatement diff --git a/mdl/visitor/visitor_security.go b/mdl/visitor/visitor_security.go index fd91ff405..b09cb7213 100644 --- a/mdl/visitor/visitor_security.go +++ b/mdl/visitor/visitor_security.go @@ -14,7 +14,8 @@ func (b *Builder) ExitCreateModuleRoleStatement(ctx *parser.CreateModuleRoleStat return } stmt := &ast.CreateModuleRoleStmt{ - Name: buildQualifiedName(qn), + Name: buildQualifiedName(qn), + CreateOrModify: ctx.MODIFY() != nil, } if ctx.DESCRIPTION() != nil { if sl := ctx.STRING_LITERAL(); sl != nil { diff --git a/mdl/visitor/visitor_security_or_modify_test.go b/mdl/visitor/visitor_security_or_modify_test.go new file mode 100644 index 000000000..22521e404 --- /dev/null +++ b/mdl/visitor/visitor_security_or_modify_test.go @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// mxcli-formula1 suggested issue 12: `create module role` had no `or modify` +// form, so a security script failed on the first role that already existed and +// role creation had to live in its own run-once file. +func TestCreateModuleRole_OrModify(t *testing.T) { + cases := []struct { + src string + wantOrModify bool + wantName, desc string + }{ + {"create module role Sec.ApiUser;", false, "ApiUser", ""}, + {"create or modify module role Sec.ApiUser;", true, "ApiUser", ""}, + {"create or modify module role Sec.ApiUser description 'API consumer';", true, "ApiUser", "API consumer"}, + } + for _, tc := range cases { + t.Run(tc.src, func(t *testing.T) { + prog, errs := Build(tc.src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + var got *ast.CreateModuleRoleStmt + for _, s := range prog.Statements { + if r, ok := s.(*ast.CreateModuleRoleStmt); ok { + got = r + } + } + if got == nil { + t.Fatal("no CreateModuleRoleStmt produced") + } + if got.CreateOrModify != tc.wantOrModify { + t.Errorf("CreateOrModify = %v, want %v", got.CreateOrModify, tc.wantOrModify) + } + if got.Name.Name != tc.wantName { + t.Errorf("Name = %q, want %q", got.Name.Name, tc.wantName) + } + if got.Description != tc.desc { + t.Errorf("Description = %q, want %q", got.Description, tc.desc) + } + }) + } +} + +// `create module` and `create module role` differ only after the third token, so +// the optional OR MODIFY must not make one shadow the other. +func TestCreateModuleAndModuleRoleStayDistinct(t *testing.T) { + prog, errs := Build("create or modify module Sec;\ncreate or modify module role Sec.ApiUser;") + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + var modules, roles int + for _, s := range prog.Statements { + switch s.(type) { + case *ast.CreateModuleStmt: + modules++ + case *ast.CreateModuleRoleStmt: + roles++ + } + } + if modules != 1 || roles != 1 { + t.Errorf("got %d module and %d module-role statements, want 1 each", modules, roles) + } +} From 3e6008082b77e52dcbd2601c015d82f5b55485dd Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 01:41:15 +0000 Subject: [PATCH 18/31] docs(fix-issue): six symptom rows from the formula1 OData batch Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .claude/skills/fix-issue.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 4998ccfbf..bf2180abb 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -402,3 +402,9 @@ extracting `OffsetExpression`/`LimitExpression`. | `$Total = 5;` does not parse — `no viable alternative at input '$Total=5'` — while `DECLARE $Total Integer = 0;` does, and so do `$X = HEAD($List)`, `$X = create M.E (…)` and `$X = execute database query …`. The error names the token, not the missing keyword | Assignment existed only as a **prefix** on specific activity statements (`(VARIABLE EQUALS)?` on CALL/CREATE/RETRIEVE/…), plus a `SET $Var = expression` statement. A plain value therefore required `SET`, which nothing in the error or the surrounding syntax suggested | `mdl/grammar/domains/MDLMicroflow.g4` (`setStatement : SET? …`), `cmd/mxcli/syntax/features_microflow.go` | Make the guessable form work rather than improve the error: `SET` is now optional and both spellings produce the same `MfSetStmt`/`ChangeVariableAction`. **Prove a grammar relaxation causes no regressions with a control binary, not by reading**: `git stash` the `.g4`, `make grammar`, build `bin/mxcli-control`, sweep every `mdl-examples/**/*.mdl` with both — 13 scripts fail, the *same* 13, all pre-existing. ANTLR's adaptive prediction picks the activity-prefixed alternatives over `setStatement` on its own; no ordering change was needed. Executed against a real .mpr, mxbuild reports 0 errors. Tests `mdl/visitor/visitor_microflow_bare_assign_test.go` (bare and keyword forms must agree on the AST, not merely both parse). mxcli-formula1 #13 | | `mxcli test tests/ -p app/App.mpr` fails with "no such file or directory" for a `tests/` that sits right next to the `.mpr` | Test paths resolved against the process CWD only. Defensible in isolation, but mxcli otherwise encourages naming the project (`-p`) rather than standing in its directory, and project auto-discovery searches outward — so the two conventions collide and the failure looks like a missing directory | `cmd/mxcli/cmd_test_run.go` (`resolveTestPaths`) | Fall back to project-relative **only when the CWD-relative path does not exist**: a `tests/` in both places must resolve to the one the user is standing in, since silently preferring the project's copy would run the wrong suite. A path that exists in neither is passed through unchanged so the error names what was typed, not a rewritten path the user never mentioned. Tests `cmd/mxcli/cmd_test_run_paths_test.go`. mxcli-formula1 #13 | | After `SET` became optional, `mx check` on a project built from `02-microflow-examples.mdl` reports six errors that no MDL change caused: `[CE0109] "Undefined variable 'ProductList.Price'."` at four Aggregate list activities, and `[CE0015] "Aggregate function must specify a valid attribute."` at the expression-based one. Identical on both engines. `mxcli check` on the same script is silent | Two conversions existed for one syntax. `$Sum = sum($List.Price)` used to reach the dedicated `aggregateListStatement` rule; making `SET` optional put `setStatement` — alternative 5 of ~50 — in front of it, so ANTLR matched the lower-numbered alternative and the statement fell through to `buildSetStatement`'s fallback conversion, which joined list and attribute into one name and dropped the per-item expression entirely. Underneath, `buildListAggregateAsFunction` never appended the expression argument, so the SET path could not have seen it either | `mdl/grammar/domains/MDLMicroflow.g4` (`setStatement` moved LAST in `microflowStatement`), `mdl/visitor/visitor_microflow_statements.go` (`buildSetAggregate` replaces `extractVariableAndAttribute`), `mdl/visitor/visitor_microflow_expression.go` (`buildListAggregateAsFunction` appends the expression argument) | **A permissive alternative belongs last.** `$X = ` overlaps every `VARIABLE EQUALS ` statement in the rule — aggregates, list operations, RANGE — and ANTLR's ALL(*) picks the lowest-numbered alternative that matches, so a new general form silently steals from every specific one above it. **The measurement trap that let this ship**: the `SET?` change was swept with a control binary over every `mdl-examples/**/*.mdl` and found no difference — but with `mxcli check`, which parses and validates and never serializes. This defect lives between the AST and the BSON, where only `exec` + `mx check` can see it. Sweeping with `check` proves the grammar still *parses*; it proves nothing about what the visitor *builds*. For a grammar change, the control sweep must run the integration gate (`go test -tags integration -run TestMxCheck_DoctypeScripts`), not `check`. Fix proven by reverting both halves and watching the new tests fail with the reported symptom. Tests `mdl/visitor/visitor_microflow_aggregate_test.go`. Upstream CI on the ako→mendixlabs sync PR | +| A published service will not build: every whole-number attribute is `[CE5016] "Attribute … has type Integer, but is published as Edm.Int32"`, and an exposed enumeration adds CE5016 plus `[CE4583] "Enumeration 'X' is not published in this service."` | `mendixAttrTypeToEdm` mapped Integer→Int32 (Mendix publishes it as **Int64**, same as Long), and the enum path wrote `Edm.String` while `EnumerationAsString` was hardcoded `false` — the one combination Mendix rejects, since with the flag false it wants the enumeration published as its own EDM enum type. The function's own comment flagged the unverified rows, and the existing unit test *pinned the wrong answer* | `mdl/executor/cmd_odata.go` (`mendixAttrTypeToEdm`, `enumPublishedAsString`, `publishedAttrType`), `model/types.go` (`PublishedMember.EnumerationAsString`), `mdl/backend/modelsdk/odata_write.go` + `sdk/mpr/writer_odata.go` (stop hardcoding the flag) | **Let mxbuild adjudicate the whole table at once**: publish one attribute of every Mendix type in one service and read the CE5016s off the build. That found Integer (reported) *and* Enumeration (only suspected), and confirmed String/Long/Decimal/Boolean/DateTime were already right — five verified rows for one build. Binary turns out to be unpublishable at all (CE5013), whatever type you give it. **A type and a flag that only work as a pair must travel as a pair** — `Edm.String` is ambiguous between String and a flattened enum, so the flag is the only thing distinguishing them and it belongs on the same struct. Watch for an existing test that encodes the bug: this one asserted `Edm.Int32`, so the fix *failed the suite* until the assertion was corrected. Tests `cmd_contract_test.go`, `cmd_odata_edm_type_test.go`. mxcli-formula1 #16 | +| `create or modify external entity Mod.E (… Countable: false)` — touching only an entity-level property — detonates every attribute: `[CE6612] "Attribute 'circuitId' of external entity 'Stg_Circuit' is not supported."`, one per attribute, leaving a project that cannot build | Not the executor: it already preserves attributes it was not asked to change (`if len(attrs) > 0`). One layer down, `attributeFromGen` handled `StoredValue` and `OqlViewValue` but **not** `Rest$ODataMappedValue`, so every attribute of an external entity read back with no `RemoteName`, and the writer's `isExternal && a.RemoteName != ""` arm then fell through to a plain StoredValue on the next read-modify-write | `mdl/backend/modelsdk/domainmodel.go` (`attributeFromGen` gains the `ODataMappedValue` / `ODataMappedPrimitiveCollectionValue` arms) | **The attribute-level half of #782**, which fixed the entity level and stopped there — when a read-modify-write loses data, check every *nesting level* of the read, not just the one named in the report. A polymorphic `Value` switch that silently ignores a variant is the shape to look for: it compiles, it reads, and it drops. Reproduce with a **local metadata file** (`MetadataUrl: './contract.xml'`) — no server needed, and the import is the same code path. Also learned here: the reported attribute *rename* (`name` → `Stg_Circuitname`) is a different thing entirely — it happens at import, from `reservedEntityAttrNames`, and `name` is **not** actually reserved (verified: Mendix builds an external entity with an attribute literally named `name`). Tests `external_entity_read_test.go`. mxcli-formula1 #25 | +| `execute database query … dynamic $Sql` reaches the runtime as the literal string `'$Sql'` — `Parser Error: syntax error at or near "$"` from the database, not from Mendix. Runtime-built SQL, and therefore query pushdown, is impossible | The builder quoted any dynamic query not already starting with a quote — right for `dynamic 'SELECT …'`, wrong for an expression — and the AST kept one `DynamicQuery` string whichever branch of the grammar produced it, so nothing downstream could tell them apart | `mdl/ast/ast_microflow.go` (`DynamicQueryIsExpression`), `mdl/visitor/visitor_microflow_actions.go` (set it in the `expr` branch), `mdl/executor/cmd_microflows_builder_calls.go` (`dynamicQueryExpression`) | **When a grammar has two alternatives that mean different things, the AST must record which one fired** — a shared field plus a "does it look quoted?" heuristic is a guess, and the workaround users find (`dynamic '' + $Sql`, which starts with a quote so the heuristic leaves it alone) is proof the heuristic is the bug. Verified by reading the stored BSON rather than by describe: `DynamicQuery\x00\x05\x00\x00\x00$Sql` — five bytes, no quotes. Tests `cmd_microflows_dynamic_query_test.go`. mxcli-formula1 #21 | +| `create odata client` against a service behind `authentication basic` prints `Warning: could not fetch $metadata: … HTTP 401`, creates the client anyway, and the following `create external entities from …` imports nothing — from a script that reports success | The statement's `HttpUsername`/`HttpPassword`/`HEADERS` are stored for the runtime, but the design-time fetch was a bare `client.Get`. The fetch failure is only a warning, so the empty client propagates silently | `mdl/executor/cmd_odata.go` (`metadataFetchAuth`, `metadataAuthFromStmt`, `fetchODataMetadata`), `mdl/ast/ast_odata.go` + `mdl/visitor/visitor_odata.go` (`HttpUsernameIsLiteral` / `HeaderDef.ValueIsLiteral`) | **Only a literal is usable at design time.** The visitor strips a quoted literal's quotes, so `'f1api'` and `Module.ApiUser` both arrive as bare strings — the AST has to record which was written, or mxcli sends a *constant's name* as the password. Unresolved names are reported instead, which is also the honest answer: mxcli has no runtime to resolve a constant against. **A warning on a step something else silently depends on needs to say what breaks next** — the message now names the empty client and the import that will do nothing. Verified against a real basic-auth server that 401s without credentials and 403s without the custom header, so both had to arrive. Tests `cmd_odata_metadata_auth_test.go`. mxcli-formula1 #23 | +| Re-running `create or modify odata service` after editing a `publish entity` block changes nothing — the served `$metadata` is identical, and only `drop odata service` + create picks the edit up | The modify branch updated the service's scalar properties and never touched `EntityTypes` / `EntitySets` | `mdl/executor/cmd_odata.go` (modify branch rebuilds published entities via `astEntityDefToModel`, and carries `AllowedModuleRoles` through) | **Replace, don't merge**: a member removed from the script has to leave the service, which merging cannot express — the script is the description of the service. **Carry through what the statement cannot express**: role grants come from a separate `grant access on odata service` and would otherwise be dropped by a modify (reported; *not* reproduced on 11.12.1 — kept as a guard, and the commit says so rather than claiming a fix). Verified: same script yields `Label as 'label'` before and `Label as 'label' (Filterable, Sortable)` after, build stays at 0 errors. Tests `cmd_odata_modify_members_test.go`. mxcli-formula1 #26 | +| `create external entities from` a contract that restricts capabilities produces a project that will not build: `'Seasons' is marked Countable=False in the OData service, but True in the app`, `'latitude' is marked Filterable=False …` — one per restricted resource or property | Insert/Update/Delete restrictions were parsed; **Count/Filter/Sort were not**, so the import had nothing to honour and defaulted all three to true — on the one command whose entire job is fidelity to the contract | `mdl/types/edmx.go` (`EdmEntitySet.Countable`, `NonFilterableProperties`, `NonSortableProperties` + the three `applyCapabilityAnnotations` arms), `mdl/executor/cmd_contract.go` | An unannotated set still means countable/filterable/sortable — **silence in a contract is not a restriction**, it is OData's own default, so `nil` and `false` must stay distinguishable (`*bool`, as with the publish-side query options). The generated entity is compared against the contract at *build* time, so anything the contract can say is something the importer must be able to read. Tests `mdl/types/edmx_test.go`. mxcli-formula1 #24 | From a5cf260419eb9b99152cc3b7b00214f8a7141037 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 11:15:16 +0000 Subject: [PATCH 19/31] fix(microflows): write the InheritanceSplit and its case values MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `split type` produced a project mxbuild could not load at all: KeyNotFoundException: The given key '' was not present in the dictionary at StreamingBsonUnitReader.ResolvePostponedProperties() `mxcli check` passed and `mxcli exec` reported success. Reproduced on Mendix 11.6.6 and 11.13.0. Found while testing whether an enum-split `else` is version-dependent. Two gaps in the modelsdk writer, both the #791 shape — an object dropped at serialization while the sequence flows pointing at it are still written: 1. microflowObjectToGen had no *microflows.InheritanceSplit case, so the split hit `default: return nil` and vanished. Three flows referenced its $ID; that is the dangling pointer the loader trips on. 2. caseValueToGen had no InheritanceCase case, so every branch degraded to a bare Microflows$NoCase and lost the entity it selects on. Its value-receiver normalisation omitted the type as well, so handling only the pointer form would still have missed half the calls. Diagnosed with the recipe the symptom table already records for this class: dump the microflow, collect every $ID, check each key ending in `Pointer` resolves. Before: 27 objects, 10 pointers, 3 dangling. After: 28, 10, 0. Field list taken from the generated type rather than the legacy serializer. Legacy writes ErrorHandlingType on the split, but initInheritanceSplit has no such property — Mendix does not define it there — so the codec omits it. Verified end-to-end: the repro script now reports 0 errors on both 11.6.6 and 11.13.0, where it previously could not be loaded. Two modelling rules were confirmed on both versions along the way and are recorded in the repro: a type split needs an outgoing flow for every type INCLUDING the base entity (CE0090 otherwise), and an `else` does not substitute for the base-type case. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- .claude/skills/fix-issue.md | 1 + .../bug-tests/split-type-dangling-pointer.mdl | 47 +++++++++++ .../microflow_inheritance_write_test.go | 78 +++++++++++++++++++ mdl/backend/modelsdk/microflow_write.go | 25 ++++++ 4 files changed, 151 insertions(+) create mode 100644 mdl-examples/bug-tests/split-type-dangling-pointer.mdl create mode 100644 mdl/backend/modelsdk/microflow_inheritance_write_test.go diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index bd06b564e..3f830f4c0 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -395,3 +395,4 @@ extracting `OffsetExpression`/`LimitExpression`. | `retrieve $L from Mod.Entity where [Attr = $Var/Mod.Assoc/Attr]` passes `mxcli check` AND `mxcli exec`, then the build fails **CE0161** "Error(s) in XPath constraint" | Mendix XPath reaches at most ONE hop off a variable, and nothing checked the hop count | `mdl/executor/validate_microflow.go` (`checkXPathVariableTraversal`, `xpathVarTraversalRe`), called from the `*ast.RetrieveStmt` arm beside MDL047/MDL048 | Add **MDL055**, error severity, matching a `$var`-rooted path with **2+ segments**. **Establish the boundary against mxbuild first — it is narrower than it looks**: `$Var/Attr` VALID, `$Var/Mod.Assoc` VALID (one hop to the associated object), `$Var/Mod.Assoc/Attr` CE0161. A rule keying on "a module-qualified segment follows a variable" would reject the middle form, which is legal; key on hop count instead. Verified on 11.6.6 by building all three and dropping the offender to confirm the other two are clean. **Reject, don't try to serialize** — there is no valid XPath for the two-hop form, so the constraint must be restructured and only the author knows which way. **Verify the suggestion you emit**: both recommended rewrites (`retrieve $Related from $Var/Mod.Assoc;` then constrain on `$Related/Attr`; or invert to `[Mod.Assoc/Mod.Entity = $Var]`) were built and confirmed at 0 errors before the message claimed "both forms build clean". Negative test `mdl-examples/bug-tests/831-xpath-variable-traversal.fail.mdl` + `-ok.mdl`; test `TestXPathVariableTraversal`. Issue #831 | | `mxcli check` reports a microflow rule (e.g. MDL048 `[id = $StringVar]`) but `mxcli exec` writes the same script silently — a script that skips `check` produces a project the build rejects | Two different validators. The exec path called `ValidateMicroflowBody` (semantic errors); the MDL0xx rule set lives in `ValidateMicroflow`, which was wired only into `cmd/mxcli/cmd_check.go` and the LSP. Same shape as #836, where a guard existed on every exec path but was never reached from validate | `mdl/executor/validate.go` (`validateMicroflowRules`, `execEnforcedMicroflowRules`) called from `mdl/executor/cmd_microflows_create.go` — the handler, mirroring `ValidateEntity` in `cmd_entities.go` (NOT `validateWithContext`, which would double-report under `check --references`) | Promote an **explicit allowlist** of rules verified against a real mxbuild, never the whole set. **Blanket promotion was tried and reverted**: it turns every error-severity rule into a write barrier, and `MDL009` ("enumeration splits require exactly one value per branch") was a **false positive** — a multi-value branch covering every enum value builds at 0 errors on 11.6.6, and the shipped `write-microflows` skill documents that very form (since retired, see the MDL056 row). It also broke an existing repo test whose fixture uses `else` on an enum split. `MDL008` by contrast IS correct (mxbuild: CE0079 per uncovered value + CE0773) — so before promoting any rule, build its construct and read the verdict. Warnings are never promoted. Tests `TestValidateMicroflowRules_ReachedFromExec` and `…_UnverifiedRulesNotPromoted` (the latter fails if the allowlist is widened carelessly). Issue #833 | | `mxcli check` errors **MDL009** "enumeration splits require exactly one value per branch" on `when Open, Pending then` — but Mendix accepts it, so check rejects valid MDL and contradicts the shipped `write-microflows` skill | The rule asserted the opposite of the platform's behaviour. Nobody had built the construct: a multi-value branch covering every value **plus `(empty)`** builds at 0 errors on 11.6.6 | `mdl/executor/validate_microflow.go` (the `*ast.EnumSplitStmt` arm; `checkEnumSplitEmptyBranch`) | Retire the assertion and replace it with what actually fails: an enum split needs an outgoing flow per condition value, so a missing branch is **CE0079**. **MDL056** checks the `(empty)` branch — universal, verified to hold even on a `not null` enum attribute, so it needs no enumeration lookup and works from the statement alone. Full value coverage (the other half of CE0079) is deliberately NOT implemented: it needs the enum's member list, i.e. resolving the split variable's type against script or project, which `ValidateMicroflow` cannot see — guessing would trade one false positive for another. **Use a NEW rule ID rather than repurposing**, so anything citing the old number still means the old, wrong thing. `MDL008` (no `else`) is correct and stays — mxbuild gives CE0079 per uncovered value **plus** CE0773 on the else flow. Fix the skill in the same change: it documented the invalid `else` form. Tests `TestValidateMicroflow_EnumSplitMultipleValuesAllowed` / `…RequiresEmptyBranch`; repro `mdl-examples/bug-tests/mdl009-enum-split-empty-branch.fail.mdl` + `-ok.mdl` | +| A microflow using `split type` writes a project mxbuild cannot **load**: `KeyNotFoundException: The given key '' was not present in the dictionary` at `StreamingBsonUnitReader.ResolvePostponedProperties`. `mxcli check` ✓ and `mxcli exec` ✓; reproduced on 11.6.6 and 11.13.0 | Two gaps in the modelsdk writer, both the #791 shape. (1) `microflowObjectToGen` had no `*microflows.InheritanceSplit` case → `default: return nil`, so the split was dropped while three sequence flows kept pointing at its `$ID`. (2) `caseValueToGen` had no `InheritanceCase` case → every branch degraded to a bare `Microflows$NoCase`, losing the entity it selects on. Its value-receiver normalisation also omitted the type, so a pointer-only fix would still miss half the calls | `mdl/backend/modelsdk/microflow_write.go` (`microflowObjectToGen`, `caseValueToGen`) — mirror `sdk/mpr/writer_microflow.go` | Add both cases. **Diagnose with the #791 recipe**: `mxcli bson dump --type microflow`, collect every `$ID`, check each key ending in `Pointer` resolves (before: 27 objects / 3 dangling; after: 28 / 0). **Take field lists from the GENERATED type, not from legacy** — legacy writes `ErrorHandlingType` on the split but `initInheritanceSplit` has no such property, i.e. legacy writes a field Mendix does not define. **When adding a case-value type, update the value-receiver normalisation too.** Modelling rules confirmed on both versions while verifying: a type split needs an outgoing flow for every type INCLUDING the base (CE0090), and an `else` does NOT substitute for the base-type case. Tests `TestMicroflowRoundTrip_InheritanceSplit`, `TestCaseValueToGen_InheritanceCase{,ValueReceiver}`; repro `mdl-examples/bug-tests/split-type-dangling-pointer.mdl` | diff --git a/mdl-examples/bug-tests/split-type-dangling-pointer.mdl b/mdl-examples/bug-tests/split-type-dangling-pointer.mdl new file mode 100644 index 000000000..7b42908db --- /dev/null +++ b/mdl-examples/bug-tests/split-type-dangling-pointer.mdl @@ -0,0 +1,47 @@ +-- ============================================================================ +-- `split type` wrote a project mxbuild could not LOAD +-- ============================================================================ +-- +-- `mxcli check` passed and `mxcli exec` reported success, but `mx check` died +-- before validating anything: +-- +-- ERROR: System.Collections.Generic.KeyNotFoundException: The given key +-- '' was not present in the dictionary +-- at StreamingBsonUnitReader.ResolvePostponedProperties() +-- +-- Reproduced on Mendix 11.6.6 and 11.13.0. Two gaps in the modelsdk writer, +-- both the #791 shape — an object dropped at serialization while the sequence +-- flows pointing at it were still written: +-- +-- 1. microflowObjectToGen had no *microflows.InheritanceSplit case, so the +-- split itself vanished. Three flows referenced its $ID. +-- 2. caseValueToGen had no InheritanceCase case, so every branch degraded to +-- a bare Microflows$NoCase and lost the entity it selects on. +-- +-- Diagnosed with the #791 recipe: dump the microflow, collect every $ID, and +-- check each key ending in `Pointer` resolves. Before: 27 objects, 10 pointers, +-- 3 dangling. After: 28 objects, 10 pointers, 0 dangling. +-- +-- A type split must give every type an outgoing flow, INCLUDING the base type +-- (CE0090 otherwise). An `else` does NOT substitute for the base-type case — +-- verified on both versions. +-- +-- To verify: run this script, then `mx check` — 0 errors. +-- ============================================================================ + +CREATE MODULE BugSplit; + +CREATE OR MODIFY PERSISTENT ENTITY BugSplit.Animal ( Name: String(50) ); + +CREATE OR MODIFY PERSISTENT ENTITY BugSplit.Dog EXTENDS BugSplit.Animal ( Breed: String(50) ); + +CREATE OR MODIFY MICROFLOW BugSplit.Classify ($A: BugSplit.Animal) +RETURNS String +BEGIN + split type $A + case BugSplit.Dog + cast $d; + case BugSplit.Animal + end split; + return 'done'; +END; diff --git a/mdl/backend/modelsdk/microflow_inheritance_write_test.go b/mdl/backend/modelsdk/microflow_inheritance_write_test.go new file mode 100644 index 000000000..50f9b1892 --- /dev/null +++ b/mdl/backend/modelsdk/microflow_inheritance_write_test.go @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// TestMicroflowRoundTrip_InheritanceSplit covers a corruption found while +// testing enum-split `else` across versions: `split type` produced a project +// mxbuild could not LOAD at all — +// +// KeyNotFoundException: The given key '' was not present in the +// dictionary at StreamingBsonUnitReader.ResolvePostponedProperties() +// +// on both 11.6.6 and 11.13.0, while `mxcli check` passed. Two gaps, both the +// #791 shape (an object dropped at serialization while the flows pointing at +// it were written): +// +// 1. microflowObjectToGen had no *microflows.InheritanceSplit case, so the +// split hit `default: return nil` and vanished. Three sequence flows +// referenced its $ID — that is the dangling pointer the loader trips on. +// 2. caseValueToGen had no InheritanceCase case, so every branch flow got a +// bare NoCase and the entity each branch selects on was lost. +func TestMicroflowRoundTrip_InheritanceSplit(t *testing.T) { + split := µflows.InheritanceSplit{VariableName: "A", Caption: "split"} + split.ID = model.ID("split-1") + + mf := µflows.Microflow{ + Name: "TypeSplit", + ObjectCollection: µflows.MicroflowObjectCollection{ + Objects: []microflows.MicroflowObject{split}, + }, + } + mf.ID = model.ID("mf-1") + + got := roundTripMicroflow(t, mf) + + var found *microflows.InheritanceSplit + if got.ObjectCollection != nil { + for _, obj := range got.ObjectCollection.Objects { + if s, ok := obj.(*microflows.InheritanceSplit); ok { + found = s + } + } + } + if found == nil { + t.Fatal("InheritanceSplit did not survive the round trip — the object is dropped at " + + "serialization while flows still point at its $ID, which is the KeyNotFoundException") + } + if found.VariableName != "A" { + t.Errorf("VariableName = %q, want A", found.VariableName) + } +} + +// The branch's case value must round-trip as an InheritanceCase naming the +// entity, not degrade to a NoCase. +func TestCaseValueToGen_InheritanceCase(t *testing.T) { + el := caseValueToGen(µflows.InheritanceCase{EntityQualifiedName: "SP.Dog"}) + if el == nil { + t.Fatal("caseValueToGen returned nil for an InheritanceCase") + } + if got := el.TypeName(); got != "Microflows$InheritanceCase" { + t.Fatalf("$Type = %q, want Microflows$InheritanceCase (a NoCase loses the branch entity)", got) + } +} + +// The visitor sometimes yields value receivers; those must dispatch the same +// way, exactly as the existing normalisation does for EnumerationCase. +func TestCaseValueToGen_InheritanceCaseValueReceiver(t *testing.T) { + el := caseValueToGen(microflows.InheritanceCase{EntityQualifiedName: "SP.Dog"}) + if el == nil || el.TypeName() != "Microflows$InheritanceCase" { + t.Fatalf("value-receiver InheritanceCase degraded to %v", el) + } +} diff --git a/mdl/backend/modelsdk/microflow_write.go b/mdl/backend/modelsdk/microflow_write.go index 05db0d046..d56947cb4 100644 --- a/mdl/backend/modelsdk/microflow_write.go +++ b/mdl/backend/modelsdk/microflow_write.go @@ -307,6 +307,21 @@ func microflowObjectToGen(obj microflows.MicroflowObject) element.Element { g.SetSplitCondition(sc) } return g + case *microflows.InheritanceSplit: + // Without this the split hit `default: return nil` and was dropped, while + // the sequence flows referencing its $ID were still written — a dangling + // pointer that mxbuild cannot even load ("KeyNotFoundException ... at + // StreamingBsonUnitReader.ResolvePostponedProperties"). Same shape as the + // ErrorEvent/BreakEvent gap in #791. Fields mirror the legacy serializer + // in sdk/mpr/writer_microflow.go. + g := genMf.NewInheritanceSplit() + g.SetID(element.ID(o.ID)) + g.SetCaption(o.Caption) + g.SetDocumentation(o.Documentation) + g.SetRelativeMiddlePoint(pointStr(o.Position)) + g.SetSize(sizeStr(o.Size)) + g.SetSplitVariableName(o.VariableName) + return g case *microflows.ExclusiveMerge: g := genMf.NewExclusiveMerge() g.SetID(element.ID(o.ID)) @@ -1087,6 +1102,8 @@ func caseValueToGen(cv microflows.CaseValue) element.Element { cv = &c case microflows.NoCase: cv = &c + case microflows.InheritanceCase: + cv = &c } switch c := cv.(type) { case *microflows.EnumerationCase: @@ -1099,6 +1116,14 @@ func caseValueToGen(cv microflows.CaseValue) element.Element { g.SetID(element.ID(c.ID)) g.SetValue(c.Expression) return g + case *microflows.InheritanceCase: + // A type-split branch selects on an entity. Without this it fell through + // to NoCase, so every branch lost the entity it matches on — the second + // half of the `split type` corruption. + g := genMf.NewInheritanceCase() + g.SetID(element.ID(c.ID)) + g.SetValueQualifiedName(c.EntityQualifiedName) + return g default: return genMf.NewNoCase() } From 4291f1dfa46ea9c9465df31822492bc2dd1547f1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 11:30:43 +0000 Subject: [PATCH 20/31] fix(external-entities): stop renaming an attribute called `name` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An attribute named `name` came out of CREATE EXTERNAL ENTITIES prefixed with the remote type — Stg_Drivername, Circuitname — so a page written against the published $metadata failed with "The selected attribute 'F1Live.Drivers.name' no longer exists", and the same field carried a different name in every module because the remote type names differ. `name` was simply not reserved. Adjudicated on 11.12.1 by importing a contract with a property for each name on the list and prefixing disabled: Mendix answered CE7247 "The name 'x' is a reserved word" for id, owner, changedBy, changedDate, createdDate, type and context — and said nothing about `name`. So this is one wrong entry, not a scheme that needs redesigning; the other seven earn their place and keep it. The remaining renames are now reported at the end of an import instead of being discovered later when a page will not build. Note for existing projects: a re-import renames `Stg_Drivername` back to `name`, which is the point, but anything referring to the old name has to follow. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/executor/cmd_contract.go | 30 ++++++++++++-- mdl/executor/cmd_contract_reserved_test.go | 48 ++++++++++++++++++++++ mdl/executor/cmd_contract_test.go | 10 +++-- 3 files changed, 82 insertions(+), 6 deletions(-) create mode 100644 mdl/executor/cmd_contract_reserved_test.go diff --git a/mdl/executor/cmd_contract.go b/mdl/executor/cmd_contract.go index 51fc15745..9fb38eec5 100644 --- a/mdl/executor/cmd_contract.go +++ b/mdl/executor/cmd_contract.go @@ -444,13 +444,18 @@ func edmToMendixType(p *types.EdmProperty) string { // reservedEntityAttrNames are Mendix-reserved attribute names that must be // renamed when imported from an OData property of the same name. -// These names conflict with Mendix system members or runtime internals. // The check is case-insensitive (see attrNameForOData). +// +// Every entry is one Mendix rejects with CE7247 "The name 'x' is a reserved +// word." Verified on 11.12.1 by importing a contract with a property for each +// name and prefixing disabled: seven errors, one per name below. `name` was on +// this list and is NOT among them — an external entity with an attribute +// literally named `name` builds clean, and prefixing it mangled the commonest +// property in any contract (mxcli-formula1 #28). Do not add a name here without +// a CE7247 to point at. var reservedEntityAttrNames = map[string]bool{ // Mendix internal identifier "id": true, - // Mendix system-managed attribute for the object name (present on many entities) - "name": true, // System ownership association (HasOwner / System.owner) "owner": true, // System audit associations (HasChangedBy / System.changedBy) @@ -523,6 +528,10 @@ func createExternalEntities(ctx *ExecContext, s *ast.CreateExternalEntitiesStmt) serviceRef := s.ServiceRef.String() var created, updated, skipped, failed int + // Attribute names the import had to change because Mendix reserves them. + // Reported at the end so the local name never silently diverges from the + // contract; the mapping still points at the remote property either way. + var renamed []string for _, schema := range doc.Schemas { for _, et := range schema.EntityTypes { @@ -649,6 +658,12 @@ func createExternalEntities(ctx *ExecContext, s *ast.CreateExternalEntitiesStmt) } attrName := attrNameForOData(p.Name, et.Name) + if attrName != p.Name { + // Never let a rename be discovered later, when a page written + // against the published $metadata fails with "The selected + // attribute … no longer exists" (mxcli-formula1 #28). + renamed = append(renamed, fmt.Sprintf("%s.%s: %s -> %s", mendixName, p.Name, p.Name, attrName)) + } attr := &domainmodel.Attribute{ Name: attrName, Type: edmToDomainModelAttrType(p, keyPropSet[p.Name]), @@ -732,6 +747,15 @@ func createExternalEntities(ctx *ExecContext, s *ast.CreateExternalEntitiesStmt) fmt.Fprintf(ctx.Output, "\nFrom %s into %s: %d created, %d updated, %d skipped, %d failed\n", svcQN, targetModule, created, updated, skipped, failed) + if len(renamed) > 0 { + sort.Strings(renamed) + fmt.Fprintf(ctx.Output, "\n %d attribute name(s) changed — Mendix reserves the contract's spelling (CE7247),\n", len(renamed)) + fmt.Fprintf(ctx.Output, " so a page or expression must use the local name, not the one in $metadata:\n") + for _, r := range renamed { + fmt.Fprintf(ctx.Output, " %s\n", r) + } + } + return nil } diff --git a/mdl/executor/cmd_contract_reserved_test.go b/mdl/executor/cmd_contract_reserved_test.go new file mode 100644 index 000000000..bed6350da --- /dev/null +++ b/mdl/executor/cmd_contract_reserved_test.go @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import "testing" + +// mxcli-formula1 findings #28: an attribute named `name` came out of +// CREATE EXTERNAL ENTITIES prefixed with the remote type — `Stg_Drivername`, +// `Circuitname` — so a page written against the published $metadata failed with +// "The selected attribute 'F1Live.Drivers.name' no longer exists", and the same +// field carried a different name in every module because the remote type names +// differ. +// +// `name` was simply not reserved. Adjudicated on 11.12.1 by importing a contract +// with a property for every name on the list, prefixing disabled: Mendix +// answered CE7247 for seven of the eight and said nothing about `name`. +func TestAttrNameForOData(t *testing.T) { + // Every one of these is a CE7247 "The name 'x' is a reserved word." + for _, reserved := range []string{"id", "owner", "changedBy", "changedDate", "createdDate", "type", "context"} { + if got := attrNameForOData(reserved, "Driver"); got != "Driver"+reserved { + t.Errorf("attrNameForOData(%q) = %q, want it disambiguated — Mendix rejects the bare name with CE7247", reserved, got) + } + } + + // `name` is an ordinary attribute name and must survive untouched. So must + // anything else the contract happens to call a property. + for _, ok := range []string{"name", "driverRef", "surname", "nationality", "label"} { + if got := attrNameForOData(ok, "Driver"); got != ok { + t.Errorf("attrNameForOData(%q) = %q, want it unchanged — Mendix accepts it", ok, got) + } + } +} + +// The check is case-insensitive: a contract using Id or TYPE hits the same +// reserved word. +func TestAttrNameForOData_CaseInsensitive(t *testing.T) { + for _, v := range []string{"Id", "ID", "TYPE", "Owner"} { + if got := attrNameForOData(v, "Thing"); got == v { + t.Errorf("attrNameForOData(%q) left it unchanged; reserved words are case-insensitive", v) + } + } + // …but a name that merely contains one is fine. + for _, v := range []string{"identifier", "typeCode", "ownerName"} { + if got := attrNameForOData(v, "Thing"); got != v { + t.Errorf("attrNameForOData(%q) = %q, want unchanged — it is not the reserved word itself", v, got) + } + } +} diff --git a/mdl/executor/cmd_contract_test.go b/mdl/executor/cmd_contract_test.go index 8ca06d2b1..2a325c626 100644 --- a/mdl/executor/cmd_contract_test.go +++ b/mdl/executor/cmd_contract_test.go @@ -23,8 +23,6 @@ func TestAttrNameForOData_ReservedWords(t *testing.T) { // Already-covered names {"Id", "Photo", "PhotoId"}, {"id", "Photo", "Photoid"}, - {"Name", "Airline", "AirlineName"}, - {"name", "Airline", "Airlinename"}, // Newly-added reserved names (issue #526) {"Owner", "Trip", "TripOwner"}, {"owner", "Trip", "Tripowner"}, @@ -38,7 +36,13 @@ func TestAttrNameForOData_ReservedWords(t *testing.T) { {"changeddate", "Event", "Eventchangeddate"}, {"CreatedDate", "Event", "EventCreatedDate"}, {"createddate", "Event", "Eventcreateddate"}, - // Non-reserved names must pass through unchanged + // Non-reserved names must pass through unchanged. `name` belongs here: + // it was on the reserved list and is not reserved — verified on 11.12.1 + // by importing a contract with a property per listed name and prefixing + // disabled, which produced CE7247 for every other name and nothing for + // this one (mxcli-formula1 #28). + {"Name", "Airline", "Name"}, + {"name", "Airline", "name"}, {"AirlineCode", "Airline", "AirlineCode"}, {"Concurrency", "Airline", "Concurrency"}, {"FirstName", "Person", "FirstName"}, From 30327d7e6b1e5819636787462f9e64f3bf4992e3 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 11:30:59 +0000 Subject: [PATCH 21/31] feat(move): MOVE JAVA ACTION and MOVE ODATA SERVICE MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MOVE accepted seven doctypes and rejected the rest at parse time (`no viable alternative at input 'MOVEJAVA'`). Neither CREATE JAVA ACTION nor CREATE ODATA SERVICE takes a folder clause either, so those documents could never leave the module root from MDL — five of the reporting project's documents were stuck there while the other 36 sorted into folders. Both are plain document units, so each reduces to the existing reparent primitive: the executor sets ContainerID and calls the backend, which persists the containment row and touches nothing else. sdk/mpr's moveUnitByID is exported for the doctypes that have no dedicated writer method of their own. Verified on 11.12.1: `move java action` and `move odata service` into 'Support' and 'Api/Published' created exactly three folders (two levels for the nested path), left the document count unchanged, and the project still loads and builds. Full doctype integration gate green — mandatory for a grammar change, and it caught a bad first draft of the new example. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .claude/skills/mendix/organize-project.md | 9 +++ cmd/mxcli/syntax/features_misc.go | 7 ++ .../doctype-tests/18-folder-examples.mdl | 43 +++++++++++ mdl/ast/ast.go | 2 + mdl/backend/java.go | 2 + mdl/backend/mcp/unsupported_gen.go | 18 +++-- mdl/backend/mock/backend.go | 2 + mdl/backend/mock/mock_service.go | 21 +++++- mdl/backend/modelsdk/move_documents_write.go | 15 ++++ mdl/backend/modelsdk/unimplemented_gen.go | 13 ++++ mdl/backend/mpr/backend.go | 14 ++++ mdl/backend/service.go | 3 + mdl/executor/cmd_move.go | 66 +++++++++++++++++ mdl/grammar/MDLParser.g4 | 4 +- mdl/visitor/visitor_entity.go | 7 +- mdl/visitor/visitor_move_doctypes_test.go | 71 +++++++++++++++++++ sdk/mpr/writer_domainmodel.go | 7 ++ 17 files changed, 296 insertions(+), 8 deletions(-) create mode 100644 mdl/visitor/visitor_move_doctypes_test.go diff --git a/.claude/skills/mendix/organize-project.md b/.claude/skills/mendix/organize-project.md index a1a3008cf..4f28da0fc 100644 --- a/.claude/skills/mendix/organize-project.md +++ b/.claude/skills/mendix/organize-project.md @@ -168,8 +168,17 @@ move page OldModule.CustomerPage to NewModule; | Nanoflow | `folder 'path'` (keyword) | `move nanoflow ...` | | Snippet | `folder: 'path'` (property) | `move snippet ...` | | Enumeration | N/A | `move enumeration ...` | +| Constant | N/A | `move constant ...` | +| Database connection | N/A | `move database connection ...` | +| Java action | N/A | `move java action ...` | +| OData service (published) | N/A | `move odata service ...` | | Entity | N/A | `move entity ...` (module only, no folders) | +**Java actions and published OData services have no folder clause on `create`**, so +`move` is the only way to place them — before this they were stuck at the module +root forever. Both are plain document units, so the move is model-level only: it +changes containment and nothing else. + **Note:** Pages and snippets use property syntax (`folder: 'path'` inside parentheses). Microflows and nanoflows use keyword syntax (`folder 'path'` before `begin`). Entities are embedded in domain models and can only be moved to a different module (no folder support). ## Example: Reorganize a Module diff --git a/cmd/mxcli/syntax/features_misc.go b/cmd/mxcli/syntax/features_misc.go index 85560be1e..d6dd51fc8 100644 --- a/cmd/mxcli/syntax/features_misc.go +++ b/cmd/mxcli/syntax/features_misc.go @@ -211,6 +211,8 @@ SHOW STRUCTURE DEPTH 1 ALL;`, "move folder", "drop folder", }, Syntax: `MOVE Module.Name TO FOLDER 'Path'; +-- doctype: PAGE | MICROFLOW | NANOFLOW | SNIPPET | ENUMERATION | CONSTANT +-- | DATABASE CONNECTION | JAVA ACTION | ODATA SERVICE | ENTITY | FOLDER MOVE Module.Name TO TargetModule; MOVE OldModule.Name TO FOLDER 'Path' IN NewModule; MOVE FOLDER Module.FolderName TO FOLDER 'Path'; @@ -224,6 +226,11 @@ MOVE MICROFLOW MyModule.ACT_ProcessOrder TO FOLDER 'Orders/Processing'; -- Move entity to different module MOVE ENTITY OldModule.Customer TO NewModule; +-- Java actions and published OData services have no folder clause on CREATE, +-- so MOVE is the only way to place them +MOVE JAVA ACTION MyModule.ODataQuery TO FOLDER 'Support'; +MOVE ODATA SERVICE MyModule.PublicApi TO FOLDER 'Api/Published'; + -- Check impact before cross-module move SHOW IMPACT OF OldModule.CustomerPage; MOVE PAGE OldModule.CustomerPage TO NewModule; diff --git a/mdl-examples/doctype-tests/18-folder-examples.mdl b/mdl-examples/doctype-tests/18-folder-examples.mdl index bc2626cc2..18e5a7cec 100644 --- a/mdl-examples/doctype-tests/18-folder-examples.mdl +++ b/mdl-examples/doctype-tests/18-folder-examples.mdl @@ -89,4 +89,47 @@ drop folder 'Resources' in FolderTest; / -- cleanup +-- ============================================================================ +-- Level 6: Java actions and published OData services +-- ============================================================================ + +/** + * Neither CREATE JAVA ACTION nor CREATE ODATA SERVICE takes a folder clause, so + * MOVE is the only way these documents ever leave the module root. + */ +create java action FolderTest.QueryHelper () returns Boolean as $$ +public class QueryHelper { } +$$; + +move java action FolderTest.QueryHelper to folder 'Support/Java'; + +create non-persistent entity FolderTest.ApiRow ( RowKey: string(60) ); + +CREATE MICROFLOW FolderTest.Read_ApiRows () + RETURNS List of FolderTest.ApiRow AS $Rows +BEGIN + $Rows = CREATE LIST OF FolderTest.ApiRow; + RETURN $Rows; +END; + +create odata service FolderTest.PublicApi ( + path: 'odata/foldertest/', + version: '1.0.0', + ODataVersion: OData4, + namespace: 'FolderTest.PublicApi', + ServiceName: 'PublicApi' +) +{ + publish entity FolderTest.ApiRow as 'ApiRows' ( + ReadMode: microflow FolderTest.Read_ApiRows, + InsertMode: not_supported, + UpdateMode: not_supported, + DeleteMode: not_supported, + Countable: false + ) + expose ( RowKey as 'rowKey' (KEY) ); +}; + +move odata service FolderTest.PublicApi to folder 'Api/Published'; + drop module FolderTest; diff --git a/mdl/ast/ast.go b/mdl/ast/ast.go index dca56fb91..256d2cfd3 100644 --- a/mdl/ast/ast.go +++ b/mdl/ast/ast.go @@ -54,6 +54,8 @@ const ( DocumentTypeEnumeration DocumentType = "ENUMERATION" DocumentTypeConstant DocumentType = "CONSTANT" DocumentTypeDatabaseConnection DocumentType = "DATABASE CONNECTION" + DocumentTypeJavaAction DocumentType = "JAVA ACTION" + DocumentTypeODataService DocumentType = "ODATA SERVICE" ) // MoveStmt represents: MOVE PAGE/MICROFLOW/SNIPPET/NANOFLOW/ENTITY/ENUMERATION Module.Name TO FOLDER 'path' IN Module diff --git a/mdl/backend/java.go b/mdl/backend/java.go index 73a3e5d72..b906eabb9 100644 --- a/mdl/backend/java.go +++ b/mdl/backend/java.go @@ -12,6 +12,8 @@ import ( type JavaBackend interface { ListJavaActions() ([]*types.JavaAction, error) ListJavaActionsFull() ([]*javaactions.JavaAction, error) + // MoveJavaAction reparents a Java action to an already-updated ContainerID. + MoveJavaAction(ja *javaactions.JavaAction) error ListJavaScriptActions() ([]*types.JavaScriptAction, error) ReadJavaActionByName(qualifiedName string) (*javaactions.JavaAction, error) ReadJavaScriptActionByName(qualifiedName string) (*types.JavaScriptAction, error) diff --git a/mdl/backend/mcp/unsupported_gen.go b/mdl/backend/mcp/unsupported_gen.go index 35878e3b9..637e5e5d6 100644 --- a/mdl/backend/mcp/unsupported_gen.go +++ b/mdl/backend/mcp/unsupported_gen.go @@ -667,13 +667,13 @@ func (unsupportedBackend) ListFolders() (r0 []*types.FolderInfo, err1 error) { return } -func (unsupportedBackend) ListImageCollections() (r0 []*types.ImageCollection, err1 error) { - err1 = errUnsupported("ListImageCollections") +func (unsupportedBackend) ListIconCollections() (r0 []*types.IconCollection, err1 error) { + err1 = errUnsupported("ListIconCollections") return } -func (unsupportedBackend) ListIconCollections() (r0 []*types.IconCollection, err1 error) { - err1 = errUnsupported("ListIconCollections") +func (unsupportedBackend) ListImageCollections() (r0 []*types.ImageCollection, err1 error) { + err1 = errUnsupported("ListImageCollections") return } @@ -827,6 +827,11 @@ func (unsupportedBackend) MoveImportMapping(_ *model.ImportMapping) (err0 error) return } +func (unsupportedBackend) MoveJavaAction(_ *javaactions.JavaAction) (err0 error) { + err0 = errUnsupported("MoveJavaAction") + return +} + func (unsupportedBackend) MoveMicroflow(_ *microflows.Microflow) (err0 error) { err0 = errUnsupported("MoveMicroflow") return @@ -842,6 +847,11 @@ func (unsupportedBackend) MovePage(_ *pages.Page) (err0 error) { return } +func (unsupportedBackend) MovePublishedODataService(_ *model.PublishedODataService) (err0 error) { + err0 = errUnsupported("MovePublishedODataService") + return +} + func (unsupportedBackend) MoveSnippet(_ *pages.Snippet) (err0 error) { err0 = errUnsupported("MoveSnippet") return diff --git a/mdl/backend/mock/backend.go b/mdl/backend/mock/backend.go index 7930593ef..b2d9e8c62 100644 --- a/mdl/backend/mock/backend.go +++ b/mdl/backend/mock/backend.go @@ -185,6 +185,8 @@ type MockBackend struct { CreateDatabaseConnectionFunc func(conn *model.DatabaseConnection) error UpdateDatabaseConnectionFunc func(conn *model.DatabaseConnection) error MoveDatabaseConnectionFunc func(conn *model.DatabaseConnection) error + MoveJavaActionFunc func(ja *javaactions.JavaAction) error + MovePublishedODataServiceFunc func(svc *model.PublishedODataService) error DeleteDatabaseConnectionFunc func(id model.ID) error ListDataTransformersFunc func() ([]*model.DataTransformer, error) CreateDataTransformerFunc func(dt *model.DataTransformer) error diff --git a/mdl/backend/mock/mock_service.go b/mdl/backend/mock/mock_service.go index 6dbf0cfd1..cb37e74fd 100644 --- a/mdl/backend/mock/mock_service.go +++ b/mdl/backend/mock/mock_service.go @@ -2,7 +2,12 @@ package mock -import "github.com/mendixlabs/mxcli/model" +import ( + "errors" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/javaactions" +) func (m *MockBackend) ListConsumedODataServices() ([]*model.ConsumedODataService, error) { if m.ListConsumedODataServicesFunc != nil { @@ -172,6 +177,20 @@ func (m *MockBackend) MoveDatabaseConnection(conn *model.DatabaseConnection) err return nil } +func (m *MockBackend) MoveJavaAction(ja *javaactions.JavaAction) error { + if m.MoveJavaActionFunc != nil { + return m.MoveJavaActionFunc(ja) + } + return errors.New("MockBackend.MoveJavaAction not configured") +} + +func (m *MockBackend) MovePublishedODataService(svc *model.PublishedODataService) error { + if m.MovePublishedODataServiceFunc != nil { + return m.MovePublishedODataServiceFunc(svc) + } + return errors.New("MockBackend.MovePublishedODataService not configured") +} + func (m *MockBackend) DeleteDatabaseConnection(id model.ID) error { if m.DeleteDatabaseConnectionFunc != nil { return m.DeleteDatabaseConnectionFunc(id) diff --git a/mdl/backend/modelsdk/move_documents_write.go b/mdl/backend/modelsdk/move_documents_write.go index b93afa3cf..9919b7f03 100644 --- a/mdl/backend/modelsdk/move_documents_write.go +++ b/mdl/backend/modelsdk/move_documents_write.go @@ -6,6 +6,7 @@ import ( "fmt" "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/javaactions" "github.com/mendixlabs/mxcli/sdk/microflows" "github.com/mendixlabs/mxcli/sdk/pages" ) @@ -69,3 +70,17 @@ func (b *Backend) MoveDatabaseConnection(conn *model.DatabaseConnection) error { } return b.moveUnit(conn.ID, conn.ContainerID, "DatabaseConnection") } + +func (b *Backend) MoveJavaAction(ja *javaactions.JavaAction) error { + if ja == nil { + return fmt.Errorf("MoveJavaAction: nil java action") + } + return b.moveUnit(ja.ID, ja.ContainerID, "JavaAction") +} + +func (b *Backend) MovePublishedODataService(svc *model.PublishedODataService) error { + if svc == nil { + return fmt.Errorf("MovePublishedODataService: nil service") + } + return b.moveUnit(svc.ID, svc.ContainerID, "PublishedODataService") +} diff --git a/mdl/backend/modelsdk/unimplemented_gen.go b/mdl/backend/modelsdk/unimplemented_gen.go index c0a682247..ce24b4208 100644 --- a/mdl/backend/modelsdk/unimplemented_gen.go +++ b/mdl/backend/modelsdk/unimplemented_gen.go @@ -595,6 +595,11 @@ func (unimplemented) ListFolders() ([]*types.FolderInfo, error) { return r0, errUnimplemented("ListFolders") } +func (unimplemented) ListIconCollections() ([]*types.IconCollection, error) { + var r0 []*types.IconCollection + return r0, errUnimplemented("ListIconCollections") +} + func (unimplemented) ListImageCollections() ([]*types.ImageCollection, error) { var r0 []*types.ImageCollection return r0, errUnimplemented("ListImageCollections") @@ -744,6 +749,10 @@ func (unimplemented) MoveImportMapping(_ *model.ImportMapping) error { return errUnimplemented("MoveImportMapping") } +func (unimplemented) MoveJavaAction(_ *javaactions.JavaAction) error { + return errUnimplemented("MoveJavaAction") +} + func (unimplemented) MoveMicroflow(_ *microflows.Microflow) error { return errUnimplemented("MoveMicroflow") } @@ -756,6 +765,10 @@ func (unimplemented) MovePage(_ *pages.Page) error { return errUnimplemented("MovePage") } +func (unimplemented) MovePublishedODataService(_ *model.PublishedODataService) error { + return errUnimplemented("MovePublishedODataService") +} + func (unimplemented) MoveSnippet(_ *pages.Snippet) error { return errUnimplemented("MoveSnippet") } diff --git a/mdl/backend/mpr/backend.go b/mdl/backend/mpr/backend.go index fe3c68050..4daeaa705 100644 --- a/mdl/backend/mpr/backend.go +++ b/mdl/backend/mpr/backend.go @@ -6,6 +6,8 @@ package mprbackend import ( + "errors" + "github.com/mendixlabs/mxcli/mdl/backend" "github.com/mendixlabs/mxcli/mdl/linter" "github.com/mendixlabs/mxcli/mdl/types" @@ -501,6 +503,18 @@ func (b *MprBackend) UpdateDatabaseConnection(conn *model.DatabaseConnection) er func (b *MprBackend) MoveDatabaseConnection(conn *model.DatabaseConnection) error { return b.writer.MoveDatabaseConnection(conn) } +func (b *MprBackend) MoveJavaAction(ja *javaactions.JavaAction) error { + if ja == nil { + return errors.New("MoveJavaAction: nil java action") + } + return b.writer.MoveUnitByID(string(ja.ID), string(ja.ContainerID)) +} +func (b *MprBackend) MovePublishedODataService(svc *model.PublishedODataService) error { + if svc == nil { + return errors.New("MovePublishedODataService: nil service") + } + return b.writer.MoveUnitByID(string(svc.ID), string(svc.ContainerID)) +} func (b *MprBackend) DeleteDatabaseConnection(id model.ID) error { return b.writer.DeleteDatabaseConnection(id) } diff --git a/mdl/backend/service.go b/mdl/backend/service.go index 2d1403064..2b61be15c 100644 --- a/mdl/backend/service.go +++ b/mdl/backend/service.go @@ -25,6 +25,9 @@ type ODataBackend interface { DeleteConsumedODataService(id model.ID) error CreatePublishedODataService(svc *model.PublishedODataService) error UpdatePublishedODataService(svc *model.PublishedODataService) error + // MovePublishedODataService reparents the service document to an + // already-updated ContainerID, leaving its contents alone. + MovePublishedODataService(svc *model.PublishedODataService) error DeletePublishedODataService(id model.ID) error } diff --git a/mdl/executor/cmd_move.go b/mdl/executor/cmd_move.go index 28e2d408d..ff5034c0c 100644 --- a/mdl/executor/cmd_move.go +++ b/mdl/executor/cmd_move.go @@ -81,6 +81,14 @@ func execMove(ctx *ExecContext, s *ast.MoveStmt) error { if err := moveDatabaseConnection(ctx, s.Name, targetContainerID); err != nil { return err } + case ast.DocumentTypeJavaAction: + if err := moveJavaAction(ctx, s.Name, targetContainerID); err != nil { + return err + } + case ast.DocumentTypeODataService: + if err := movePublishedODataService(ctx, s.Name, targetContainerID); err != nil { + return err + } default: return mdlerrors.NewUnsupported("unsupported document type: " + string(s.DocumentType)) } @@ -388,3 +396,61 @@ func moveDatabaseConnection(ctx *ExecContext, name ast.QualifiedName, targetCont return mdlerrors.NewNotFound("database connection", name.String()) } + +// moveJavaAction moves a Java action to a new container. +// +// Java actions and published OData services had no MOVE doctype at all, and +// neither CREATE form takes a folder clause — so those documents could never +// leave the module root from MDL (mxcli-formula1 #32). +func moveJavaAction(ctx *ExecContext, name ast.QualifiedName, targetContainerID model.ID) error { + actions, err := ctx.Backend.ListJavaActionsFull() + if err != nil { + return mdlerrors.NewBackend("list java actions", err) + } + + h, err := getHierarchy(ctx) + if err != nil { + return mdlerrors.NewBackend("build hierarchy", err) + } + + for _, ja := range actions { + modID := h.FindModuleID(ja.ContainerID) + if h.GetModuleName(modID) == name.Module && ja.Name == name.Name { + ja.ContainerID = targetContainerID + if err := ctx.Backend.MoveJavaAction(ja); err != nil { + return mdlerrors.NewBackend("move java action", err) + } + fmt.Fprintf(ctx.Output, "Moved java action %s to new location\n", name.String()) + return nil + } + } + + return mdlerrors.NewNotFound("java action", name.String()) +} + +// movePublishedODataService moves a published OData service to a new container. +func movePublishedODataService(ctx *ExecContext, name ast.QualifiedName, targetContainerID model.ID) error { + services, err := ctx.Backend.ListPublishedODataServices() + if err != nil { + return mdlerrors.NewBackend("list published OData services", err) + } + + h, err := getHierarchy(ctx) + if err != nil { + return mdlerrors.NewBackend("build hierarchy", err) + } + + for _, svc := range services { + modID := h.FindModuleID(svc.ContainerID) + if h.GetModuleName(modID) == name.Module && svc.Name == name.Name { + svc.ContainerID = targetContainerID + if err := ctx.Backend.MovePublishedODataService(svc); err != nil { + return mdlerrors.NewBackend("move published OData service", err) + } + fmt.Fprintf(ctx.Output, "Moved odata service %s to new location\n", name.String()) + return nil + } + } + + return mdlerrors.NewNotFound("odata service", name.String()) +} diff --git a/mdl/grammar/MDLParser.g4 b/mdl/grammar/MDLParser.g4 index 7b2d2e0a9..ad7b02d17 100644 --- a/mdl/grammar/MDLParser.g4 +++ b/mdl/grammar/MDLParser.g4 @@ -363,8 +363,8 @@ renameTarget * ``` */ moveStatement - : MOVE (PAGE | MICROFLOW | SNIPPET | NANOFLOW | ENUMERATION | CONSTANT | DATABASE CONNECTION) qualifiedName TO FOLDER STRING_LITERAL (IN (qualifiedName | IDENTIFIER))? - | MOVE (PAGE | MICROFLOW | SNIPPET | NANOFLOW | ENUMERATION | CONSTANT | DATABASE CONNECTION) qualifiedName TO (qualifiedName | IDENTIFIER) + : MOVE (PAGE | MICROFLOW | SNIPPET | NANOFLOW | ENUMERATION | CONSTANT | DATABASE CONNECTION | JAVA ACTION | ODATA SERVICE) qualifiedName TO FOLDER STRING_LITERAL (IN (qualifiedName | IDENTIFIER))? + | MOVE (PAGE | MICROFLOW | SNIPPET | NANOFLOW | ENUMERATION | CONSTANT | DATABASE CONNECTION | JAVA ACTION | ODATA SERVICE) qualifiedName TO (qualifiedName | IDENTIFIER) | MOVE ENTITY qualifiedName TO (qualifiedName | IDENTIFIER) | MOVE FOLDER qualifiedName TO FOLDER STRING_LITERAL (IN (qualifiedName | IDENTIFIER))? | MOVE FOLDER qualifiedName TO (qualifiedName | IDENTIFIER) diff --git a/mdl/visitor/visitor_entity.go b/mdl/visitor/visitor_entity.go index 483f5b0d5..c4b043d6e 100644 --- a/mdl/visitor/visitor_entity.go +++ b/mdl/visitor/visitor_entity.go @@ -939,7 +939,8 @@ func (b *Builder) ExitMoveStatement(ctx *parser.MoveStatementContext) { // MOVE FOLDER is identified by having FOLDER as the first token after MOVE (no document type keyword) if len(ctx.AllFOLDER()) > 0 && ctx.PAGE() == nil && ctx.MICROFLOW() == nil && ctx.SNIPPET() == nil && ctx.NANOFLOW() == nil && ctx.ENTITY() == nil && - ctx.ENUMERATION() == nil && ctx.CONSTANT() == nil && ctx.DATABASE() == nil { + ctx.ENUMERATION() == nil && ctx.CONSTANT() == nil && ctx.DATABASE() == nil && + ctx.JAVA() == nil && ctx.ODATA() == nil { b.exitMoveFolderStatement(ctx, names) return } @@ -965,6 +966,10 @@ func (b *Builder) ExitMoveStatement(ctx *parser.MoveStatementContext) { stmt.DocumentType = ast.DocumentTypeConstant } else if ctx.DATABASE() != nil { stmt.DocumentType = ast.DocumentTypeDatabaseConnection + } else if ctx.JAVA() != nil { + stmt.DocumentType = ast.DocumentTypeJavaAction + } else if ctx.ODATA() != nil { + stmt.DocumentType = ast.DocumentTypeODataService } // Parse folder path if specified diff --git a/mdl/visitor/visitor_move_doctypes_test.go b/mdl/visitor/visitor_move_doctypes_test.go new file mode 100644 index 000000000..67fbac839 --- /dev/null +++ b/mdl/visitor/visitor_move_doctypes_test.go @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// mxcli-formula1 findings #32: MOVE accepted seven doctypes and rejected the +// rest at parse time (`no viable alternative at input 'MOVEJAVA'`). Neither +// CREATE JAVA ACTION nor CREATE ODATA SERVICE takes a folder clause either, so +// those documents could never leave the module root from MDL — five of that +// backend's documents were stuck there. +func TestMoveStatement_JavaActionAndODataService(t *testing.T) { + cases := []struct { + src string + wantType ast.DocumentType + wantName string + wantFolder string + }{ + {"move java action Mv.Helper to folder 'Support';", ast.DocumentTypeJavaAction, "Helper", "Support"}, + {"move odata service Mv.Api to folder 'Api/Published';", ast.DocumentTypeODataService, "Api", "Api/Published"}, + // The doctypes that already worked must keep working. + {"move microflow Mv.Flow to folder 'Live';", ast.DocumentTypeMicroflow, "Flow", "Live"}, + {"move database connection Mv.Db to folder 'Warehouse';", ast.DocumentTypeDatabaseConnection, "Db", "Warehouse"}, + } + + for _, tc := range cases { + t.Run(tc.src, func(t *testing.T) { + prog, errs := Build(tc.src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + var got *ast.MoveStmt + for _, s := range prog.Statements { + if m, ok := s.(*ast.MoveStmt); ok { + got = m + } + } + if got == nil { + t.Fatal("no MoveStmt produced") + } + if got.DocumentType != tc.wantType { + t.Errorf("DocumentType = %q, want %q", got.DocumentType, tc.wantType) + } + if got.Name.Name != tc.wantName { + t.Errorf("Name = %q, want %q", got.Name.Name, tc.wantName) + } + if got.Folder != tc.wantFolder { + t.Errorf("Folder = %q, want %q", got.Folder, tc.wantFolder) + } + }) + } +} + +// MOVE FOLDER is told apart from a document move by the absence of a doctype +// keyword, so adding two more keywords must not make a folder move look like a +// document move. +func TestMoveStatement_FolderStillDistinct(t *testing.T) { + prog, errs := Build("move folder Mv.Old to folder 'New';") + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + for _, s := range prog.Statements { + if m, ok := s.(*ast.MoveStmt); ok { + t.Fatalf("MOVE FOLDER produced a document MoveStmt (%q)", m.DocumentType) + } + } +} diff --git a/sdk/mpr/writer_domainmodel.go b/sdk/mpr/writer_domainmodel.go index fe755c0bc..4fac522f7 100644 --- a/sdk/mpr/writer_domainmodel.go +++ b/sdk/mpr/writer_domainmodel.go @@ -334,6 +334,13 @@ func (w *Writer) UpdateOqlQueriesForMovedEntity(oldQualifiedName, newQualifiedNa } // moveUnitByID changes a unit's ContainerID without modifying its contents. +// MoveUnitByID reparents any top-level document unit. Exported so backends can +// move doctypes that have no dedicated writer method of their own (Java actions, +// published OData services) — the containment row is all that changes. +func (w *Writer) MoveUnitByID(unitID string, newContainerID string) error { + return w.moveUnitByID(unitID, newContainerID) +} + func (w *Writer) moveUnitByID(unitID string, newContainerID string) error { unitIDBlob := uuidToBlob(unitID) containerIDBlob := uuidToBlob(newContainerID) From 50cccd9d98226052586c522d2b5fe7dd012eba3d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 11:31:26 +0000 Subject: [PATCH 22/31] docs(fix-issue): two symptom rows from the formula1 folders/naming batch Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .claude/skills/fix-issue.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index bf2180abb..0e387a560 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -408,3 +408,5 @@ extracting `OffsetExpression`/`LimitExpression`. | `create odata client` against a service behind `authentication basic` prints `Warning: could not fetch $metadata: … HTTP 401`, creates the client anyway, and the following `create external entities from …` imports nothing — from a script that reports success | The statement's `HttpUsername`/`HttpPassword`/`HEADERS` are stored for the runtime, but the design-time fetch was a bare `client.Get`. The fetch failure is only a warning, so the empty client propagates silently | `mdl/executor/cmd_odata.go` (`metadataFetchAuth`, `metadataAuthFromStmt`, `fetchODataMetadata`), `mdl/ast/ast_odata.go` + `mdl/visitor/visitor_odata.go` (`HttpUsernameIsLiteral` / `HeaderDef.ValueIsLiteral`) | **Only a literal is usable at design time.** The visitor strips a quoted literal's quotes, so `'f1api'` and `Module.ApiUser` both arrive as bare strings — the AST has to record which was written, or mxcli sends a *constant's name* as the password. Unresolved names are reported instead, which is also the honest answer: mxcli has no runtime to resolve a constant against. **A warning on a step something else silently depends on needs to say what breaks next** — the message now names the empty client and the import that will do nothing. Verified against a real basic-auth server that 401s without credentials and 403s without the custom header, so both had to arrive. Tests `cmd_odata_metadata_auth_test.go`. mxcli-formula1 #23 | | Re-running `create or modify odata service` after editing a `publish entity` block changes nothing — the served `$metadata` is identical, and only `drop odata service` + create picks the edit up | The modify branch updated the service's scalar properties and never touched `EntityTypes` / `EntitySets` | `mdl/executor/cmd_odata.go` (modify branch rebuilds published entities via `astEntityDefToModel`, and carries `AllowedModuleRoles` through) | **Replace, don't merge**: a member removed from the script has to leave the service, which merging cannot express — the script is the description of the service. **Carry through what the statement cannot express**: role grants come from a separate `grant access on odata service` and would otherwise be dropped by a modify (reported; *not* reproduced on 11.12.1 — kept as a guard, and the commit says so rather than claiming a fix). Verified: same script yields `Label as 'label'` before and `Label as 'label' (Filterable, Sortable)` after, build stays at 0 errors. Tests `cmd_odata_modify_members_test.go`. mxcli-formula1 #26 | | `create external entities from` a contract that restricts capabilities produces a project that will not build: `'Seasons' is marked Countable=False in the OData service, but True in the app`, `'latitude' is marked Filterable=False …` — one per restricted resource or property | Insert/Update/Delete restrictions were parsed; **Count/Filter/Sort were not**, so the import had nothing to honour and defaulted all three to true — on the one command whose entire job is fidelity to the contract | `mdl/types/edmx.go` (`EdmEntitySet.Countable`, `NonFilterableProperties`, `NonSortableProperties` + the three `applyCapabilityAnnotations` arms), `mdl/executor/cmd_contract.go` | An unannotated set still means countable/filterable/sortable — **silence in a contract is not a restriction**, it is OData's own default, so `nil` and `false` must stay distinguishable (`*bool`, as with the publish-side query options). The generated entity is compared against the contract at *build* time, so anything the contract can say is something the importer must be able to read. Tests `mdl/types/edmx_test.go`. mxcli-formula1 #24 | +| A contract property called `name` is generated as `Stg_Drivername` / `Circuitname` — prefixed with the remote type. A page written against the published `$metadata` then fails with `The selected attribute 'F1Live.Drivers.name' no longer exists`, and the *same* field carries a different name in every module because the remote type names differ | `attrNameForOData` disambiguates any name in `reservedEntityAttrNames`, and `name` was on that list with the comment "Mendix system-managed attribute for the object name". It is not: Mendix builds an external entity with an attribute literally named `name` | `mdl/executor/cmd_contract.go` (`reservedEntityAttrNames` loses one entry; the import now reports the renames it does make) | **Test the whole list at once, not the reported entry.** One contract with a property per listed name, prefixing disabled, then `mx check`: CE7247 "The name 'x' is a reserved word" for `id`/`owner`/`changedBy`/`changedDate`/`createdDate`/`type`/`context`, and silence for `name`. That turns "is the list wrong?" into "which rows are wrong?" for the cost of a single build, and it *earns* the seven entries that stay rather than leaving them as folklore. Two existing tests pinned the old behaviour and had to be corrected — a hand-maintained list of platform rules will accrete guesses unless each row can point at an error code. **Migration**: a re-import renames the attribute back, so references to the prefixed name must follow. Tests `cmd_contract_reserved_test.go`. mxcli-formula1 #28 | +| `MOVE JAVA ACTION …` / `MOVE ODATA SERVICE …` is a parse error (`no viable alternative at input 'MOVEJAVA'`), and neither `CREATE` form takes a folder clause — so those documents can never leave the module root from MDL | The `moveStatement` rule listed seven doctypes and nothing else; the missing ones were never unimplemented, just unlisted | `mdl/grammar/MDLParser.g4` (two alternatives), `mdl/ast/ast.go`, `mdl/visitor/visitor_entity.go` (dispatch **and** the MOVE FOLDER discriminator), `mdl/executor/cmd_move.go`, backend `MoveJavaAction` / `MovePublishedODataService`, `sdk/mpr` exports `MoveUnitByID` | Both reduce to the existing reparent primitive — a top-level document move is one containment row, so a new doctype is a list entry plus a lookup, not new machinery. **Watch the discriminator**: `MOVE FOLDER` is told apart from a document move by the *absence* of a doctype keyword, so every keyword added to the rule must also be added to that condition or a folder move starts parsing as a document move. **Verify placement by differential count, not by reading the model**: run the script with and without the MOVE lines and diff `select ContainmentName, count(*) from Unit` — three new Folders rows (a nested path creates two) and an unchanged Documents count says reparented rather than copied or dropped. Grepping blobs for names is a trap; stock modules are full of the same words. Tests `visitor_move_doctypes_test.go`, example in `18-folder-examples.mdl` — which must sit **before** that script's `drop module`, a mistake the integration gate caught and `mxcli check` did not. mxcli-formula1 #32 | From 1ec638c4a6042e2e396e4353b4ae7f023c19a9e0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 11:38:15 +0000 Subject: [PATCH 23/31] docs(microflows): fix the type-split examples that fail CE0090 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The write-microflows skill taught `case Spec` + `else` with no branch for the base entity, and described `else` as handling "objects that do not match any listed specialization". It does not. An object-type decision needs an outgoing flow for every listed type, and without the base entity the build fails [error] [CE0090] "The 'X' value should be configured for an outgoing flow." `else` IS accepted — it serializes as Microflows$NoCase — which is what made the guidance look right. It simply does not satisfy coverage, so it is redundant once every type has a branch. Matrix verified on 11.6.6 and 11.13.0: specializations + base 0 errors specializations + base + else 0 errors (else redundant) specializations + else only CE0090 The examples also omitted a return after `end split;`. Branch bodies converge on a merge that continues to the microflow's end event, so a non-void microflow needs one — otherwise mxcli check reports MDL003 and the build fails CE0067 "The 'Return value' property is required." Two shipped examples had the same defect and did not build: mdl-examples/bug-tests/365-microflow-inheritance-split.mdl and 475-inheritance-split-continuing-branch-merge.mdl — the latter's own header claimed "mx check against the resulting MPR reports 0 errors", which had not been true. Both now build clean on 11.6.6 and 11.13.0. 475's added base case is deliberately a TERMINATING branch. The scenario it pins is "exactly one non-split branch continues"; an empty, falling-through body would make two branches continue and quietly retire the regression. Verified after the edit that the post-split activity still renders outside both case bodies and that the describe→exec roundtrip is mxbuild-clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- .claude/skills/fix-issue.md | 1 + .claude/skills/mendix/write-microflows.md | 28 ++++++++++++++++--- .../365-microflow-inheritance-split.mdl | 18 ++++++++++-- ...eritance-split-continuing-branch-merge.mdl | 9 ++++++ 4 files changed, 49 insertions(+), 7 deletions(-) diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 3f830f4c0..600b10dad 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -396,3 +396,4 @@ extracting `OffsetExpression`/`LimitExpression`. | `mxcli check` reports a microflow rule (e.g. MDL048 `[id = $StringVar]`) but `mxcli exec` writes the same script silently — a script that skips `check` produces a project the build rejects | Two different validators. The exec path called `ValidateMicroflowBody` (semantic errors); the MDL0xx rule set lives in `ValidateMicroflow`, which was wired only into `cmd/mxcli/cmd_check.go` and the LSP. Same shape as #836, where a guard existed on every exec path but was never reached from validate | `mdl/executor/validate.go` (`validateMicroflowRules`, `execEnforcedMicroflowRules`) called from `mdl/executor/cmd_microflows_create.go` — the handler, mirroring `ValidateEntity` in `cmd_entities.go` (NOT `validateWithContext`, which would double-report under `check --references`) | Promote an **explicit allowlist** of rules verified against a real mxbuild, never the whole set. **Blanket promotion was tried and reverted**: it turns every error-severity rule into a write barrier, and `MDL009` ("enumeration splits require exactly one value per branch") was a **false positive** — a multi-value branch covering every enum value builds at 0 errors on 11.6.6, and the shipped `write-microflows` skill documents that very form (since retired, see the MDL056 row). It also broke an existing repo test whose fixture uses `else` on an enum split. `MDL008` by contrast IS correct (mxbuild: CE0079 per uncovered value + CE0773) — so before promoting any rule, build its construct and read the verdict. Warnings are never promoted. Tests `TestValidateMicroflowRules_ReachedFromExec` and `…_UnverifiedRulesNotPromoted` (the latter fails if the allowlist is widened carelessly). Issue #833 | | `mxcli check` errors **MDL009** "enumeration splits require exactly one value per branch" on `when Open, Pending then` — but Mendix accepts it, so check rejects valid MDL and contradicts the shipped `write-microflows` skill | The rule asserted the opposite of the platform's behaviour. Nobody had built the construct: a multi-value branch covering every value **plus `(empty)`** builds at 0 errors on 11.6.6 | `mdl/executor/validate_microflow.go` (the `*ast.EnumSplitStmt` arm; `checkEnumSplitEmptyBranch`) | Retire the assertion and replace it with what actually fails: an enum split needs an outgoing flow per condition value, so a missing branch is **CE0079**. **MDL056** checks the `(empty)` branch — universal, verified to hold even on a `not null` enum attribute, so it needs no enumeration lookup and works from the statement alone. Full value coverage (the other half of CE0079) is deliberately NOT implemented: it needs the enum's member list, i.e. resolving the split variable's type against script or project, which `ValidateMicroflow` cannot see — guessing would trade one false positive for another. **Use a NEW rule ID rather than repurposing**, so anything citing the old number still means the old, wrong thing. `MDL008` (no `else`) is correct and stays — mxbuild gives CE0079 per uncovered value **plus** CE0773 on the else flow. Fix the skill in the same change: it documented the invalid `else` form. Tests `TestValidateMicroflow_EnumSplitMultipleValuesAllowed` / `…RequiresEmptyBranch`; repro `mdl-examples/bug-tests/mdl009-enum-split-empty-branch.fail.mdl` + `-ok.mdl` | | A microflow using `split type` writes a project mxbuild cannot **load**: `KeyNotFoundException: The given key '' was not present in the dictionary` at `StreamingBsonUnitReader.ResolvePostponedProperties`. `mxcli check` ✓ and `mxcli exec` ✓; reproduced on 11.6.6 and 11.13.0 | Two gaps in the modelsdk writer, both the #791 shape. (1) `microflowObjectToGen` had no `*microflows.InheritanceSplit` case → `default: return nil`, so the split was dropped while three sequence flows kept pointing at its `$ID`. (2) `caseValueToGen` had no `InheritanceCase` case → every branch degraded to a bare `Microflows$NoCase`, losing the entity it selects on. Its value-receiver normalisation also omitted the type, so a pointer-only fix would still miss half the calls | `mdl/backend/modelsdk/microflow_write.go` (`microflowObjectToGen`, `caseValueToGen`) — mirror `sdk/mpr/writer_microflow.go` | Add both cases. **Diagnose with the #791 recipe**: `mxcli bson dump --type microflow`, collect every `$ID`, check each key ending in `Pointer` resolves (before: 27 objects / 3 dangling; after: 28 / 0). **Take field lists from the GENERATED type, not from legacy** — legacy writes `ErrorHandlingType` on the split but `initInheritanceSplit` has no such property, i.e. legacy writes a field Mendix does not define. **When adding a case-value type, update the value-receiver normalisation too.** Modelling rules confirmed on both versions while verifying: a type split needs an outgoing flow for every type INCLUDING the base (CE0090), and an `else` does NOT substitute for the base-type case. Tests `TestMicroflowRoundTrip_InheritanceSplit`, `TestCaseValueToGen_InheritanceCase{,ValueReceiver}`; repro `mdl-examples/bug-tests/split-type-dangling-pointer.mdl` | +| The `split type` docs and examples teach a shape that fails the build: `case Spec` + `else`, with no branch for the base entity → **CE0090** "The 'X' value should be configured for an outgoing flow". `mxcli check` passes, so the drift survived; `mdl-examples/bug-tests/365` and `475` both shipped it, and 475's own header claimed "mx check reports 0 errors" | `else` on an inheritance split serializes as `Microflows$NoCase` and IS accepted, so it looks like it covers the remainder — but it does not satisfy type coverage. The base entity needs its own `case` | `.claude/skills/mendix/write-microflows.md` (Type Split section) + `mdl-examples/bug-tests/365-…`, `475-…` | Cover EVERY type including the base; `else` is then redundant. Also give the split somewhere to go: branches converge on a merge continuing to the end event, so a non-void microflow needs a `return` after `end split;` (else MDL003 + **CE0067**). Matrix verified on 11.6.6 AND 11.13.0: `specs+base` 0 errors, `specs+base+else` 0 errors, `specs+else only` CE0090. **When repairing a bug-test fixture, preserve the scenario it pins** — 475 tests "exactly ONE non-split branch continues", so its added base case must TERMINATE; an empty (falling-through) body would make two branches continue and silently retire the regression. Confirmed after the edit that the post-split activity still renders outside both case bodies and the describe→exec roundtrip is mxbuild-clean. Known cosmetic artifact: DESCRIBE emits an empty `else` block that was never authored; it re-parses and builds clean | diff --git a/.claude/skills/mendix/write-microflows.md b/.claude/skills/mendix/write-microflows.md index f316c25fa..326486134 100644 --- a/.claude/skills/mendix/write-microflows.md +++ b/.claude/skills/mendix/write-microflows.md @@ -448,16 +448,36 @@ Use `split type` when a microflow branches on an object's runtime specialization Use `cast` inside a type branch to create the specialized variable used by the branch body. ```mdl +declare $IsSpecialized boolean = false; split type $Input case Sample.SpecializedInput cast $SpecificInput; - return true; -else - return false; + set $IsSpecialized = true; +case Sample.BaseInput end split; +return $IsSpecialized; ``` -`case` values are qualified entity names. The optional `else` branch handles objects that do not match any listed specialization. +`case` values are qualified entity names. + +> **Every type needs a branch — including the base entity.** An object-type +> decision gets one outgoing flow per listed type, and a type with no flow fails +> the build with **CE0090** *"The 'X' value should be configured for an outgoing +> flow."* The base entity (the split variable's own type) counts: `case +> Sample.BaseInput` above is what covers "it is not any of the specializations". +> +> **`else` does not stand in for the base-type case.** It is accepted — it +> serializes as `Microflows$NoCase` — but it does not satisfy coverage, so +> `case Spec` + `else` still fails CE0090. Once every type has a branch, `else` +> is redundant. Verified on Mendix 11.6.6 and 11.13.0. +> +> **The split needs somewhere to go afterwards.** Branch bodies converge on a +> merge that continues to the microflow's end event, so a non-void microflow +> needs a `return` after `end split;` — otherwise `mxcli check` reports MDL003 +> and the build fails **CE0067** *"The 'Return value' property is required."* +> Doing the per-branch work into a variable and returning it once (above) is the +> clearest shape; returning inside every branch also works, but still needs the +> trailing `return`. **`cast` only stores the output variable.** Studio Pro persists Microflows$CastAction with a single `VariableName` field — the source variable is implicit (the type-split's input). Use `cast $SpecificName;` to give the specialized variable its name. The two-variable form `$Output = cast $Source;` parses but `$Source` is dropped on roundtrip; prefer the single-variable form. diff --git a/mdl-examples/bug-tests/365-microflow-inheritance-split.mdl b/mdl-examples/bug-tests/365-microflow-inheritance-split.mdl index b938d49c0..879a21753 100644 --- a/mdl-examples/bug-tests/365-microflow-inheritance-split.mdl +++ b/mdl-examples/bug-tests/365-microflow-inheritance-split.mdl @@ -10,17 +10,29 @@ create persistent entity InheritanceSplitExample.SpecializedInput extends Inheri ); / +-- An object-type decision needs an outgoing flow for EVERY listed type, +-- including the base entity. This example used `case Specialized` + `else`, +-- which fails the build with CE0090 ("The 'InheritanceSplitExample.BaseInput' +-- value should be configured for an outgoing flow") — `else` serializes as +-- Microflows$NoCase and is accepted, but it does not satisfy coverage. +-- +-- The branches also converge on a merge that continues to the end event, so a +-- non-void microflow needs a `return` after `end split;` (otherwise CE0067 +-- "The 'Return value' property is required", and mxcli check reports MDL003). +-- +-- Verified with mxbuild 11.6.6 and 11.13.0: 0 errors. create microflow InheritanceSplitExample.RouteInput ( $Input: InheritanceSplitExample.BaseInput ) returns boolean begin + declare $IsSpecialized boolean = false; split type $Input case InheritanceSplitExample.SpecializedInput cast $SpecializedInput; - return true; - else - return false; + set $IsSpecialized = true; + case InheritanceSplitExample.BaseInput end split; + return $IsSpecialized; end; / diff --git a/mdl-examples/bug-tests/475-inheritance-split-continuing-branch-merge.mdl b/mdl-examples/bug-tests/475-inheritance-split-continuing-branch-merge.mdl index 9c443a5d8..64d1c7cea 100644 --- a/mdl-examples/bug-tests/475-inheritance-split-continuing-branch-merge.mdl +++ b/mdl-examples/bug-tests/475-inheritance-split-continuing-branch-merge.mdl @@ -23,6 +23,13 @@ -- Validation: -- `mxcli check` parses the script. -- `mx check` against the resulting MPR reports 0 errors. +-- +-- The base-type case (`case BugTest475.Vehicle`) is required for that: an +-- object-type decision needs an outgoing flow for every listed type, and +-- without the base entity the build fails CE0090 regardless of this bug. +-- It is deliberately a TERMINATING branch — the scenario under test is +-- "exactly ONE non-split branch continues", and giving Vehicle a falling +-- -through body would make two branches continue and lose the regression. -- Roundtrip (describe → exec → describe) preserves the structure -- byte-for-byte: the post-split log activity stays outside both case -- bodies. @@ -66,6 +73,8 @@ begin case BugTest475.Boat log info node 'BugTest475' 'Dispatching boat'; return false; + case BugTest475.Vehicle + return false; end split; log info node 'BugTest475' 'Dispatched'; return true; From f64115e6ae2f1d678f001f177498704ca3f024a5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 11:48:19 +0000 Subject: [PATCH 24/31] fix(theme): add the widget-module layer, so Data Grid 2 follows the palette MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reported from the Formula1 test build (FINDINGS §33): a themed app is on-palette everywhere, and then a few things are not. The Data Grid 2 pager caption — "1-15 of 77", the only thing telling a user where they are in the result set — measured 1.02:1 against a dark ground and was invisible. Row-select checkboxes stayed stock Mendix blue in a re-branded app, the loader flashed white on every page turn, and popovers cast light-mode shadows. One cause. _mxcli-atlas-map.scss re-points Atlas Core's custom properties, which covers the app. The theme source shipped by the *widget modules* under themesource/ styles some things with Sass variables and literals instead, and Sass resolves those at compile time, before any custom property exists — so the value is baked into theme.compiled.css and no --mxt-* can move it. The parts that did work resolve var(--gray-darker, …) through Atlas: same pager bar, two mechanisms, one of them reachable. Adds _mxcli-widgets.scss, a third shared partial imported after the theme's own, correcting each baked declaration through a token so both palettes follow. The obvious fix does not work, and it is worth writing down why. Each module's main.scss imports theme/web/custom-variables *before* its own `!default` variables, so setting `$pagination-caption-color: var(--mxt-ink-muted)` there would win and Sass would substitute the var() into every use site. But the names collide with Atlas Core's, and Atlas Core feeds them to Sass colour functions — atlas_core/web/_variables.scss:20 computes mix($brand-primary, #e7e7e9, 10%), and handing mix() a var() is a compile error, so the app stops building. And the worst offenders are not behind a variable at all: _three-state-checkbox.scss writes #264ae5 and rgba(#264ae5, 0.4) directly. Every selector was read out of a compiled theme.compiled.css rather than from the SCSS sources. That distinction halved the work: the sources are full of `var(--token, #fallback)` declarations that already resolve correctly, and of the 46 declarations mentioning the stock blue, 24 were harmless fallbacks. The report's own list was assembled from the sources and is correspondingly longer. Verified in a browser, both variants, both light-first and dark-first themes: pager caption 1.02:1 -> 6.99:1 on console dark and 6.39:1 light (the exact rgb(154,166,180) / rgb(85,96,110) the report measured for its own fix), 6.78 / 5.93 on signal. Checked-checkbox fill and loader background resolve to --mxt-brand and --mxt-surface in the compiled output, with mxcli's declaration last. A test asserts the layer reintroduces no literal colour, and the shared-partial drift guard now covers both shared files rather than only the Atlas map. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JEZmExJUvn2nWTWE9mrd4i --- .claude/skills/fix-issue.md | 2 + .claude/skills/mendix/theme-styling.md | 37 +++++ .../files/theme/web/_mxcli-widgets.scss | 148 ++++++++++++++++++ .../assets/console/files/theme/web/main.scss | 1 + cmd/mxcli/theme/assets/console/theme.json | 5 + .../files/theme/web/_mxcli-widgets.scss | 148 ++++++++++++++++++ .../assets/ledger/files/theme/web/main.scss | 1 + cmd/mxcli/theme/assets/ledger/theme.json | 5 + .../files/theme/web/_mxcli-widgets.scss | 148 ++++++++++++++++++ .../assets/signal/files/theme/web/main.scss | 1 + cmd/mxcli/theme/assets/signal/theme.json | 5 + cmd/mxcli/theme/theme_test.go | 53 +++++-- docs-site/src/tools/theme.md | 10 +- 13 files changed, 552 insertions(+), 12 deletions(-) create mode 100644 cmd/mxcli/theme/assets/console/files/theme/web/_mxcli-widgets.scss create mode 100644 cmd/mxcli/theme/assets/ledger/files/theme/web/_mxcli-widgets.scss create mode 100644 cmd/mxcli/theme/assets/signal/files/theme/web/_mxcli-widgets.scss diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 5a984b569..77d575f57 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -356,6 +356,8 @@ cases for these three BSON types — they fell to `default: return nil`. | An unquoted negative number in an XPath constraint fails to parse: `where [Amount > -7]` → `Parse error: extraneous input '7' expecting {',', ')'}`. Reported as "negative numeric literals truncate (`-7` becomes `-`)" | `xpathWord` — the name-part rule inside XPath — is a **negated token set** that did not exclude `MINUS`, so the sign was consumed as a name word and the digits were left stranded (hence the truncation appearance). The lexer deliberately keeps `-` out of `NUMBER_LITERAL` (a leading sign there mis-tokenises `$x -2`), leaving negation to the parser; the general grammar has `unaryExpression` for this and the XPath grammar simply never got the equivalent | `mdl/grammar/domains/MDLPage.g4` (`xpathValueExpr` gains `MINUS xpathValueExpr`; `MINUS` added to the `xpathWord` exclusion set), `mdl/visitor/visitor_xpath.go` (`buildXPathValueExpr`), `mdl/visitor/visitor_page_v3.go` (`xpathExprToString` emits `-7`, not `- 7`) | **The grammar fix alone is worse than the bug.** With the parser accepting `-7` but the XPath AST builder having no case for the new alternative, the constraint parses and silently serializes to `[Amount > ]` — a dropped operand instead of a loud parse error. Caught only because the visitor has a round-trip helper; the microflow write path uses `GetText()` and looked fine. **When adding a grammar alternative, check every consumer of that rule, not just the one your repro exercises.** **Scope correction**: the finding's own example (`addDays([%CurrentDateTime%], -7)`) still fails — `addDays` is a *microflow expression* function, not an XPath one, and it fails `CE0161` with a POSITIVE argument too, so the sign was never its problem. Repro `mdl-examples/bug-tests/it-18-xpath-negative-literal.mdl`; A/B on Mendix 11.12.1: pre-fix the script does not parse, fixed binary writes it and `mx check` reports 0 errors. issuetracker #18 | | Text painted by an **Atlas topbar widget is invisible in a dark theme** — the language selector measures ~1.13:1 contrast, glyph pixels spanning 4 luminance values out of 255. A theme override exists and *names the right element*, so it looks handled | Two separate mistakes stacked. (1) **Specificity**: Atlas's own rule is `.navbar-brand .widget-language-selector .current-language-text` at (0,3,0); a bare `.current-language-text` at (0,1,0) never wins, and only appears to on layouts that do not nest the selector under `.navbar-brand`. (2) **Wrong value**: `color: inherit` inherits *body ink*, which is dark, while the rail is dark in both palettes — so even at the winning specificity it measures 1.00:1. Atlas paints from `--bg-color-secondary` with a `#fff` fallback because it assumes a dark rail | `cmd/mxcli/theme/assets/*/files/theme/web/_mxcli-atlas-map.scss` (the "Atlas fixups" block) | Re-declare **Atlas's own selector shape** so the weights match and source order decides, and resolve the colour through the rail token (`var(--mxt-rail-ink-active, var(--mxt-rail-ink))`) rather than `inherit`. List the bare and the `.navbar-brand`-nested selectors together — each is matched at its own specificity, so one rule covers both layouts. **Generalisable — the shape to look for**: a guard that names the right element is not evidence it applies. Read the *winning* declaration (`CSS.getMatchedStylesForNode` in DevTools, or the computed value) instead of the one you wrote. **Measure contrast, not colour**: reading `getComputedStyle(el).color` once and seeing a plausible value proves nothing — compute the WCAG ratio against the first non-transparent ancestor background, which is what turns "looks fine" into 1.13 vs 19.47. Reported from the RssReader test build; tests in `cmd/mxcli/theme/theme_test.go`, verified in a browser at 17.79:1 light / 19.47:1 dark | + +| A themed app is on-palette everywhere except a few widget details — the **Data Grid 2 pager caption is invisible** (1.02:1 on a dark ground), row-select checkboxes stay stock Mendix blue, popovers cast light-mode shadows. Re-pointing tokens changes nothing, and the same widget's other parts (the pager *buttons*) are fine | The theme source shipped by the **widget modules** (`themesource/datawidgets`, `atlas_web_content`) styles some things with Sass variables and literals — `datawidgets/web/variables.scss:18` is `$pagination-caption-color: #0a1325`. Sass resolves those at compile time, before any custom property exists, so the value is baked into `theme.compiled.css` and no `--mxt-*` can reach it. The parts that *do* work resolve `var(--gray-darker, …)` through Atlas: same bar, two mechanisms | `cmd/mxcli/theme/assets/*/files/theme/web/_mxcli-widgets.scss` (the shared widget layer, imported after the theme partial) | Add a CSS rule per baked declaration, resolving through a token so both palettes follow. **The obvious fix does not work**: each module's `main.scss` imports `theme/web/custom-variables` before its own `!default` vars, so `$pagination-caption-color: var(--mxt-ink-muted)` there *would* win — but (1) the names collide with Atlas Core's, which feeds them to Sass colour functions (`atlas_core/web/_variables.scss:20` computes `mix($brand-primary, #e7e7e9, 10%)`; handing `mix()` a `var()` is a compile error) and (2) the worst offenders are not behind a variable at all — `_three-state-checkbox.scss` writes `#264ae5` and `rgba(#264ae5, 0.4)` directly. **Generalisable — the shape to look for**: read the **compiled CSS, not the SCSS**, when deciding what to override. The sources are full of `var(--token, #fallback)` declarations that already resolve correctly; in one measured app `#264ae5` appeared in 46 declarations and **24 were harmless fallbacks**, so grepping the source would have produced twice the rules for no benefit. Reported from the Formula1 test build (§33); verified in a browser: pager caption 1.02:1 → 6.99:1 console dark, 6.39:1 light, 6.78/5.93 on signal | | A loop's variable used after `end loop;` passes `mxcli check`, then `mx check` fails `[error] [CE0108] "Variable 'item' is defined but not in scope at this location."` at the referencing activity. Applies to the **iterator** and to anything the body introduces (a `retrieve`, a `$X = create …`, a call output) | Nothing tracked loop-variable *visibility*. MDL052 already covered the sibling rule — names are unique across the whole microflow (CE0111) — and the wording of that rule ("scoped to the WHOLE microflow") reads as if the variable is readable flow-wide. Uniqueness and visibility are different: the name is reserved everywhere, readable only inside the loop body | `mdl/executor/validate_microflow_loop_scope.go` (new `MDL053`, wired from `microflowValidator.validate`), skill `.claude/skills/mendix/write-microflows.md` | Map each loop-scoped name to the loop whose **own** body introduces it (nested loops keep their own names), then walk the flow with the set of enclosing loops and flag any reference from outside the owner. **A name claimed by two loops is dropped, not reported** — that is the MDL052/CE0111 case, and without the guard the MDL052 negative example started failing for the wrong reason: the first loop's own use of `$R` was blamed on the second loop's claim. **Generalisable**: a rule keyed by variable *name* needs an ambiguity escape hatch whenever another rule exists precisely because names can collide. Both flavours verified against mxbuild 11.12.1 (2 × CE0108 in one probe). Repro `mdl-examples/bug-tests/loop-variable-out-of-scope.fail.mdl`; tests `mdl/executor/validate_microflow_loop_scope_test.go`. Found while working the sudoku findings, but **not** one of them — the numbered finding it was filed under is an app bug in that project, not an mxcli defect | | `mxcli oql` silently omits a whole column: a `select A, B …` renders only `A`, and the JSON output has no `B` key at all — no error, no empty column. Reproduces whenever the value of `B` is **null in the first row** | The runtime omits a null-valued column from a row's JSON object entirely, and `parseOQLFeedback` took the column list from `extractColumnOrder(rows[0])` — one row, chosen for its key *order*. Every later row was then projected onto that short list, so the column vanished from the result rather than showing NULLs | `cmd/mxcli/docker/oql.go` (`parseOQLFeedback`, `mergeColumnOrder`, `hasOnlyKnownKeys`) | Union the keys of **all** rows, inserting each new key directly after the last key already known rather than appending — merging `[A, C]` with `[A, B, C]` must give `[A, B, C]`, not `[A, C, B]`, or a column that is null early in the result set jumps to the end of the table. The per-row re-scan is skipped when a row carries no unseen key, so the uniform case still costs one length check. **Generalisable**: any "take the shape from the first element" over a sparse encoding is a silent-wrong-answer bug, not a formatting bug — the output looks complete. Tests `TestParseOQLFeedback_ColumnUnionAcrossRows`, `TestMergeColumnOrder`; proven by stubbing the union back to first-row-only and watching the reported symptom return. sudoku #39, first half | | `mxcli test` cannot run at all in a container without a Docker daemon — parsing, runner generation, model injection and the **entire mxbuild build** succeed natively, then `docker up` fails with "failed to connect to the docker API at unix:///var/run/docker.sock". So microflow tests are unavailable in exactly the environment mxcli targets (Claude Code web containers ship `/usr/bin/docker` with no daemon) | Only one step of the run was containerised — start the runtime and read its log — but it was wired directly to `docker compose`, with no seam for another way to run the app. `run --local` had all three pieces already (boot a standalone runtime, tee its log, restore) | `cmd/mxcli/docker/localapp.go` (new `StartLocalApp`), `cmd/mxcli/testrunner/runner_local.go` (new), `cmd/mxcli/testrunner/runner.go` (docker path extracted to `runDockerAndCapture`), `cmd/mxcli/cmd_test_run.go` + `main.go` (`--local`) | Give the run a seam — `runLocalAndCapture` / `runDockerAndCapture` — so both modes share parse/inject/parse-results/cleanup and differ only in how the app is started. **Two traps, both found by running it rather than reasoning about it.** (a) The runner reports via an **after-startup** microflow, so its LOG output happens DURING the start action, before the runtime's log subscriber attaches; registering the subscriber early is not possible (the runtime answers `LoggingException` pre-start). What actually carries the output is the JVM console tee, live from spawn — verified by A/B running with the early attach removed. (b) A **failing** test makes the runner return false, which makes the after-startup action fail, which makes `start` return an error — the first version reported that as a broken run and dumped a stack trace instead of the test report. A failed boot whose log shows a verdict is a normal outcome. Local runs use their own ports (8081/8091) and a `_test` database so a `run --local` dev loop can keep serving. Verified end-to-end in a daemon-less container: 2 passed; then 1 passed / 1 failed with exit 1, project restored. sudoku #41 | diff --git a/.claude/skills/mendix/theme-styling.md b/.claude/skills/mendix/theme-styling.md index 218fe8aff..51c778282 100644 --- a/.claude/skills/mendix/theme-styling.md +++ b/.claude/skills/mendix/theme-styling.md @@ -135,6 +135,43 @@ Keep the rail dark in both variants, or force `color: inherit` on those widgets. For a working implementation of all of the above, read the generated `theme/web/_mxcli-atlas-map.scss` in any themed project. +### Tokens stop at Atlas Core — the widget modules bake their colours + +Re-pointing Atlas's custom properties covers the app, and then a few things stay +stubbornly off-palette: the Data Grid 2 pager caption, row-select checkboxes, +popover shadows. One cause: the theme source shipped by the **widget modules** +(`themesource/datawidgets`, `atlas_web_content`) styles some things with Sass +variables and literals. Sass resolves those at compile time, before any custom +property exists, so the value is baked into `theme.compiled.css` and **no token +can move it**. Only a later CSS rule can. + +The worst case is `datawidgets/web/variables.scss:18`, +`$pagination-caption-color: #0a1325` — the "1–15 of 77" caption, which measured +**1.02:1** on a dark ground. The pager *buttons* beside it were fine, because +they resolve `var(--gray-darker, …)` through Atlas. Same bar, two mechanisms. + +**The obvious fix does not work.** Each module's `main.scss` imports +`theme/web/custom-variables` *before* its own `!default` variables, so setting +`$pagination-caption-color: var(--my-muted)` there would win and Sass would +substitute the `var()` into every use site. Tempting, and wrong here: + +1. The names collide with Atlas Core's, and Atlas Core feeds them to Sass colour + functions — `atlas_core/web/_variables.scss:20` computes + `mix($brand-primary, #e7e7e9, 10%)`. Handing `mix()` a `var()` is a compile + error, so the app stops building. +2. The worst offenders are not behind a variable at all: + `_three-state-checkbox.scss` writes `#264ae5` and `rgba(#264ae5, 0.4)` + directly, so overriding `$brand-primary` would not reach them. + +So it is a rule set, in a partial imported after the theme's own — see the +generated `theme/web/_mxcli-widgets.scss`. + +**Read the compiled CSS, not the SCSS, when building one.** The sources are full +of `var(--token, #fallback)` declarations that already resolve correctly; only +the bare literals are a problem. In one measured app the stock blue `#264ae5` +appeared in 46 declarations — **24 of them harmless fallbacks**. Grepping the +source would have produced twice the rules for no benefit. + ## CSS Hot-Reload Workflow For theme/styling changes during Docker development: diff --git a/cmd/mxcli/theme/assets/console/files/theme/web/_mxcli-widgets.scss b/cmd/mxcli/theme/assets/console/files/theme/web/_mxcli-widgets.scss new file mode 100644 index 000000000..afd531717 --- /dev/null +++ b/cmd/mxcli/theme/assets/console/files/theme/web/_mxcli-widgets.scss @@ -0,0 +1,148 @@ +// The widget-module layer — shared by every mxcli theme, identical in each. +// +// _mxcli-atlas-map.scss re-points Atlas Core's CSS custom properties at the +// palette, and that covers the app: ground, surfaces, ink, brand, type, cards, +// buttons and form controls all follow. What it cannot reach is the theme +// source shipped by the *widget modules* under themesource/, which styles a +// number of things with Sass variables and literals. Sass resolves those at +// compile time, before any custom property exists, so the value is baked into +// theme.compiled.css and no --mxt-* can move it. Only a later CSS rule can. +// +// This file is that rule set. It is imported after the theme partial, so it +// wins on source order without !important, and every declaration resolves +// through a token so both palettes follow. +// +// THE OBVIOUS FIX DOES NOT WORK. Each module's main.scss imports +// theme/web/custom-variables *before* its own `!default` variables, so setting +// e.g. `$pagination-caption-color: var(--mxt-ink-muted)` there would win, and +// Sass would substitute the var() reference into every use site. It is a real +// technique — but not here, for two reasons: +// +// 1. The names collide with Atlas Core's own, and Atlas Core feeds them to +// Sass colour functions: atlas_core/web/_variables.scss:20 computes +// `mix($brand-primary, #e7e7e9, 10%)`. Handing mix() a var() reference is +// a compile error, so the app stops building. +// 2. The worst offenders are not behind a variable at all. +// _three-state-checkbox.scss writes #264ae5 and rgba(#264ae5, 0.4) +// directly, so overriding $brand-primary would not reach them anyway. +// +// Every selector below was read out of a compiled theme.compiled.css, not from +// the SCSS sources and not guessed — the sources contain many +// `var(--token, #fallback)` declarations that already resolve correctly, and +// only the bare literals are actually a problem. Each rule names the source it +// corrects. + +// --------------------------------------------------------------------------- +// Data Grid 2 (themesource/datawidgets) +// --------------------------------------------------------------------------- + +// variables.scss:18 — $pagination-caption-color: #0a1325, a Sass variable, so +// unreachable. This is the caption that reads "1–15 of 77": the only thing +// telling a user where they are in the result set. Against a dark ground it +// measured 1.02:1 and was simply invisible. The pager *buttons* either side +// were always fine, because they resolve var(--gray-darker, …) through Atlas — +// same bar, two mechanisms, one of them reachable. +.pagination-bar { + color: var(--mxt-ink-muted); +} + +// _datagrid.scss:442 — background-color: rgba(255, 255, 255, 1). A full-width +// panel that replaces the rows while a page loads, so every page turn flashed +// white on a dark app. +.widget-datagrid-loader-container { + background-color: var(--mxt-surface); +} + +// _datagrid.scss:212-214 and 370-372 — box-shadow: 0 2px 20px 1px +// rgba(32, 43, 54, 0.08). A light-mode drop shadow under the column selector, +// which on a dark ground reads as a smudge rather than elevation. Themes that +// set --mxt-shadow: none get a hairline instead, which is what carries the +// separation for them. +.table .column-selector .column-selector-content .column-selectors, +.column-selectors { + border-color: var(--mxt-line); + box-shadow: var(--mxt-shadow); +} + +// _three-state-checkbox.scss — the row-select boxes down the left of every +// grid. Nine baked literals: #e7e7e9 borders and #f8f8f8 disabled fills that +// vanish on a dark surface, #ffffff checkmarks, and #264ae5 for the checked and +// indeterminate states — the one place stock Mendix blue still showed through a +// re-branded app. +input[type="checkbox"].three-state-checkbox { + &:before { + border-color: var(--mxt-line); + } + + &:not(:indeterminate):after, + &:indeterminate:after { + border-color: var(--mxt-brand-ink); + } + + &:not(:disabled):not(:checked):hover:after { + border-color: var(--mxt-line); + } + + &:indeterminate:before, + &:checked:before { + border-color: var(--mxt-brand); + background-color: var(--mxt-brand); + } + + &:disabled:before, + &:disabled:after { + border-color: var(--mxt-surface-alt); + background-color: var(--mxt-surface-alt); + } + + &:indeterminate:disabled:before, + &:checked:disabled:before { + border-color: transparent; + background-color: var(--mxt-surface-alt); + } +} + +// _datagrid-dropdown-filter.scss:246 — --wdf-input-placeholder-color: +// rgb(117, 117, 117), a mid grey chosen for a white field. +:where(.widget-dropdown-filter.variant-select[data-empty]) { + --wdf-input-placeholder-color: var(--mxt-ink-faint); +} + +// _datagrid-dropdown-filter.scss:267 — color: #000 on the selected tag, black +// text on a chip that is no longer light. +:where(.widget-dropdown-filter.variant-tag-picker) .widget-dropdown-filter-selected-item, +:where(.widget-dropdown-filter.variant-tag-picker-text) .widget-dropdown-filter-selected-item { + color: var(--mxt-ink); +} + +// _export-alert.scss — the cancel button's focus ring, the one part of the +// export UI that is a bare literal rather than a var() fallback. +.widget-datagrid-export-alert-cancel.btn:focus-visible { + outline-color: var(--mxt-brand); +} + +// --------------------------------------------------------------------------- +// Other modules that bake the stock brand colour +// +// These render only if the module is present — the feedback widget ships in a +// blank app and draws a floating button over every page. Harmless when absent: +// an unmatched selector costs nothing. +// --------------------------------------------------------------------------- +.mxfeedback-start-button--side { + background-color: var(--mxt-brand); +} + +.mxfeedback-tool-button--active, +.mxfeedback-screenshot-preview__delete-button { + color: var(--mxt-brand); +} + +.mxfeedback-tool-button__color:focus, +.mxfeedback-tool-button__thickness:focus { + outline-color: var(--mxt-brand); +} + +.take-picture-save-button { + background-color: var(--mxt-brand); + color: var(--mxt-brand-ink); +} diff --git a/cmd/mxcli/theme/assets/console/files/theme/web/main.scss b/cmd/mxcli/theme/assets/console/files/theme/web/main.scss index fa97920ad..ee5c07a0a 100644 --- a/cmd/mxcli/theme/assets/console/files/theme/web/main.scss +++ b/cmd/mxcli/theme/assets/console/files/theme/web/main.scss @@ -7,3 +7,4 @@ $mxcli-theme-variant: {{VARIANT}}; @import "mxcli-atlas-map"; @import "mxcli-console"; +@import "mxcli-widgets"; diff --git a/cmd/mxcli/theme/assets/console/theme.json b/cmd/mxcli/theme/assets/console/theme.json index 1d151bde0..ce9cbc3af 100644 --- a/cmd/mxcli/theme/assets/console/theme.json +++ b/cmd/mxcli/theme/assets/console/theme.json @@ -29,6 +29,11 @@ "mode": "block", "purpose": "Layer 2 \u2014 light palette, variant blocks, fonts, recipes" }, + { + "path": "theme/web/_mxcli-widgets.scss", + "mode": "block", + "purpose": "the widget-module layer \u2014 rules for colours Sass bakes before any token exists" + }, { "path": "theme/web/main.scss", "mode": "block", diff --git a/cmd/mxcli/theme/assets/ledger/files/theme/web/_mxcli-widgets.scss b/cmd/mxcli/theme/assets/ledger/files/theme/web/_mxcli-widgets.scss new file mode 100644 index 000000000..afd531717 --- /dev/null +++ b/cmd/mxcli/theme/assets/ledger/files/theme/web/_mxcli-widgets.scss @@ -0,0 +1,148 @@ +// The widget-module layer — shared by every mxcli theme, identical in each. +// +// _mxcli-atlas-map.scss re-points Atlas Core's CSS custom properties at the +// palette, and that covers the app: ground, surfaces, ink, brand, type, cards, +// buttons and form controls all follow. What it cannot reach is the theme +// source shipped by the *widget modules* under themesource/, which styles a +// number of things with Sass variables and literals. Sass resolves those at +// compile time, before any custom property exists, so the value is baked into +// theme.compiled.css and no --mxt-* can move it. Only a later CSS rule can. +// +// This file is that rule set. It is imported after the theme partial, so it +// wins on source order without !important, and every declaration resolves +// through a token so both palettes follow. +// +// THE OBVIOUS FIX DOES NOT WORK. Each module's main.scss imports +// theme/web/custom-variables *before* its own `!default` variables, so setting +// e.g. `$pagination-caption-color: var(--mxt-ink-muted)` there would win, and +// Sass would substitute the var() reference into every use site. It is a real +// technique — but not here, for two reasons: +// +// 1. The names collide with Atlas Core's own, and Atlas Core feeds them to +// Sass colour functions: atlas_core/web/_variables.scss:20 computes +// `mix($brand-primary, #e7e7e9, 10%)`. Handing mix() a var() reference is +// a compile error, so the app stops building. +// 2. The worst offenders are not behind a variable at all. +// _three-state-checkbox.scss writes #264ae5 and rgba(#264ae5, 0.4) +// directly, so overriding $brand-primary would not reach them anyway. +// +// Every selector below was read out of a compiled theme.compiled.css, not from +// the SCSS sources and not guessed — the sources contain many +// `var(--token, #fallback)` declarations that already resolve correctly, and +// only the bare literals are actually a problem. Each rule names the source it +// corrects. + +// --------------------------------------------------------------------------- +// Data Grid 2 (themesource/datawidgets) +// --------------------------------------------------------------------------- + +// variables.scss:18 — $pagination-caption-color: #0a1325, a Sass variable, so +// unreachable. This is the caption that reads "1–15 of 77": the only thing +// telling a user where they are in the result set. Against a dark ground it +// measured 1.02:1 and was simply invisible. The pager *buttons* either side +// were always fine, because they resolve var(--gray-darker, …) through Atlas — +// same bar, two mechanisms, one of them reachable. +.pagination-bar { + color: var(--mxt-ink-muted); +} + +// _datagrid.scss:442 — background-color: rgba(255, 255, 255, 1). A full-width +// panel that replaces the rows while a page loads, so every page turn flashed +// white on a dark app. +.widget-datagrid-loader-container { + background-color: var(--mxt-surface); +} + +// _datagrid.scss:212-214 and 370-372 — box-shadow: 0 2px 20px 1px +// rgba(32, 43, 54, 0.08). A light-mode drop shadow under the column selector, +// which on a dark ground reads as a smudge rather than elevation. Themes that +// set --mxt-shadow: none get a hairline instead, which is what carries the +// separation for them. +.table .column-selector .column-selector-content .column-selectors, +.column-selectors { + border-color: var(--mxt-line); + box-shadow: var(--mxt-shadow); +} + +// _three-state-checkbox.scss — the row-select boxes down the left of every +// grid. Nine baked literals: #e7e7e9 borders and #f8f8f8 disabled fills that +// vanish on a dark surface, #ffffff checkmarks, and #264ae5 for the checked and +// indeterminate states — the one place stock Mendix blue still showed through a +// re-branded app. +input[type="checkbox"].three-state-checkbox { + &:before { + border-color: var(--mxt-line); + } + + &:not(:indeterminate):after, + &:indeterminate:after { + border-color: var(--mxt-brand-ink); + } + + &:not(:disabled):not(:checked):hover:after { + border-color: var(--mxt-line); + } + + &:indeterminate:before, + &:checked:before { + border-color: var(--mxt-brand); + background-color: var(--mxt-brand); + } + + &:disabled:before, + &:disabled:after { + border-color: var(--mxt-surface-alt); + background-color: var(--mxt-surface-alt); + } + + &:indeterminate:disabled:before, + &:checked:disabled:before { + border-color: transparent; + background-color: var(--mxt-surface-alt); + } +} + +// _datagrid-dropdown-filter.scss:246 — --wdf-input-placeholder-color: +// rgb(117, 117, 117), a mid grey chosen for a white field. +:where(.widget-dropdown-filter.variant-select[data-empty]) { + --wdf-input-placeholder-color: var(--mxt-ink-faint); +} + +// _datagrid-dropdown-filter.scss:267 — color: #000 on the selected tag, black +// text on a chip that is no longer light. +:where(.widget-dropdown-filter.variant-tag-picker) .widget-dropdown-filter-selected-item, +:where(.widget-dropdown-filter.variant-tag-picker-text) .widget-dropdown-filter-selected-item { + color: var(--mxt-ink); +} + +// _export-alert.scss — the cancel button's focus ring, the one part of the +// export UI that is a bare literal rather than a var() fallback. +.widget-datagrid-export-alert-cancel.btn:focus-visible { + outline-color: var(--mxt-brand); +} + +// --------------------------------------------------------------------------- +// Other modules that bake the stock brand colour +// +// These render only if the module is present — the feedback widget ships in a +// blank app and draws a floating button over every page. Harmless when absent: +// an unmatched selector costs nothing. +// --------------------------------------------------------------------------- +.mxfeedback-start-button--side { + background-color: var(--mxt-brand); +} + +.mxfeedback-tool-button--active, +.mxfeedback-screenshot-preview__delete-button { + color: var(--mxt-brand); +} + +.mxfeedback-tool-button__color:focus, +.mxfeedback-tool-button__thickness:focus { + outline-color: var(--mxt-brand); +} + +.take-picture-save-button { + background-color: var(--mxt-brand); + color: var(--mxt-brand-ink); +} diff --git a/cmd/mxcli/theme/assets/ledger/files/theme/web/main.scss b/cmd/mxcli/theme/assets/ledger/files/theme/web/main.scss index 260016738..0ae379c9e 100644 --- a/cmd/mxcli/theme/assets/ledger/files/theme/web/main.scss +++ b/cmd/mxcli/theme/assets/ledger/files/theme/web/main.scss @@ -7,3 +7,4 @@ $mxcli-theme-variant: {{VARIANT}}; @import "mxcli-atlas-map"; @import "mxcli-ledger"; +@import "mxcli-widgets"; diff --git a/cmd/mxcli/theme/assets/ledger/theme.json b/cmd/mxcli/theme/assets/ledger/theme.json index 988837b18..b9c06c4ee 100644 --- a/cmd/mxcli/theme/assets/ledger/theme.json +++ b/cmd/mxcli/theme/assets/ledger/theme.json @@ -29,6 +29,11 @@ "mode": "block", "purpose": "Layer 2 \u2014 dark palette, variant blocks, fonts, recipes" }, + { + "path": "theme/web/_mxcli-widgets.scss", + "mode": "block", + "purpose": "the widget-module layer \u2014 rules for colours Sass bakes before any token exists" + }, { "path": "theme/web/main.scss", "mode": "block", diff --git a/cmd/mxcli/theme/assets/signal/files/theme/web/_mxcli-widgets.scss b/cmd/mxcli/theme/assets/signal/files/theme/web/_mxcli-widgets.scss new file mode 100644 index 000000000..afd531717 --- /dev/null +++ b/cmd/mxcli/theme/assets/signal/files/theme/web/_mxcli-widgets.scss @@ -0,0 +1,148 @@ +// The widget-module layer — shared by every mxcli theme, identical in each. +// +// _mxcli-atlas-map.scss re-points Atlas Core's CSS custom properties at the +// palette, and that covers the app: ground, surfaces, ink, brand, type, cards, +// buttons and form controls all follow. What it cannot reach is the theme +// source shipped by the *widget modules* under themesource/, which styles a +// number of things with Sass variables and literals. Sass resolves those at +// compile time, before any custom property exists, so the value is baked into +// theme.compiled.css and no --mxt-* can move it. Only a later CSS rule can. +// +// This file is that rule set. It is imported after the theme partial, so it +// wins on source order without !important, and every declaration resolves +// through a token so both palettes follow. +// +// THE OBVIOUS FIX DOES NOT WORK. Each module's main.scss imports +// theme/web/custom-variables *before* its own `!default` variables, so setting +// e.g. `$pagination-caption-color: var(--mxt-ink-muted)` there would win, and +// Sass would substitute the var() reference into every use site. It is a real +// technique — but not here, for two reasons: +// +// 1. The names collide with Atlas Core's own, and Atlas Core feeds them to +// Sass colour functions: atlas_core/web/_variables.scss:20 computes +// `mix($brand-primary, #e7e7e9, 10%)`. Handing mix() a var() reference is +// a compile error, so the app stops building. +// 2. The worst offenders are not behind a variable at all. +// _three-state-checkbox.scss writes #264ae5 and rgba(#264ae5, 0.4) +// directly, so overriding $brand-primary would not reach them anyway. +// +// Every selector below was read out of a compiled theme.compiled.css, not from +// the SCSS sources and not guessed — the sources contain many +// `var(--token, #fallback)` declarations that already resolve correctly, and +// only the bare literals are actually a problem. Each rule names the source it +// corrects. + +// --------------------------------------------------------------------------- +// Data Grid 2 (themesource/datawidgets) +// --------------------------------------------------------------------------- + +// variables.scss:18 — $pagination-caption-color: #0a1325, a Sass variable, so +// unreachable. This is the caption that reads "1–15 of 77": the only thing +// telling a user where they are in the result set. Against a dark ground it +// measured 1.02:1 and was simply invisible. The pager *buttons* either side +// were always fine, because they resolve var(--gray-darker, …) through Atlas — +// same bar, two mechanisms, one of them reachable. +.pagination-bar { + color: var(--mxt-ink-muted); +} + +// _datagrid.scss:442 — background-color: rgba(255, 255, 255, 1). A full-width +// panel that replaces the rows while a page loads, so every page turn flashed +// white on a dark app. +.widget-datagrid-loader-container { + background-color: var(--mxt-surface); +} + +// _datagrid.scss:212-214 and 370-372 — box-shadow: 0 2px 20px 1px +// rgba(32, 43, 54, 0.08). A light-mode drop shadow under the column selector, +// which on a dark ground reads as a smudge rather than elevation. Themes that +// set --mxt-shadow: none get a hairline instead, which is what carries the +// separation for them. +.table .column-selector .column-selector-content .column-selectors, +.column-selectors { + border-color: var(--mxt-line); + box-shadow: var(--mxt-shadow); +} + +// _three-state-checkbox.scss — the row-select boxes down the left of every +// grid. Nine baked literals: #e7e7e9 borders and #f8f8f8 disabled fills that +// vanish on a dark surface, #ffffff checkmarks, and #264ae5 for the checked and +// indeterminate states — the one place stock Mendix blue still showed through a +// re-branded app. +input[type="checkbox"].three-state-checkbox { + &:before { + border-color: var(--mxt-line); + } + + &:not(:indeterminate):after, + &:indeterminate:after { + border-color: var(--mxt-brand-ink); + } + + &:not(:disabled):not(:checked):hover:after { + border-color: var(--mxt-line); + } + + &:indeterminate:before, + &:checked:before { + border-color: var(--mxt-brand); + background-color: var(--mxt-brand); + } + + &:disabled:before, + &:disabled:after { + border-color: var(--mxt-surface-alt); + background-color: var(--mxt-surface-alt); + } + + &:indeterminate:disabled:before, + &:checked:disabled:before { + border-color: transparent; + background-color: var(--mxt-surface-alt); + } +} + +// _datagrid-dropdown-filter.scss:246 — --wdf-input-placeholder-color: +// rgb(117, 117, 117), a mid grey chosen for a white field. +:where(.widget-dropdown-filter.variant-select[data-empty]) { + --wdf-input-placeholder-color: var(--mxt-ink-faint); +} + +// _datagrid-dropdown-filter.scss:267 — color: #000 on the selected tag, black +// text on a chip that is no longer light. +:where(.widget-dropdown-filter.variant-tag-picker) .widget-dropdown-filter-selected-item, +:where(.widget-dropdown-filter.variant-tag-picker-text) .widget-dropdown-filter-selected-item { + color: var(--mxt-ink); +} + +// _export-alert.scss — the cancel button's focus ring, the one part of the +// export UI that is a bare literal rather than a var() fallback. +.widget-datagrid-export-alert-cancel.btn:focus-visible { + outline-color: var(--mxt-brand); +} + +// --------------------------------------------------------------------------- +// Other modules that bake the stock brand colour +// +// These render only if the module is present — the feedback widget ships in a +// blank app and draws a floating button over every page. Harmless when absent: +// an unmatched selector costs nothing. +// --------------------------------------------------------------------------- +.mxfeedback-start-button--side { + background-color: var(--mxt-brand); +} + +.mxfeedback-tool-button--active, +.mxfeedback-screenshot-preview__delete-button { + color: var(--mxt-brand); +} + +.mxfeedback-tool-button__color:focus, +.mxfeedback-tool-button__thickness:focus { + outline-color: var(--mxt-brand); +} + +.take-picture-save-button { + background-color: var(--mxt-brand); + color: var(--mxt-brand-ink); +} diff --git a/cmd/mxcli/theme/assets/signal/files/theme/web/main.scss b/cmd/mxcli/theme/assets/signal/files/theme/web/main.scss index 6ebdf074d..48c3e338b 100644 --- a/cmd/mxcli/theme/assets/signal/files/theme/web/main.scss +++ b/cmd/mxcli/theme/assets/signal/files/theme/web/main.scss @@ -7,3 +7,4 @@ $mxcli-theme-variant: {{VARIANT}}; @import "mxcli-atlas-map"; @import "mxcli-signal"; +@import "mxcli-widgets"; diff --git a/cmd/mxcli/theme/assets/signal/theme.json b/cmd/mxcli/theme/assets/signal/theme.json index 9ddc93bf8..c4d35e3a6 100644 --- a/cmd/mxcli/theme/assets/signal/theme.json +++ b/cmd/mxcli/theme/assets/signal/theme.json @@ -28,6 +28,11 @@ "mode": "block", "purpose": "Layer 2 \u2014 dark palette, variant blocks, fonts, recipes" }, + { + "path": "theme/web/_mxcli-widgets.scss", + "mode": "block", + "purpose": "the widget-module layer \u2014 rules for colours Sass bakes before any token exists" + }, { "path": "theme/web/main.scss", "mode": "block", diff --git a/cmd/mxcli/theme/theme_test.go b/cmd/mxcli/theme/theme_test.go index 6af792e72..546624119 100644 --- a/cmd/mxcli/theme/theme_test.go +++ b/cmd/mxcli/theme/theme_test.go @@ -6,6 +6,7 @@ import ( "errors" "os" "path/filepath" + "regexp" "strings" "testing" ) @@ -300,24 +301,56 @@ func TestAllThemesAreWellFormed(t *testing.T) { // The Atlas wiring is what makes a palette swap cheap, so every theme has to // run through the same one. Shipped per theme (a theme package is meant to be // self-contained), which is exactly why it can drift. -func TestAtlasMapIsIdenticalInEveryTheme(t *testing.T) { +func TestSharedPartialsAreIdenticalInEveryTheme(t *testing.T) { themes, err := List() if err != nil { t.Fatal(err) } - var reference []byte - var referenceName string + for _, shared := range []string{"_mxcli-atlas-map.scss", "_mxcli-widgets.scss"} { + var reference []byte + var referenceName string + for _, th := range themes { + body, err := assetsFS.ReadFile("assets/" + th.Name + "/files/theme/web/" + shared) + if err != nil { + t.Fatalf("%s ships no %s: %v", th.Name, shared, err) + } + if reference == nil { + reference, referenceName = body, th.Name + continue + } + if string(body) != string(reference) { + t.Errorf("%s's %s has drifted from %s's", th.Name, shared, referenceName) + } + } + } +} + +// The widget layer exists because Sass bakes these colours before any custom +// property exists, so a rule that reintroduces a literal defeats the point. +func TestWidgetLayerResolvesEveryColourThroughAToken(t *testing.T) { + themes, err := List() + if err != nil { + t.Fatal(err) + } + literal := regexp.MustCompile(`(color|background|background-color|border-color|outline-color|box-shadow)\s*:\s*[^;]*(#[0-9a-fA-F]{3,8}|\brgba?\()`) for _, th := range themes { - body, err := assetsFS.ReadFile("assets/" + th.Name + "/files/theme/web/_mxcli-atlas-map.scss") + body, err := assetsFS.ReadFile("assets/" + th.Name + "/files/theme/web/_mxcli-widgets.scss") if err != nil { - t.Fatalf("%s ships no Atlas map: %v", th.Name, err) + t.Fatal(err) } - if reference == nil { - reference, referenceName = body, th.Name - continue + for i, line := range strings.Split(string(body), "\n") { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "//") { + continue + } + if literal.MatchString(trimmed) { + t.Errorf("%s _mxcli-widgets.scss:%d reintroduces a literal colour: %q", + th.Name, i+1, trimmed) + } } - if string(body) != string(reference) { - t.Errorf("%s's Atlas map has drifted from %s's", th.Name, referenceName) + // And it has to actually carry the fix that prompted the layer. + if !strings.Contains(string(body), ".pagination-bar") { + t.Errorf("%s: no rule for the Data Grid 2 pager caption", th.Name) } } } diff --git a/docs-site/src/tools/theme.md b/docs-site/src/tools/theme.md index cfe4bc97f..bdd491dfb 100644 --- a/docs-site/src/tools/theme.md +++ b/docs-site/src/tools/theme.md @@ -33,14 +33,15 @@ because two themes mapping the same Atlas variables would fight in the cascade. ## What it writes -Five things, all under `theme/`: +Six things, all under `theme/`: | File | What | |---|---| | `theme/web/custom-variables.scss` | the theme's palette — this is the file to edit | | `theme/web/_mxcli-atlas-map.scss` | the Atlas wiring: ~60 Atlas variables expressed in terms of the palette | | `theme/web/_mxcli-.scss` | the other palette, the variant blocks, `@font-face`, recipe classes | -| `theme/web/main.scss` | the variant switch plus two `@import` lines | +| `theme/web/_mxcli-widgets.scss` | the widget-module layer: colours Sass bakes before any token exists | +| `theme/web/main.scss` | the variant switch plus the `@import` lines | | `theme/web/mxcli-fonts/` | vendored fonts (SIL OFL 1.1) | **The model is never touched.** No `.mpr` changes, so nothing here can affect a @@ -137,6 +138,11 @@ mxcli block — anything outside the fence is never touched. - **`themesource//` is only compiled when `` matches a real module**, so a theme never writes there. `theme/web/main.scss` compiles last and is the correct home for app-level styling. +- **The widget modules bake some colours as Sass literals**, before any custom + property exists, so no token can move them — the Data Grid 2 pager caption is + the worst case, at 1.02:1 on a dark ground. `_mxcli-widgets.scss` corrects + those with ordinary rules; it is regenerated with the theme, so leave it alone + and put your own overrides outside the fence. ## Recipe classes From 81422227944d5f400632b65843bde25589e1a3e9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 12:10:44 +0000 Subject: [PATCH 25/31] fix(describe): stop inventing an `else` on a type split MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DESCRIBE printed a bare `else` for a `split type` that never had one, and a describe→exec roundtrip accumulated another each pass. An object-type decision always carries an `(empty)` outgoing flow — the null-object case — which the builder emits whether or not an `else` was written. DESCRIBE rendered that flow as an `else`. The artifact was invisible until the InheritanceCase writer landed: before that every branch flow degraded to a bare NoCase, so nothing distinguished the `(empty)` flow from a real case. Fixed in the describer: drop the `else` line when its body renders empty, using the same elseLineIdx/truncate pattern the if/else emitters already use. Exec re-creates the flow, so the omission is lossless — verified that describe→exec →describe is byte-stable for an empty-branch split, an all-branches-return split, and an authored else. The obvious fix — not emitting the branch in the builder when no `else` is written — was implemented first and is wrong. Without that flow the build fails CE0089 "The '(empty)' value should be configured for an outgoing flow." so it is load-bearing, and MDL's `else` on an inheritance split IS the `(empty)` case. That also explains a result from the previous commit: an `else` cannot substitute for the base entity's own case (CE0090) because `(empty)` and the base type cover different things. Worth recording how the wrong fix was caught: every shape was re-run through mxbuild, not just the test suite. The unit tests passed against it — the type splits it broke only failed at build time. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- .claude/skills/fix-issue.md | 1 + .../cmd_microflows_builder_actions.go | 13 ++++ .../cmd_microflows_inheritance_test.go | 75 +++++++++++++++++++ mdl/executor/cmd_microflows_show_helpers.go | 11 +++ mdl/executor/cmd_microflows_traverse_test.go | 50 +++++++++++++ 5 files changed, 150 insertions(+) diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 51cad6f20..41f92afd0 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -411,3 +411,4 @@ extracting `OffsetExpression`/`LimitExpression`. | A suite passes under `mxcli test --attach` and fails under `--local`, with assertions that depend on startup state (a loaded cache, seeded reference data) seeing zero rows | The `--local` runner pointed after-startup at its own registration microflow and did **not** chain the project's own, so the app's startup logic never ran. It was a deliberate choice (a known baseline) but was invisible: the run printed only `After-startup set to MxTest.RegisterEndpoint`, never that the user's microflow had been displaced | `cmd/mxcli/testrunner/runner.go` (`runEndpoint`), `cmd/mxcli/testrunner/cleanup_strategy.go` (`describeStartup`) | Capture project state **before** generating the endpoint MDL, and pass `state.afterStartup` to `GenerateEndpointMDL` so the generated flow chains it — the hosted `--test-endpoint` path already did this, and the mismatch between the two was the bug. Add `--skip-app-startup` for a deterministic empty baseline, and always print which of the two happened. Note the startup microflow's writes run at boot, outside any test transaction, so `@cleanup rollback` does not undo them. mxcli-formula1 findings #19 | | `mxcli test tests/ -p app/App.mpr --list` fails with `stat tests/: no such file or directory` while the same command without `--list` runs fine | The `--list` branch passed raw `args` to `ListTests`, bypassing `resolveTestPaths` — so a path relative to the project (rather than the working directory) resolved for execution but not for listing | `cmd/mxcli/cmd_test_run.go` (the `if list` branch) | Pass `resolveTestPaths(args, projectPath)` there too. When a command has two entry points into the same input, check both go through the same path resolution. mxcli-formula1 findings #15 | | After `SET` became optional, `mx check` on a project built from `02-microflow-examples.mdl` reports six errors that no MDL change caused: `[CE0109] "Undefined variable 'ProductList.Price'."` at four Aggregate list activities, and `[CE0015] "Aggregate function must specify a valid attribute."` at the expression-based one. Identical on both engines. `mxcli check` on the same script is silent | Two conversions existed for one syntax. `$Sum = sum($List.Price)` used to reach the dedicated `aggregateListStatement` rule; making `SET` optional put `setStatement` — alternative 5 of ~50 — in front of it, so ANTLR matched the lower-numbered alternative and the statement fell through to `buildSetStatement`'s fallback conversion, which joined list and attribute into one name and dropped the per-item expression entirely. Underneath, `buildListAggregateAsFunction` never appended the expression argument, so the SET path could not have seen it either | `mdl/grammar/domains/MDLMicroflow.g4` (`setStatement` moved LAST in `microflowStatement`), `mdl/visitor/visitor_microflow_statements.go` (`buildSetAggregate` replaces `extractVariableAndAttribute`), `mdl/visitor/visitor_microflow_expression.go` (`buildListAggregateAsFunction` appends the expression argument) | **A permissive alternative belongs last.** `$X = ` overlaps every `VARIABLE EQUALS ` statement in the rule — aggregates, list operations, RANGE — and ANTLR's ALL(*) picks the lowest-numbered alternative that matches, so a new general form silently steals from every specific one above it. **The measurement trap that let this ship**: the `SET?` change was swept with a control binary over every `mdl-examples/**/*.mdl` and found no difference — but with `mxcli check`, which parses and validates and never serializes. This defect lives between the AST and the BSON, where only `exec` + `mx check` can see it. Sweeping with `check` proves the grammar still *parses*; it proves nothing about what the visitor *builds*. For a grammar change, the control sweep must run the integration gate (`go test -tags integration -run TestMxCheck_DoctypeScripts`), not `check`. Fix proven by reverting both halves and watching the new tests fail with the reported symptom. Tests `mdl/visitor/visitor_microflow_aggregate_test.go`. Upstream CI on the ako→mendixlabs sync PR | +| `DESCRIBE microflow` prints a bare `else` on a `split type` the author never wrote one for, and each describe→exec pass accumulates another | An object-type decision always carries an `(empty)` outgoing flow (the null-object case), emitted by the builder whether or not an `else` was written. DESCRIBE rendered that flow as `else`. Invisible until the `InheritanceCase` writer landed — before that every branch degraded to `NoCase`, so nothing distinguished it from a real case | `mdl/executor/cmd_microflows_show_helpers.go` (the `elseFlow` block in the inheritance-split traversal) | Drop the `else` line when its body renders empty — the same `elseLineIdx`/truncate pattern the if/else emitters already use. Exec re-creates the flow, so the omission is lossless and the roundtrip is stable. **Do NOT 'fix' this in the builder**: removing the empty-entity branch there fails the build with **CE0089** "The '(empty)' value should be configured for an outgoing flow" — that flow is load-bearing and is why `else` cannot substitute for the base entity's case (CE0090); `(empty)` and the base type cover different things. That wrong fix was implemented first and caught only because every shape was re-run through mxbuild, not because a unit test failed. Tests `TestBuilder_InheritanceSplitKeepsEmptyCaseFlow` (builder must KEEP it) and `TestTraverseFlow_InheritanceSplitOmitsEmptyElse` (describe must not print it) | diff --git a/mdl/executor/cmd_microflows_builder_actions.go b/mdl/executor/cmd_microflows_builder_actions.go index 684ecd01e..5a4b0694c 100644 --- a/mdl/executor/cmd_microflows_builder_actions.go +++ b/mdl/executor/cmd_microflows_builder_actions.go @@ -601,6 +601,19 @@ func (fb *flowBuilder) addStructuredInheritanceSplit(s *ast.InheritanceSplitStmt for _, c := range s.Cases { addBranch(qualifiedNameString(c.Entity), c.Body) } + // The empty-entity branch is NOT optional, and is not a "default" case: on an + // object-type decision it is the `(empty)` flow, for a null object. Dropping + // it when no `else` is written fails the build with + // + // CE0089 "The '(empty)' value should be configured for an outgoing flow." + // + // (verified on mxbuild 11.6.6). So it is emitted unconditionally, and MDL's + // `else` on an inheritance split IS that `(empty)` case — which is also why + // an `else` cannot substitute for the base entity's own case (CE0090): the + // two cover different things. + // + // DESCRIBE suppresses this flow when its body is empty, so a describe→exec + // roundtrip does not accumulate `else` blocks; exec re-creates it here. addBranch("", s.ElseBody) fb.posX = mergeX diff --git a/mdl/executor/cmd_microflows_inheritance_test.go b/mdl/executor/cmd_microflows_inheritance_test.go index ef93009ad..069d7d576 100644 --- a/mdl/executor/cmd_microflows_inheritance_test.go +++ b/mdl/executor/cmd_microflows_inheritance_test.go @@ -422,3 +422,78 @@ func assertLineContains(t *testing.T, lines []string, want string) { } t.Fatalf("expected output to contain %q, got %v", want, lines) } + +// TestBuilder_InheritanceSplitKeepsEmptyCaseFlow pins something that LOOKS +// like a phantom and is not. +// +// A `split type` with no authored `else` still gets an outgoing flow whose +// InheritanceCase has an empty entity name. That flow is the `(empty)` case of +// an object-type decision — the null-object branch — not a "default" case. +// Dropping it when no `else` is written fails the build with +// +// CE0089 "The '(empty)' value should be configured for an outgoing flow." +// +// verified on mxbuild 11.6.6. An earlier attempt to remove it as unauthored +// broke every type split, so the builder must keep emitting it unconditionally; +// what changed instead is that DESCRIBE no longer renders it as a bare `else` +// (see TestDescribe_InheritanceSplitOmitsEmptyElse). +// +// This also explains why an `else` cannot substitute for the base entity's own +// case (CE0090): `(empty)` and the base type cover different things. +func TestBuilder_InheritanceSplitKeepsEmptyCaseFlow(t *testing.T) { + build := func(elseBody []ast.MicroflowStatement) (splitID model.ID, flows []*microflows.SequenceFlow) { + fb := &flowBuilder{spacing: HorizontalSpacing, measurer: &layoutMeasurer{}} + oc := fb.buildFlowGraph([]ast.MicroflowStatement{ + &ast.InheritanceSplitStmt{ + Variable: "A", + Cases: []ast.InheritanceSplitCase{ + {Entity: ast.QualifiedName{Module: "SP", Name: "Dog"}}, + {Entity: ast.QualifiedName{Module: "SP", Name: "Animal"}}, + }, + ElseBody: elseBody, + }, + &ast.ReturnStmt{}, + }, nil) + for _, obj := range oc.Objects { + if s, ok := obj.(*microflows.InheritanceSplit); ok { + splitID = s.ID + } + } + for _, f := range oc.Flows { + if f.OriginID == splitID { + flows = append(flows, f) + } + } + return splitID, flows + } + + countEmptyCase := func(flows []*microflows.SequenceFlow) int { + n := 0 + for _, f := range flows { + if ic, ok := f.CaseValue.(*microflows.InheritanceCase); ok && ic.EntityQualifiedName == "" { + n++ + } + } + return n + } + + t.Run("no else authored still emits the (empty) flow", func(t *testing.T) { + _, flows := build(nil) + if len(flows) != 3 { + t.Errorf("split has %d outgoing flows, want 3 (two cases + the (empty) case)", len(flows)) + } + if got := countEmptyCase(flows); got != 1 { + t.Errorf("got %d empty-entity case flows, want exactly 1 — without it the build fails CE0089", got) + } + }) + + t.Run("authored else reuses the same flow", func(t *testing.T) { + _, flows := build([]ast.MicroflowStatement{&ast.ReturnStmt{}}) + if len(flows) != 3 { + t.Errorf("split has %d outgoing flows, want 3", len(flows)) + } + if got := countEmptyCase(flows); got != 1 { + t.Errorf("got %d empty-entity case flows, want exactly 1", got) + } + }) +} diff --git a/mdl/executor/cmd_microflows_show_helpers.go b/mdl/executor/cmd_microflows_show_helpers.go index 55b6491a3..499efac40 100644 --- a/mdl/executor/cmd_microflows_show_helpers.go +++ b/mdl/executor/cmd_microflows_show_helpers.go @@ -1399,8 +1399,19 @@ func emitInheritanceSplitStatement( traverseFlowUntilMerge(ctx, flow.DestinationID, branchStopID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+1, sourceMap, headerLineCount, annotationsByTarget) } if elseFlow != nil { + elseLineIdx := len(*lines) *lines = append(*lines, indentStr+"else") traverseFlowUntilMerge(ctx, elseFlow.DestinationID, branchStopID, activityMap, flowsByOrigin, flowsByDest, splitMergeMap, cloneVisited(visited), entityNames, microflowNames, lines, indent+1, sourceMap, headerLineCount, annotationsByTarget) + // Remove an empty else block, as the if/else emitters above do. On an + // object-type decision this flow is the `(empty)` case (for a null + // object), which the builder emits unconditionally — CE0089 without it — + // so a split that never had an authored `else` still has the flow. Left + // in, DESCRIBE printed a bare `else` the author never wrote and a + // describe→exec roundtrip accumulated one each pass. Exec re-creates the + // flow, so dropping the empty rendering is lossless. + if len(*lines) == elseLineIdx+1 { + *lines = (*lines)[:elseLineIdx] + } } *lines = append(*lines, indentStr+"end split;") } diff --git a/mdl/executor/cmd_microflows_traverse_test.go b/mdl/executor/cmd_microflows_traverse_test.go index ff2b2f4aa..4264a1930 100644 --- a/mdl/executor/cmd_microflows_traverse_test.go +++ b/mdl/executor/cmd_microflows_traverse_test.go @@ -1536,3 +1536,53 @@ func TestTraverseFlow_Issue528_NestedGuardDoesNotSwallowSharedActivities(t *test t.Errorf("issue #528: shared activity emitted inside outer if block instead of after end if;\n%s", out) } } + +// TestTraverseFlow_InheritanceSplitOmitsEmptyElse covers the `else` DESCRIBE +// used to invent on a type split. +// +// An object-type decision always has an `(empty)` outgoing flow — the +// null-object case, which the builder emits whether or not an `else` was +// written, because without it the build fails CE0089. DESCRIBE rendered that +// flow as a bare `else`, so describing a split the author wrote without one +// produced MDL with an `else`, and each describe→exec pass accumulated another. +// +// The empty rendering is dropped, matching what the if/else emitters already +// do. Exec re-creates the flow, so nothing is lost. An else that HAS a body +// still renders. +func TestTraverseFlow_InheritanceSplitOmitsEmptyElse(t *testing.T) { + e := &Executor{} + run := func(emptyGoesToMerge bool) string { + activityMap := map[model.ID]microflows.MicroflowObject{ + mkID("split"): µflows.InheritanceSplit{ + BaseMicroflowObject: mkObj("split"), + VariableName: "A", + }, + mkID("log"): µflows.ActionActivity{BaseActivity: microflows.BaseActivity{BaseMicroflowObject: mkObj("log")}, Action: µflows.LogMessageAction{LogLevel: "Info", LogNodeName: "'T'"}}, + mkID("merge"): µflows.ExclusiveMerge{BaseMicroflowObject: mkObj("merge")}, + } + // The (empty) case flow: straight to the merge means "no else body". + emptyDest := mkID("merge") + if !emptyGoesToMerge { + emptyDest = mkID("log") + } + flowsByOrigin := map[model.ID][]*microflows.SequenceFlow{ + mkID("split"): { + {OriginID: mkID("split"), DestinationID: mkID("merge"), CaseValue: µflows.InheritanceCase{EntityQualifiedName: "SP.Dog"}}, + {OriginID: mkID("split"), DestinationID: emptyDest, CaseValue: µflows.InheritanceCase{EntityQualifiedName: ""}}, + }, + mkID("log"): {{OriginID: mkID("log"), DestinationID: mkID("merge")}}, + } + var lines []string + traverseFlowUntilMerge(e.newExecContext(context.Background()), mkID("split"), mkID("merge"), + activityMap, flowsByOrigin, nil, map[model.ID]model.ID{mkID("split"): mkID("merge")}, + map[model.ID]bool{}, nil, nil, &lines, 0, nil, 0, nil) + return strings.Join(lines, "\n") + } + + if out := run(true); strings.Contains(out, "else") { + t.Errorf("DESCRIBE invented an `else` for the (empty) flow with no body:\n%s", out) + } + if out := run(false); !strings.Contains(out, "else") { + t.Errorf("an else WITH a body must still render:\n%s", out) + } +} From e62b3da1e5540eb3af70a773e23da27ad1e8d312 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 12:35:41 +0000 Subject: [PATCH 26/31] fix(odata): resolve constant credentials for the $metadata fetch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Literal credentials started working when the fetch learned to authenticate, but a constant reference still got a 401 and an empty client. That is the shape MDL pushes users towards — mxcli requires a constant for ServiceUrl, so a client written the documented way has constants for its credentials too. The tool insisted on the shape whose credentials it would not read. The quoted spelling was the sharp edge. `'@Module.ApiUser'` is a STRING_LITERAL, so the isLiteral flag says "literal" and the previous code sent the fifteen characters `@Module.ApiUser` as the username — worse than a 401, because it looks like it tried, and no unresolved-credential note fired either. All three spellings now resolve: a literal, `@Module.Name`, and the same reference quoted. A constant's design-time default is exactly what Studio Pro uses for its own fetch, so reading it is not a workaround — it is the value. An unknown constant, or one with no default, still reports itself unresolved rather than sending something that merely looks like a credential. Verified against a basic-auth server that 401s without credentials and 403s without a custom header: all three spellings cache the contract, where the quoted form previously failed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- mdl/executor/cmd_odata.go | 102 ++++++++++++++++--- mdl/executor/cmd_odata_metadata_auth_test.go | 52 +++++++++- 2 files changed, 139 insertions(+), 15 deletions(-) diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index 2e98d8cf9..bbdf8034b 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -1127,7 +1127,7 @@ Got: %s`, stmt.ServiceUrl) } newSvc.MetadataUrl = normalizedUrl - auth := metadataAuthFromStmt(stmt) + auth := metadataAuthFromStmt(ctx, stmt) metadata, hash, err := fetchODataMetadata(normalizedUrl, auth) if err != nil { fmt.Fprintf(ctx.Output, "Warning: could not fetch $metadata: %v\n", err) @@ -1896,27 +1896,26 @@ func (a *metadataFetchAuth) apply(req *http.Request) { // metadataAuthFromStmt collects the statement's own credentials and headers for // the design-time fetch, keeping only the literals. -func metadataAuthFromStmt(stmt *ast.CreateODataClientStmt) *metadataFetchAuth { +func metadataAuthFromStmt(ctx *ExecContext, stmt *ast.CreateODataClientStmt) *metadataFetchAuth { auth := &metadataFetchAuth{Headers: map[string]string{}} - switch { - case stmt.HttpUsernameIsLiteral: - auth.Username = stmt.HttpUsername - case stmt.HttpUsername != "": + consts := designTimeConstants(ctx) + + if v, ok := resolveCredential(stmt.HttpUsername, stmt.HttpUsernameIsLiteral, consts); ok { + auth.Username = v + } else if stmt.HttpUsername != "" { auth.Unresolved = append(auth.Unresolved, "HttpUsername ("+stmt.HttpUsername+")") } - switch { - case stmt.HttpPasswordIsLiteral: - auth.Password = stmt.HttpPassword - case stmt.HttpPassword != "": + if v, ok := resolveCredential(stmt.HttpPassword, stmt.HttpPasswordIsLiteral, consts); ok { + auth.Password = v + } else if stmt.HttpPassword != "" { // Named, not printed: a constant reference is a name, but the value it // resolves to is a secret and this line goes to the console. auth.Unresolved = append(auth.Unresolved, "HttpPassword ("+stmt.HttpPassword+")") } for _, h := range stmt.Headers { - switch { - case h.ValueIsLiteral: - auth.Headers[h.Key] = h.Value - case h.Value != "": + if v, ok := resolveCredential(h.Value, h.ValueIsLiteral, consts); ok { + auth.Headers[h.Key] = v + } else if h.Value != "" { auth.Unresolved = append(auth.Unresolved, "header "+h.Key+" ("+h.Value+")") } } @@ -1924,6 +1923,81 @@ func metadataAuthFromStmt(stmt *ast.CreateODataClientStmt) *metadataFetchAuth { return auth } +// resolveCredential turns an MDL property value into the string to send on the +// design-time fetch. +// +// Three spellings reach here and all three have to work, because the shape MDL +// pushes users towards is the constant reference — mxcli requires a constant for +// ServiceUrl, so a client written the documented way has constants for its +// credentials too (mxcli-formula1 #23 follow-up): +// +// HttpUsername: 'f1api' a literal +// HttpUsername: @Module.ApiUser a constant reference +// HttpUsername: '@Module.ApiUser' the same reference, quoted +// +// The quoted form is the trap: it is a STRING_LITERAL, so the isLiteral flag says +// "literal" and the naive reading sends the eleven characters `@Module.ApiUser` +// as the username. Worse than a 401, because it looks like it tried. +// +// A constant's design-time default is exactly what Studio Pro uses for the same +// fetch, so resolving it here is not a workaround — it is the value. +func resolveCredential(value string, isLiteral bool, consts map[string]string) (string, bool) { + if value == "" { + return "", false + } + if ref, ok := constantReference(value, isLiteral); ok { + v, found := consts[strings.ToLower(ref)] + return v, found && v != "" + } + if isLiteral { + return value, true + } + return "", false +} + +// constantReference reports whether a property value names a constant, and which +// one. A leading @ marks a reference in either spelling; an unquoted qualified +// name is one too, since a bare Module.Name cannot be a credential. +func constantReference(value string, isLiteral bool) (string, bool) { + if rest, found := strings.CutPrefix(value, "@"); found { + return rest, true + } + if !isLiteral && strings.Contains(value, ".") { + return value, true + } + return "", false +} + +// designTimeConstants maps a constant's qualified name (lowercased) to its +// default value. Best-effort: a project that cannot be read yields an empty map, +// and every reference then reports itself unresolved rather than failing the +// statement. +func designTimeConstants(ctx *ExecContext) map[string]string { + out := map[string]string{} + if ctx == nil || ctx.Backend == nil { + return out + } + consts, err := ctx.Backend.ListConstants() + if err != nil { + return out + } + h, err := getHierarchy(ctx) + if err != nil { + return out + } + for _, c := range consts { + if c == nil { + continue + } + mod := h.GetModuleName(h.FindModuleID(c.ContainerID)) + if mod == "" { + continue + } + out[strings.ToLower(mod+"."+c.Name)] = c.DefaultValue + } + return out +} + // hints explains a failed fetch when the reason is credentials mxcli could not // resolve, and points at the workaround that also happens to be better practice. func (a *metadataFetchAuth) hints() []string { diff --git a/mdl/executor/cmd_odata_metadata_auth_test.go b/mdl/executor/cmd_odata_metadata_auth_test.go index 455286d29..ec11fe78b 100644 --- a/mdl/executor/cmd_odata_metadata_auth_test.go +++ b/mdl/executor/cmd_odata_metadata_auth_test.go @@ -69,7 +69,7 @@ func TestMetadataAuthFromStmt_LiteralsOnly(t *testing.T) { {Key: "X-Token", Value: "Module.Token"}, }, } - auth := metadataAuthFromStmt(stmt) + auth := metadataAuthFromStmt(nil, stmt) if auth.Username != "f1api" { t.Errorf("Username = %q, want the literal f1api", auth.Username) @@ -95,3 +95,53 @@ func TestMetadataAuthFromStmt_LiteralsOnly(t *testing.T) { } } } + +// mxcli-formula1 #23 follow-up: literal credentials worked after the first fix, +// but a constant reference still produced a 401 and an empty client — and the +// same release made a constant `ServiceUrl` mandatory, so the shape mxcli +// insists on for the URL was the shape whose credentials it would not read. +// +// The quoted spelling was the sharp edge. `'@Module.ApiUser'` is a STRING_LITERAL, +// so the isLiteral flag says "literal" and the naive reading sent the fifteen +// characters `@Module.ApiUser` as the username: worse than a 401, because it +// looks like it tried. +func TestResolveCredential(t *testing.T) { + consts := map[string]string{ + "m.apiuser": "f1api", + "m.apipass": "s3cret", + "m.empty": "", + } + cases := []struct { + name string + value string + isLiteral bool + want string + wantOK bool + }{ + {"a literal is itself", "f1api", true, "f1api", true}, + {"a quoted constant reference resolves", "@M.ApiUser", true, "f1api", true}, + {"a bare constant reference resolves", "@M.ApiUser", false, "f1api", true}, + {"an unquoted qualified name is a reference too", "M.ApiPass", false, "s3cret", true}, + {"case-insensitive, as MDL is elsewhere", "@m.APIUSER", true, "f1api", true}, + // Unresolvable cases must report themselves rather than send something + // that merely looks like a credential. + {"an unknown constant is unresolved", "@M.Nope", true, "", false}, + {"a constant with no default is unresolved", "@M.Empty", true, "", false}, + {"an empty value is nothing", "", true, "", false}, + // A literal that happens to contain a dot is still a literal — passwords + // contain dots, and that must not be read as a reference. + {"a dotted literal stays a literal", "s3.cret", true, "s3.cret", true}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got, ok := resolveCredential(tc.value, tc.isLiteral, consts) + if ok != tc.wantOK { + t.Fatalf("resolved = %v, want %v (value %q, literal %v)", ok, tc.wantOK, tc.value, tc.isLiteral) + } + if got != tc.want { + t.Errorf("got %q, want %q", got, tc.want) + } + }) + } +} From 9eed180180098eee42cda54a59ea568ea0a95b48 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 12:45:44 +0000 Subject: [PATCH 27/31] fix(alter-page): reach widgets inside a customContent column MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `alter page … set on ` reported "widget not found" when the widget lived inside a datagrid column rendered as customContent, so the only way to touch it was CREATE OR REPLACE PAGE — a full page rewrite. findInWidgetChildren's pluggable branch searched the grid's own Object.Properties[].Value.Widgets and matched columns by their derived name, but never descended into a COLUMN's own content. Columns live at Object.Properties[columns].Value.Objects[]; a column's widgets are one level deeper, at Properties[content].Value.Widgets[]. Addressing is by the nested widget's OWN name. A `grid.column.widget` path was considered and rejected: DataGrid2 columns carry no stored name in the MPR (the existing findBsonColumn documents this), so the column segment could only ever be a derived name — the bound attribute, or the caption — which changes the moment someone edits the caption, leaving such a path silently stale. The nested widget's name is real and stable, and the grammar needs no change. A second test pins that a column still resolves by its derived name, since the new descent runs in the same loop and could otherwise shadow it. Fixes #834 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8 --- mdl/backend/pagemutator/mutator.go | 62 +++++++-- mdl/backend/pagemutator/mutator_test.go | 162 ++++++++++++++++++++++++ 2 files changed, 210 insertions(+), 14 deletions(-) diff --git a/mdl/backend/pagemutator/mutator.go b/mdl/backend/pagemutator/mutator.go index a320725f6..a3cbfa940 100644 --- a/mdl/backend/pagemutator/mutator.go +++ b/mdl/backend/pagemutator/mutator.go @@ -1082,6 +1082,26 @@ func findInWidgetChildren(wDoc bson.D, widgetName string) *bsonWidgetResult { colPropKeys: colPropKeyMap, } } + // Descend into the column's OWN content widgets. A column + // rendered as customContent holds a widget tree at + // Properties[content].Value.Widgets — one level deeper than the + // pluggable search above, which only reaches the grid's own + // Object.Properties[].Value.Widgets. Without this, a widget + // inside a customContent column was unreachable by ALTER PAGE + // and the only remedy was rewriting the page (issue #834). + for _, cProp := range bsonnav.DGetArrayElements(bsonnav.DGet(colDoc, "Properties")) { + cPropDoc, ok := cProp.(bson.D) + if !ok { + continue + } + cValDoc := bsonnav.DGetDoc(cPropDoc, "Value") + if cValDoc == nil { + continue + } + if result := findInWidgetArray(cValDoc, "Widgets", widgetName); result != nil { + return result + } + } } break // only one "columns" property per widget } @@ -2352,26 +2372,32 @@ func buildDesignPropertyValueDoc(valueType, option string) bson.D { } func setWidgetCaptionMut(widget bson.D, value any) error { - caption := bsonnav.DGetDoc(widget, "Caption") - if caption == nil { - return mdlerrors.NewValidation("widget has no Caption property") + if caption := bsonnav.DGetDoc(widget, "Caption"); caption != nil { + setTranslatableText(caption, "", value) + return nil } - setTranslatableText(caption, "", value) - return nil + // An ActionButton has no `Caption` document: its caption is a + // Forms$ClientTemplate stored under `CaptionTemplate` (Template → Items[] → + // Translation.Text), the same shape setWidgetContentMut walks. Without this + // branch `alter page … set Caption = '…' on