Record bundle behavior with faults for every DMS API by shreyas-goenka · Pull Request #7231 · databricks/cli

shreyas-goenka · 2026-10-28T14:38:28Z

Changes

Add a fault case for each DMS API using a shared acceptance runner. Exercise bundle commands and deployment-history toggles, and record warnings and endpoint usage in snapshots.

Why

Ensure DMS outages do not fail bundle operations when resource state is file-backed.

Tests

./task fmt, ./task checks, ./task lint, and ./task test.

This PR was written by Codex.

github-actions · 2026-10-28T14:45:03Z

Commit: 939aecd1f | Workflow run

Regression test report

Selected 15 changed tests (limit=20, 0 not selected)

15 tests: regression=0 unreleased=0 coverage=15 failing=0

test	branch	main (`c9e71ca`)	latest (v1.23.0)
TestAccept/bundle/dms_fault/complete_version/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/create_deployment/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/create_version/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/delete_deployment/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/get_deployment/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/get_operation/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/get_resource/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/get_version/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/heartbeat/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/list_deployments/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/list_operations/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/list_resources/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/list_versions/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/update_deployment/DMS=true	✅	✅	✅
TestAccept/bundle/dms_fault/update_operation/DMS=true	✅	✅	✅

eng-dev-ecosystem-bot · 2026-10-28T15:03:37Z

Integration test report

Commit: 939aecd

Run: 41499239795

	Env	🔄flaky	✅pass	🙈skip	Time
✅	aws linux-2core-8gb		281	30	9:36
✅	aws-windows-latest-4core-16gb		283	28	3:39
✅	azure linux-2core-8gb		280	30	10:18
🔄	azure-windows-latest-4core-16gb	1	281	28	4:46
✅	gcp linux-2core-8gb		281	30	10:02
🔄	gcp-windows-latest-4core-16gb	1	282	28	6:37

	Test Name	azure-windows-latest-4core-16gb	gcp-windows-latest-4core-16gb
🔄	TestAccept	🔄f	🔄f

Top 5 slowest tests (at least 2 minutes):

duration	env	testname
4:20	azure linux-2core-8gb	TestAccept
4:19	aws linux-2core-8gb	TestAccept
4:10	gcp linux-2core-8gb	TestAccept
3:14	gcp linux-2core-8gb	TestFilerWorkspaceFilesExtensionsReadDir
2:30	gcp linux-2core-8gb	TestFilerWorkspaceFilesExtensionsRead

## Changes Support `*` within a path segment and `**` across path segments in `fault.py` patterns. Both match zero or more characters; all other characters are literal. Compile each pattern once when its rule is registered.. ## Why useful for tests being added in #7231.

shreyas-goenka · 2026-10-28T22:38:34Z

+=== commands: get_deployment fault enabled
+
+>>> bundle deploy --auto-approve
+  Warn: retrying after 500 Internal Server Error from GET /api/2.0/bundle/deployments/[DEPLOYMENT_ID]


these repeated warnings can be made prettier, we can iterate on them separately. It is good enough for now tbh.

shreyas-goenka · 2026-10-28T22:39:48Z

+
+  Destroy: 2 deleted
+
+DMS API called: no


this is an important signal, shows whether the API is called or not (should we thus expect a warning in the output or not).

ilyakuz-db

This approach is fine, but maybe it will be simpler instead of making a suite for each API, do suite for each CLI command, and force all DMS APIs to return 500 there (and maybe also add some other Go error to ensure that any bugs won't break the direct engine flow)

@denik WDYT?

ilyakuz-db · 2026-10-29T10:31:14Z

@@ -0,0 +1,148 @@
+


Should we add init / config-remote-sync? I assume they don't interact directly in DMS, but is it possible through some shared code?

shreyas-goenka · 2026-10-29T10:41:23Z

@ilyakuz-db if we make all APIs return 500, that'll not exercise all code paths since later flows can depend upon earlier ones succeeding. This is the most robust way to ensure nothing can break for customers.

shreyas-goenka changed the title ~~Test bundle commands with individual DMS API failures~~ Record bundle behavior with faults for every DMS API Oct 28, 2026

shreyas-goenka mentioned this pull request Oct 28, 2026

Support segment and recursive wildcards in fault patterns #7238

Merged

shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/test-bundle-dms-fail-open branch from 5dbdc5d to 9b95f43 Compare October 28, 2026 16:06

shreyas-goenka changed the base branch from shreyas-goenka_data/shreyas/generate-omit-deployment-metadata to shreyas-goenka_data/shreyas/test-fault-glob-segments October 28, 2026 16:06

shreyas-goenka added this pull request to stack #7239 October 28, 2026 16:07

shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/test-bundle-dms-fail-open branch from 9b95f43 to c769fef Compare October 28, 2026 16:15

Base automatically changed from shreyas-goenka_data/shreyas/test-fault-glob-segments to shreyas-goenka_data/shreyas/generate-omit-deployment-metadata October 28, 2026 16:50

shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/test-bundle-dms-fail-open branch from c769fef to 15d2a7e Compare October 28, 2026 16:50

shreyas-goenka mentioned this pull request Oct 28, 2026

Support segment and recursive wildcards in fault patterns #7241

Merged

shreyas-goenka removed this pull request from stack #7239 October 28, 2026 16:56

shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/test-bundle-dms-fail-open branch from 15d2a7e to e1c4557 Compare October 28, 2026 16:56

shreyas-goenka changed the base branch from shreyas-goenka_data/shreyas/generate-omit-deployment-metadata to shreyas-goenka_data/shreyas/test-fault-glob-main October 28, 2026 16:56

shreyas-goenka added this pull request to stack #7242 October 28, 2026 16:56

github-actions Bot added the DABs DABs related issues label Oct 28, 2026

Base automatically changed from shreyas-goenka_data/shreyas/test-fault-glob-main to main October 28, 2026 17:34

shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/test-bundle-dms-fail-open branch from 9e2109d to 1e09ad4 Compare October 28, 2026 17:34

shreyas-goenka commented Oct 28, 2026

View reviewed changes

shreyas-goenka requested review from denik and ilyakuz-db October 28, 2026 22:41

shreyas-goenka marked this pull request as ready for review October 28, 2026 22:53

shreyas-goenka requested review from a team as code owners October 28, 2026 22:53

ilyakuz-db approved these changes Oct 29, 2026

View reviewed changes

shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/test-bundle-dms-fail-open branch from 260fb4e to 0bb4e74 Compare October 29, 2026 12:57

shreyas-goenka enabled stacked auto-merge November 2, 2026 08:34

shreyas-goenka disabled stacked auto-merge November 2, 2026 09:49

shreyas-goenka enabled stacked auto-merge November 2, 2026 09:49

shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/test-bundle-dms-fail-open branch from 0bb4e74 to f1e38ff Compare November 2, 2026 12:58

shreyas-goenka added 10 commits November 2, 2026 19:17

Test bundle commands with individual DMS API failures

a264367

Record bundle output for faults across all DMS APIs

e47670d

Simplify and document the DMS fault test workflow

8402c0e

Support middle wildcards in fault injection patterns

80d7ebc

Document DMS fault test helpers with short comments

a471dc6

Test deployment history toggles under DMS API faults

4658694

Keep GetDeployment faults active across deployment IDs

1211013

Clarify the DMS fault bundle setup comment

30e8757

Make DMS fault runner safe to import for doctests

75b3de2

Resolve DMS test helper scripts by path on Windows

939aecd

shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/test-bundle-dms-fail-open branch from f1e38ff to 939aecd Compare November 2, 2026 18:17

Navigation Menu

Repository navigation

Record bundle behavior with faults for every DMS API - #7231

Record bundle behavior with faults for every DMS API#7231
shreyas-goenka wants to merge 10 commits into
mainfrom
shreyas-goenka_data/shreyas/test-bundle-dms-fail-open

shreyas-goenka commented Oct 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

shreyas-goenka Oct 28, 2026

Uh oh!

shreyas-goenka Oct 28, 2026

Uh oh!

ilyakuz-db left a comment

Uh oh!

ilyakuz-db Oct 29, 2026

Uh oh!

shreyas-goenka commented Oct 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

shreyas-goenka commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Tests

Uh oh!

github-actions Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression test report

Uh oh!

eng-dev-ecosystem-bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

shreyas-goenka Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

ilyakuz-db left a comment

Choose a reason for hiding this comment

Uh oh!

ilyakuz-db Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka commented Oct 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shreyas-goenka commented Oct 28, 2026 •

edited

Loading

github-actions Bot commented Oct 28, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Oct 28, 2026 •

edited

Loading