Skip to content

fix: prevent version bump script from following symlinks - #2789

Merged
Rana Singh (ranadeepsingh) merged 3 commits into
microsoft:masterfrom
ranadeepsingh:fix/prevent-version-bump-symlink-writes
Oct 29, 2026
Merged

Rana Singh (ranadeepsingh) merged 3 commits into
microsoft:masterfrom
ranadeepsingh:fix/prevent-version-bump-symlink-writes

Conversation

@ranadeepsingh

@ranadeepsingh Rana Singh (ranadeepsingh) commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Related Issues/PRs

Related: #2628 also changes this script and must preserve these safeguards when integrated.

What changes are proposed in this pull request?

Prevent the version-bump script from changing files outside the selected repository when a file, parent directory, or repository-root component is replaced with a symlink.

The script refuses symlinks throughout root acquisition, using macOS's all-components no-follow flag or directory-relative component opens on Linux. Search-only ancestors remain supported. It then retains the root descriptor for reads, writes, discovery, validation, and docs working-directory selection. It rejects unsafe write targets and unsupported platforms, closes descriptors on failure, and leaves normal version replacements unchanged. Regression tests cover root-leaf and ancestor swaps, search-only ancestors, descriptor cleanup, and positive sweep warnings.

This is a standalone release-tooling fix: no product APIs, dependencies, pipeline definitions, or workflows change. POSIX descriptor support is required; Windows users can use WSL. External sbt/npm build definitions are not sandboxed.

Status: fix published in 82c7f9f723cbf2c810b34414b15a3afcd6dbca49; all four review threads are resolved. The maintainer-requested single light review is clean (runtime reviewer: gpt-6-astra), and the current-head automated review reports no open issues. Its Linux-validation concern is addressed by 57 passing non-root Linux integration tests on the unchanged commit. Azure build 243463209 succeeded on the exact target/head merge. GitHub has no failed, pending, or missing required checks, and the branch is up to date with master. Maintainer approval is the remaining merge gate. The independently reproduced local baseline snapshot failure remains disclosed below.

How is this patch tested?

  • I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

  • Before the root-acquisition fix, the new ancestor-swap CLI regression failed: the script reported success against the outside fixture.

  • Before the permission-compatibility follow-up, PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -I -B -m pytest -q -p no:cacheprovider scripts/test_bump_version.py -k 'cli_supports_search_only_ancestor or sweep_reports_unscanned_files': 1 failed, 1 passed; the search-only ancestor failed with a permission error.

  • On the final local patch based on e58441ffd60e5a2f64b83e6521154ef840b85091, PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -I -B -m pytest -q -p no:cacheprovider scripts/test_bump_version.py -k 'not test_no_unexpected_unanchored': 259 passed, 1 deselected.

  • The deselected TestSnapshotRegression.test_no_unexpected_unanchored reports unrelated version strings in docs and existing review artifacts. The earlier full-suite run had 255 passed, 1 baseline failure, before four follow-up cases were added.

  • On the unchanged target, PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -I -B -m pytest -q -p no:cacheprovider scripts/test_bump_version.py -k TestSnapshotRegression: 3 passed, 1 failed, with the same unanchored references. The assertion was not weakened.

  • Black 22.3.0, python -I -m black --check scripts/bump-version.py and python -I -m black --check scripts/test_bump_version.py: passed.

  • git diff --check: passed.

  • Scala compilation, codegen, and Spark tests were not run locally; no Scala/product code changes. The script regressions run locally, not through the current Azure pipeline. Full Azure CI is assessed separately.

  • On the unchanged pushed commit, an isolated Linux aarch64/Python 3.11.18 container ran as UID 65534 with a read-only source mount and network disabled during tests. PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -I -B -m pytest -v -rs -p no:cacheprovider scripts/test_bump_version.py -k 'TestIntegration or TestDocsCommands': 57 passed, 203 deselected, zero skips. This exercises Linux O_PATH, search-only ancestors, root/ancestor swaps, descriptor cleanup, and docs child pre-exec success/failure paths. The temporary VM and containers were stopped after validation.

  • Earlier review findings and their dispositions are retained under reviews/pr-2789/. The maintainer explicitly requested one light review in place of further six-round passes; task-unknown-attempt-9-review-0-harness.md records its clean result.

  • Azure build 243463209 completed successfully on merge 1e321f25fd4879b99a88c5730c3b97c1846f7f92 (target e58441ffd60e5a2f64b83e6521154ef840b85091, head 82c7f9f723cbf2c810b34414b15a3afcd6dbca49). All 65 executed jobs succeeded. language and lightgbm5 initially failed during Azure TLS authentication setup, before tests ran; one targeted retry produced 98 and 35 passing tests respectively, with no skips, while preserving the other 63 successful jobs.

  • InternalCompat and FabricE2E were intentionally excluded by the existing fork-PR conditions. The website coverage warning is expected for its out-of-process Spark tests; its published test run passed 111/111. Existing non-gating task-runtime and dependency-scan warnings are not claimed fixed by this PR.

Does this PR change any dependencies?

  • No. You can skip this section.
  • Yes. Make sure the dependencies are resolved correctly, and list changes here.

Does this PR add a new feature? If so, have you added samples on website?

  • No. You can skip this section.
  • Yes. Make sure you have added samples following below steps.
  1. Find the corresponding markdown file for your new feature in website/docs/documentation folder.
    Make sure you choose the correct class estimators/transformers and namespace.
  2. Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
  3. Make sure the DocTable points to correct API link.
  4. Navigate to website folder, and run npm start to make sure the website renders correctly.
  5. Don't forget to add <!--pytest-codeblocks:cont--> before each python code blocks to enable auto-tests for python samples.
  6. Make sure the WebsiteSamplesTests job pass in the pipeline.

Copilot AI balanced review requested due to automatic review settings October 23, 2026 10:13
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions

Copy link
Copy Markdown

Hey Rana Singh (@ranadeepsingh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

  • fix: Fix LightGBM crashes with empty partitions
  • feat: Make HTTP on Spark back-offs configurable
  • docs: Update Spark Serving usage
  • build: Add codecov support
  • perf: improve LightGBM memory usage
  • refactor: make python code generation rely on classes
  • style: Remove nulls from CNTKModel
  • test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Concurrent path replacement can bypass the checks and redirect a write outside the selected repository.

1 open issue
What changed in this PR

This PR hardens the version-bump maintenance script against symlinks that point outside the selected repository.

Changes:

  • Skips symlinked files during discovery and warns when they are encountered.
  • Adds containment checks before writing and excludes unsafe paths from broad verification sweeps.
  • Adds a regression test confirming that an existing external symlink target remains unchanged.
File Description
scripts/​test_bump_version.py Adds coverage for an external symlink target.
scripts/​bump-version.py Adds symlink and repository-containment checks.

Review effort: Balanced

Comment thread scripts/bump-version.py Resolved
Copilot AI balanced review requested due to automatic review settings October 23, 2026 11:20
@ranadeepsingh
Rana Singh (ranadeepsingh) force-pushed the fix/prevent-version-bump-symlink-writes branch from 2ed1380 to a9bb6ec Compare October 23, 2026 11:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Root-directory swaps can still permit outside writes, and unsupported platforms can report success without updating files.

2 open issues
1 resolved since last review

Review effort: Balanced

Comment thread scripts/bump-version.py Resolved
Comment thread scripts/bump-version.py Resolved
Rana Singh (ranadeepsingh) added a commit to ranadeepsingh/SynapseML that referenced this pull request Oct 27, 2026
## Summary
Retain a validated repository directory descriptor across discovery, reads,
writes, verification and docs subprocess working-directory selection. Reject
unsupported platforms at CLI startup rather than reporting a no-op success.
This is a non-feature maintenance-tooling fix for microsoft#2789.

## Prompting Intent
The engineer asked to continue the existing PR, triage every unresolved review
thread, and fix confirmed findings using the trusted SynapseML PR loop. Preserve
the original approach, existing substitutions and prior symlink defenses.
Do not merge the PR or claim that missing Azure validation has passed.
After being told that four review lenses remained, the engineer explicitly
requested pushing the changes and running /azp run. Publish this tested patch
with outstanding review gates visible; do not claim a completed gauntlet.

## Linked Sources
- Root descriptor finding:
  microsoft#2789 (comment)
- Unsupported-platform CLI finding:
  microsoft#2789 (comment)
- Original resolved file/parent-symlink finding:
  microsoft#2789 (comment)
- Trusted upstream AGENTS.md, branch context and PR-loop guidance at
  acd6a8c.
- Existing standalone regression suite: scripts/test_bump_version.py.
- Versioned pre-commit review findings and resolution evidence:
  reviews/pr-2789/.

## Rationale
Pin directory identity instead of resolving a potentially replaced pathname
again. Descriptor-relative traversal and per-component no-follow opens keep
the source tree selected consistently. Select descriptor-based working
directories only in docs child processes without changing the caller's cwd;
preflight existing website/docs paths before version-file writes and recheck
before running those tools. Report late refusals with explicit partial-bump
status and a working retry command, never advice to bypass safety checks.
This is not a sandbox for arbitrary user-selected build definitions.

Use an early capability error for unsupported platforms, preserve actionable
OS errors, skip ordinary binary sweep inputs quietly, and use nonblocking
opens so special-file validation cannot hang on FIFOs. Focused CLI regressions
reproduce the old failures and exercise normal and docs-enabled paths, real
child cwd errors, pre-write refusals, auto-detect retries and fstat cleanup.
Validate .git relative to the pinned root, require the retained descriptor on
rooted helpers, and run snapshot reads through that same API. Distinguish
ordinary I/O errors from policy refusals, deduplicate sweep warnings, and
document POSIX/WSL support.

Validation: 249 focused tests passed, with 4 snapshot tests deselected.
The full suite has 252 passes and one snapshot failure independently reproduced
on unmodified upstream; unrelated docs/review version strings cause it.
Black 22.3.0 checks and git diff --check passed.
Azure validation is authorized and must run on the published tested revision;
it has not run before this commit. Pre-commit review is incomplete: round one
is clean, round two observations have explicit driver dispositions, and the
remaining four lenses continue on the same frozen source patch. The repeated
explicit publication request takes precedence over waiting to publish; no
review or CI gate is marked passed by this commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 480c93f4-c1f8-4cf4-80f1-d19407097bbb
Copilot AI balanced review requested due to automatic review settings October 27, 2026 08:59
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A verified ancestor-symlink race during initial root acquisition can redirect version writes into an outside repository.

1 open issue
2 resolved since last review

Review effort: Balanced

Comment thread scripts/bump-version.py Resolved
## Summary
Keep the version bump script inside the repository selected with
`--repo-root`. Skip symlinks, reject unsafe paths, and use rooted
directory file descriptors for race-safe reads and writes.

## Prompting Intent
The engineer asked to implement the confirmed hardening case from the
private security review, add focused regression coverage, validate it,
and prepare the public pull request. The change should be small and
should not describe private report details in public Git or PR text.

## Linked Sources
- Private security triage: reviewed in the SynapseML advisory thread;
  detailed report information deliberately omitted from public history.
- Repository guidance: AGENTS.md and the master branch context.
- Existing test suite: scripts/test_bump_version.py.

## Rationale
Resolving and checking each candidate path closes the concrete symlink
escape while preserving normal in-repository version replacement. The
script now skips symlinks during discovery and post-verification, and
opens files by their relative path through the repository root. It also
rejects symlinked or non-regular final paths and fails closed on
platforms without the directory file descriptor and no-follow support
needed for a race-safe open. The change is narrower than changing CLI
behavior or adding a global scanner suppression.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: af31ef86-1171-422a-abeb-c63e597a01a2
## Summary
Retain a validated repository directory descriptor across discovery, reads,
writes, verification and docs subprocess working-directory selection. Reject
unsupported platforms at CLI startup rather than reporting a no-op success.
This is a non-feature maintenance-tooling fix for microsoft#2789.

## Prompting Intent
The engineer asked to continue the existing PR, triage every unresolved review
thread, and fix confirmed findings using the trusted SynapseML PR loop. Preserve
the original approach, existing substitutions and prior symlink defenses.
Do not merge the PR or claim that missing Azure validation has passed.
After being told that four review lenses remained, the engineer explicitly
requested pushing the changes and running /azp run. Publish this tested patch
with outstanding review gates visible; do not claim a completed gauntlet.

## Linked Sources
- Root descriptor finding:
  microsoft#2789 (comment)
- Unsupported-platform CLI finding:
  microsoft#2789 (comment)
- Original resolved file/parent-symlink finding:
  microsoft#2789 (comment)
- Trusted upstream AGENTS.md, branch context and PR-loop guidance at
  acd6a8c.
- Existing standalone regression suite: scripts/test_bump_version.py.
- Versioned pre-commit review findings and resolution evidence:
  reviews/pr-2789/.

## Rationale
Pin directory identity instead of resolving a potentially replaced pathname
again. Descriptor-relative traversal and per-component no-follow opens keep
the source tree selected consistently. Select descriptor-based working
directories only in docs child processes without changing the caller's cwd;
preflight existing website/docs paths before version-file writes and recheck
before running those tools. Report late refusals with explicit partial-bump
status and a working retry command, never advice to bypass safety checks.
This is not a sandbox for arbitrary user-selected build definitions.

Use an early capability error for unsupported platforms, preserve actionable
OS errors, skip ordinary binary sweep inputs quietly, and use nonblocking
opens so special-file validation cannot hang on FIFOs. Focused CLI regressions
reproduce the old failures and exercise normal and docs-enabled paths, real
child cwd errors, pre-write refusals, auto-detect retries and fstat cleanup.
Validate .git relative to the pinned root, require the retained descriptor on
rooted helpers, and run snapshot reads through that same API. Distinguish
ordinary I/O errors from policy refusals, deduplicate sweep warnings, and
document POSIX/WSL support.

Validation: 249 focused tests passed, with 4 snapshot tests deselected.
The full suite has 252 passes and one snapshot failure independently reproduced
on unmodified upstream; unrelated docs/review version strings cause it.
Black 22.3.0 checks and git diff --check passed.
Azure validation is authorized and must run on the published tested revision;
it has not run before this commit. Pre-commit review is incomplete: round one
is clean, round two observations have explicit driver dispositions, and the
remaining four lenses continue on the same frozen source patch. The repeated
explicit publication request takes precedence over waiting to publish; no
review or CI gate is marked passed by this commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 480c93f4-c1f8-4cf4-80f1-d19407097bbb
Type: maintenance

## Summary
Prevent initial repository opening from following a swapped ancestor symlink.
Use all-component no-follow opening on macOS and descriptor-relative component
opens on Linux, preserving search-only ancestor access and descriptor cleanup.
Extend the CLI regressions and cover positive stale-version sweep warnings.
Retain review findings, their dispositions, the user-canceled attempt, and the
requested single light review.

## Prompting Intent
The maintainer requested fixes for unresolved PR feedback, publication, review,
and CI, then explicitly limited local review to one light round. Keep the fix
scoped to the version-bump script and tests; do not merge the PR.

## Linked Sources
- PR: microsoft#2789
- Ancestor-symlink finding: microsoft#2789 (comment)
- Confirmed reproduction: microsoft#2789 (comment)
- Related release tooling: microsoft#2628
- Local review and dispositions: reviews/pr-2789/

## Rationale
O_NOFOLLOW on a whole pathname protects only its final component. Checking
every component at open time closes that gap without a vulnerable pathname
recheck. Search-capable ancestor opens preserve legitimate restricted layouts;
the final root remains readable for traversal. Keep existing sequential writes
rather than introducing an unrelated multi-file transaction or unbounded open
descriptor set. The known snapshot assertion also fails on unchanged master.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 12b72e17-cd57-4809-86c0-353b15b571a5
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Platform-specific containment and child pre-exec behavior need maintainer confirmation because Linux execution remains unverified and current-head CI is still pending.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

Follow-up to the current-head automated review: Linux containment and child pre-exec behavior are now exercised on the unchanged pushed head 82c7f9f. In an isolated Linux aarch64 container (Python 3.11.18, UID 65534, read-only source mount, network disabled during tests), PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -I -B -m pytest -v -rs -p no:cacheprovider scripts/test_bump_version.py -k "TestIntegration or TestDocsCommands" passed 57 tests with no skips. This includes native Linux O_PATH ancestor/root swaps, search-only ancestors, descriptor cleanup, and docs child-cwd success/failure paths. The source fingerprint is unchanged; no additional code or review round was needed. Azure build 243463209 remains the separate repository-wide CI gate.

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

Final validation: Azure build 243463209 succeeded on the reviewed head 82c7f9f merged with current master. The two infrastructure-failed jobs passed on attempt 2: language 98/98 and lightgbm5 35/35, with no skips. All 63 previously successful jobs were preserved. All four review threads are resolved, the single requested light review is clean, and non-root Linux regression evidence is linked in the description. Maintainer approval is the remaining merge gate; no merge has been performed.

@ranadeepsingh
Rana Singh (ranadeepsingh) merged commit e65143c into microsoft:master Oct 29, 2026
79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants