Repository navigation
fix: prevent version bump script from following symlinks - #2789
Rana Singh (ranadeepsingh) merged 3 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Hey Rana Singh (@ranadeepsingh) 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
There was a problem hiding this comment.
🟡 Changes recommended
Concurrent path replacement can bypass the checks and redirect a write outside the selected repository.
1 open issue
What changed in this PR
This PR hardens the version-bump maintenance script against symlinks that point outside the selected repository.
Changes:
- Skips symlinked files during discovery and warns when they are encountered.
- Adds containment checks before writing and excludes unsafe paths from broad verification sweeps.
- Adds a regression test confirming that an existing external symlink target remains unchanged.
| File | Description |
|---|---|
scripts/test_bump_version.py |
Adds coverage for an external symlink target. |
scripts/bump-version.py |
Adds symlink and repository-containment checks. |
Review effort: Balanced
2ed1380 to
a9bb6ec
Compare
## Summary Retain a validated repository directory descriptor across discovery, reads, writes, verification and docs subprocess working-directory selection. Reject unsupported platforms at CLI startup rather than reporting a no-op success. This is a non-feature maintenance-tooling fix for microsoft#2789. ## Prompting Intent The engineer asked to continue the existing PR, triage every unresolved review thread, and fix confirmed findings using the trusted SynapseML PR loop. Preserve the original approach, existing substitutions and prior symlink defenses. Do not merge the PR or claim that missing Azure validation has passed. After being told that four review lenses remained, the engineer explicitly requested pushing the changes and running /azp run. Publish this tested patch with outstanding review gates visible; do not claim a completed gauntlet. ## Linked Sources - Root descriptor finding: microsoft#2789 (comment) - Unsupported-platform CLI finding: microsoft#2789 (comment) - Original resolved file/parent-symlink finding: microsoft#2789 (comment) - Trusted upstream AGENTS.md, branch context and PR-loop guidance at acd6a8c. - Existing standalone regression suite: scripts/test_bump_version.py. - Versioned pre-commit review findings and resolution evidence: reviews/pr-2789/. ## Rationale Pin directory identity instead of resolving a potentially replaced pathname again. Descriptor-relative traversal and per-component no-follow opens keep the source tree selected consistently. Select descriptor-based working directories only in docs child processes without changing the caller's cwd; preflight existing website/docs paths before version-file writes and recheck before running those tools. Report late refusals with explicit partial-bump status and a working retry command, never advice to bypass safety checks. This is not a sandbox for arbitrary user-selected build definitions. Use an early capability error for unsupported platforms, preserve actionable OS errors, skip ordinary binary sweep inputs quietly, and use nonblocking opens so special-file validation cannot hang on FIFOs. Focused CLI regressions reproduce the old failures and exercise normal and docs-enabled paths, real child cwd errors, pre-write refusals, auto-detect retries and fstat cleanup. Validate .git relative to the pinned root, require the retained descriptor on rooted helpers, and run snapshot reads through that same API. Distinguish ordinary I/O errors from policy refusals, deduplicate sweep warnings, and document POSIX/WSL support. Validation: 249 focused tests passed, with 4 snapshot tests deselected. The full suite has 252 passes and one snapshot failure independently reproduced on unmodified upstream; unrelated docs/review version strings cause it. Black 22.3.0 checks and git diff --check passed. Azure validation is authorized and must run on the published tested revision; it has not run before this commit. Pre-commit review is incomplete: round one is clean, round two observations have explicit driver dispositions, and the remaining four lenses continue on the same frozen source patch. The repeated explicit publication request takes precedence over waiting to publish; no review or CI gate is marked passed by this commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 480c93f4-c1f8-4cf4-80f1-d19407097bbb
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
## Summary Keep the version bump script inside the repository selected with `--repo-root`. Skip symlinks, reject unsafe paths, and use rooted directory file descriptors for race-safe reads and writes. ## Prompting Intent The engineer asked to implement the confirmed hardening case from the private security review, add focused regression coverage, validate it, and prepare the public pull request. The change should be small and should not describe private report details in public Git or PR text. ## Linked Sources - Private security triage: reviewed in the SynapseML advisory thread; detailed report information deliberately omitted from public history. - Repository guidance: AGENTS.md and the master branch context. - Existing test suite: scripts/test_bump_version.py. ## Rationale Resolving and checking each candidate path closes the concrete symlink escape while preserving normal in-repository version replacement. The script now skips symlinks during discovery and post-verification, and opens files by their relative path through the repository root. It also rejects symlinked or non-regular final paths and fails closed on platforms without the directory file descriptor and no-follow support needed for a race-safe open. The change is narrower than changing CLI behavior or adding a global scanner suppression. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: af31ef86-1171-422a-abeb-c63e597a01a2
## Summary Retain a validated repository directory descriptor across discovery, reads, writes, verification and docs subprocess working-directory selection. Reject unsupported platforms at CLI startup rather than reporting a no-op success. This is a non-feature maintenance-tooling fix for microsoft#2789. ## Prompting Intent The engineer asked to continue the existing PR, triage every unresolved review thread, and fix confirmed findings using the trusted SynapseML PR loop. Preserve the original approach, existing substitutions and prior symlink defenses. Do not merge the PR or claim that missing Azure validation has passed. After being told that four review lenses remained, the engineer explicitly requested pushing the changes and running /azp run. Publish this tested patch with outstanding review gates visible; do not claim a completed gauntlet. ## Linked Sources - Root descriptor finding: microsoft#2789 (comment) - Unsupported-platform CLI finding: microsoft#2789 (comment) - Original resolved file/parent-symlink finding: microsoft#2789 (comment) - Trusted upstream AGENTS.md, branch context and PR-loop guidance at acd6a8c. - Existing standalone regression suite: scripts/test_bump_version.py. - Versioned pre-commit review findings and resolution evidence: reviews/pr-2789/. ## Rationale Pin directory identity instead of resolving a potentially replaced pathname again. Descriptor-relative traversal and per-component no-follow opens keep the source tree selected consistently. Select descriptor-based working directories only in docs child processes without changing the caller's cwd; preflight existing website/docs paths before version-file writes and recheck before running those tools. Report late refusals with explicit partial-bump status and a working retry command, never advice to bypass safety checks. This is not a sandbox for arbitrary user-selected build definitions. Use an early capability error for unsupported platforms, preserve actionable OS errors, skip ordinary binary sweep inputs quietly, and use nonblocking opens so special-file validation cannot hang on FIFOs. Focused CLI regressions reproduce the old failures and exercise normal and docs-enabled paths, real child cwd errors, pre-write refusals, auto-detect retries and fstat cleanup. Validate .git relative to the pinned root, require the retained descriptor on rooted helpers, and run snapshot reads through that same API. Distinguish ordinary I/O errors from policy refusals, deduplicate sweep warnings, and document POSIX/WSL support. Validation: 249 focused tests passed, with 4 snapshot tests deselected. The full suite has 252 passes and one snapshot failure independently reproduced on unmodified upstream; unrelated docs/review version strings cause it. Black 22.3.0 checks and git diff --check passed. Azure validation is authorized and must run on the published tested revision; it has not run before this commit. Pre-commit review is incomplete: round one is clean, round two observations have explicit driver dispositions, and the remaining four lenses continue on the same frozen source patch. The repeated explicit publication request takes precedence over waiting to publish; no review or CI gate is marked passed by this commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 480c93f4-c1f8-4cf4-80f1-d19407097bbb
Type: maintenance ## Summary Prevent initial repository opening from following a swapped ancestor symlink. Use all-component no-follow opening on macOS and descriptor-relative component opens on Linux, preserving search-only ancestor access and descriptor cleanup. Extend the CLI regressions and cover positive stale-version sweep warnings. Retain review findings, their dispositions, the user-canceled attempt, and the requested single light review. ## Prompting Intent The maintainer requested fixes for unresolved PR feedback, publication, review, and CI, then explicitly limited local review to one light round. Keep the fix scoped to the version-bump script and tests; do not merge the PR. ## Linked Sources - PR: microsoft#2789 - Ancestor-symlink finding: microsoft#2789 (comment) - Confirmed reproduction: microsoft#2789 (comment) - Related release tooling: microsoft#2628 - Local review and dispositions: reviews/pr-2789/ ## Rationale O_NOFOLLOW on a whole pathname protects only its final component. Checking every component at open time closes that gap without a vulnerable pathname recheck. Search-capable ancestor opens preserve legitimate restricted layouts; the final root remains readable for traversal. Keep existing sequential writes rather than introducing an unrelated multi-file transaction or unbounded open descriptor set. The known snapshot assertion also fails on unchanged master. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 12b72e17-cd57-4809-86c0-353b15b571a5
1deb126 to
82c7f9f
Compare
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
🔵 Needs a closer look
Platform-specific containment and child pre-exec behavior need maintainer confirmation because Linux execution remains unverified and current-head CI is still pending.
No open issues 🎉
1 resolved since last review
Review effort: Balanced
|
Follow-up to the current-head automated review: Linux containment and child pre-exec behavior are now exercised on the unchanged pushed head 82c7f9f. In an isolated Linux aarch64 container (Python 3.11.18, UID 65534, read-only source mount, network disabled during tests), |
|
Final validation: Azure build 243463209 succeeded on the reviewed head 82c7f9f merged with current master. The two infrastructure-failed jobs passed on attempt 2: language 98/98 and lightgbm5 35/35, with no skips. All 63 previously successful jobs were preserved. All four review threads are resolved, the single requested light review is clean, and non-root Linux regression evidence is linked in the description. Maintainer approval is the remaining merge gate; no merge has been performed. |
e65143c
into
microsoft:master


Related Issues/PRs
Related: #2628 also changes this script and must preserve these safeguards when integrated.
What changes are proposed in this pull request?
Prevent the version-bump script from changing files outside the selected repository when a file, parent directory, or repository-root component is replaced with a symlink.
The script refuses symlinks throughout root acquisition, using macOS's all-components no-follow flag or directory-relative component opens on Linux. Search-only ancestors remain supported. It then retains the root descriptor for reads, writes, discovery, validation, and docs working-directory selection. It rejects unsafe write targets and unsupported platforms, closes descriptors on failure, and leaves normal version replacements unchanged. Regression tests cover root-leaf and ancestor swaps, search-only ancestors, descriptor cleanup, and positive sweep warnings.
This is a standalone release-tooling fix: no product APIs, dependencies, pipeline definitions, or workflows change. POSIX descriptor support is required; Windows users can use WSL. External sbt/npm build definitions are not sandboxed.
Status: fix published in
82c7f9f723cbf2c810b34414b15a3afcd6dbca49; all four review threads are resolved. The maintainer-requested single light review is clean (runtime reviewer:gpt-6-astra), and the current-head automated review reports no open issues. Its Linux-validation concern is addressed by 57 passing non-root Linux integration tests on the unchanged commit. Azure build 243463209 succeeded on the exact target/head merge. GitHub has no failed, pending, or missing required checks, and the branch is up to date withmaster. Maintainer approval is the remaining merge gate. The independently reproduced local baseline snapshot failure remains disclosed below.How is this patch tested?
I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.
Before the root-acquisition fix, the new ancestor-swap CLI regression failed: the script reported success against the outside fixture.
Before the permission-compatibility follow-up,
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -I -B -m pytest -q -p no:cacheprovider scripts/test_bump_version.py -k 'cli_supports_search_only_ancestor or sweep_reports_unscanned_files': 1 failed, 1 passed; the search-only ancestor failed with a permission error.On the final local patch based on
e58441ffd60e5a2f64b83e6521154ef840b85091,PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -I -B -m pytest -q -p no:cacheprovider scripts/test_bump_version.py -k 'not test_no_unexpected_unanchored': 259 passed, 1 deselected.The deselected
TestSnapshotRegression.test_no_unexpected_unanchoredreports unrelated version strings in docs and existing review artifacts. The earlier full-suite run had 255 passed, 1 baseline failure, before four follow-up cases were added.On the unchanged target,
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -I -B -m pytest -q -p no:cacheprovider scripts/test_bump_version.py -k TestSnapshotRegression: 3 passed, 1 failed, with the same unanchored references. The assertion was not weakened.Black 22.3.0,
python -I -m black --check scripts/bump-version.pyandpython -I -m black --check scripts/test_bump_version.py: passed.git diff --check: passed.Scala compilation, codegen, and Spark tests were not run locally; no Scala/product code changes. The script regressions run locally, not through the current Azure pipeline. Full Azure CI is assessed separately.
On the unchanged pushed commit, an isolated Linux aarch64/Python 3.11.18 container ran as UID 65534 with a read-only source mount and network disabled during tests.
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -I -B -m pytest -v -rs -p no:cacheprovider scripts/test_bump_version.py -k 'TestIntegration or TestDocsCommands': 57 passed, 203 deselected, zero skips. This exercises LinuxO_PATH, search-only ancestors, root/ancestor swaps, descriptor cleanup, and docs child pre-exec success/failure paths. The temporary VM and containers were stopped after validation.Earlier review findings and their dispositions are retained under
reviews/pr-2789/. The maintainer explicitly requested one light review in place of further six-round passes;task-unknown-attempt-9-review-0-harness.mdrecords its clean result.Azure build 243463209 completed successfully on merge
1e321f25fd4879b99a88c5730c3b97c1846f7f92(targete58441ffd60e5a2f64b83e6521154ef840b85091, head82c7f9f723cbf2c810b34414b15a3afcd6dbca49). All 65 executed jobs succeeded.languageandlightgbm5initially failed during Azure TLS authentication setup, before tests ran; one targeted retry produced 98 and 35 passing tests respectively, with no skips, while preserving the other 63 successful jobs.InternalCompatandFabricE2Ewere intentionally excluded by the existing fork-PR conditions. The website coverage warning is expected for its out-of-process Spark tests; its published test run passed 111/111. Existing non-gating task-runtime and dependency-scan warnings are not claimed fixed by this PR.Does this PR change any dependencies?
Does this PR add a new feature? If so, have you added samples on website?
website/docs/documentationfolder.Make sure you choose the correct class
estimators/transformersand namespace.DocTablepoints to correct API link.npm startto make sure the website renders correctly.<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples.WebsiteSamplesTestsjob pass in the pipeline.