Skip to content

feat: refactor pull chain to extract and decode pull data - #1744

Open
jescalada wants to merge 30 commits into
mainfrom
refactor-pull-chain
Open

jescalada wants to merge 30 commits into
mainfrom
refactor-pull-chain

Conversation

@jescalada

@jescalada jescalada commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Description

Refactors the pull chain to actually decode pull data, and store it temporarily in .remote:

  • Gets the list of wants and haves commit hashes, stores in the Action
  • Makes a bare clone in .remote, which can be accessed by plugins/other processors
    • Bare clone is deleted via clearBareClone
  • Also refactors clearBareClone execution logic to simplify both chain.ts and clearBareClone itself

Related Issue

Based on the following PR that revamps the plugin system: #1683

Checklist

General

Documentation

  • Documentation has been added/updated for any new features

Tests

  • Tests have been added/updated for new functionality
  • Unit tests pass (npm test)
  • Linting and formatting pass (npm run lint and npm run format:check)
  • Type checks pass (npm run check-types)

@jescalada
jescalada requested a review from a team as a code owner September 21, 2026 03:01
@netlify

netlify Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for endearing-brigadeiros-63f9d0 ready!

Name Link
🔨 Latest commit 47a04cb
🔍 Latest deploy log https://app.netlify.com/projects/endearing-brigadeiros-63f9d0/deploys/6acf1cfda8f35a68bffee0f1
😎 Deploy Preview https://deploy-preview-1744.git-proxy.preview.finos.org
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@jescalada jescalada changed the title Refactor pull chain feat: refactor pull chain to extract and decode pull data Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
npm/simple-git ^4.0.2 UnknownUnknown

Scanned Files

  • plugins/git-proxy-plugin-samples/package.json

@codecov

codecov Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 86.97%. Comparing base (96413f2) to head (47a04cb).
⚠️ Report is 48 commits behind head on main.

Files with missing lines Patch % Lines
src/proxy/chain.ts 87.50% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1744      +/-   ##
==========================================
+ Coverage   86.84%   86.97%   +0.13%     
==========================================
  Files         113      113              
  Lines        6453     6472      +19     
  Branches     1253     1255       +2     
==========================================
+ Hits         5604     5629      +25     
+ Misses        543      537       -6     
  Partials      306      306              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jescalada jescalada mentioned this pull request Sep 21, 2026
18 of 22 tasks
@jescalada jescalada mentioned this pull request Oct 6, 2026
7 of 8 tasks
@jescalada

jescalada commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Reminder to self: check whether the action.diff override on AFTER_DIFF can still be reproduced, if so, find a fix (potentially preventing override or checking that the diff hasn't been tampered with?). Related: #1701

@jescalada jescalada added this to the 2.4.0 milestone Oct 19, 2026
@kriswest kriswest modified the milestone: 2.4.0 Oct 19, 2026

@dcoric dcoric left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Core changes look good and I verified them locally (build, unit, e2e, and a live clone through the proxy with both sample plugins loaded). One blocking issue in the sample plugin and two optional notes.

Blocking: want-set cache can be poisoned to skip scanning (extractPullData/fetchWanted.ts, roundKey / rememberRecentFetch)

roundKey is url|wants and onChainSuccess records the key for every allowed fetch, but pullSecretScanner only diffs base..want. Repro:

  1. POST a fetch with want X + have X. Diff is empty, scan passes, key url|X is cached.
  2. Any later git clone of X (base = empty tree) hits the cache, skips checkout and scan, and receives X unscanned.

Reproduced locally against trufflesecurity/test_keys: the poisoned clone delivered the AWS key. Repeating step 1 refreshes the 5 min TTL, and the victim's want set is discoverable via ls-refs.

Suggested fix: only remember a round when it was a full-tree scan, e.g. in rememberRecentFetch return early unless pullData.base === EMPTY_COMMIT_HASH. A full-tree pass is a superset of any incremental scan, so a cache hit is then always safe. Alternatively include the haves in the key.

Optional:

  • fetchWanted.ts treats a fetch with only want-ref lines (no want) as "client up to date" and passes it unscanned. Harmless against GitHub (no ref-in-want), but exploitable against upstreams with uploadpack.allowRefInWant.
  • README.md / plugins.mdx suggest GIT_SSL_NO_VERIFY as a fallback for self-signed upstreams. For a security sample I would drop that and only mention configuring the CA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants