Repository navigation
fix: authenticate the GitHub repo-size preflight and log its outcome - #124
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Credential reuse can cross service boundaries, and URL-embedded credentials remain unsupported by the preflight.
2 open issues
What changed in this PR
Updates GitHub’s repository-size preflight to support authenticated lookups and make bootstrap mode selection visible.
Changes:
- Adds authenticated requests with anonymous fallback on 401/403.
- Adds outcome logging, failure notices, timeout and response safeguards.
- Adds regression tests and updates documentation.
| File | Description |
|---|---|
unstable/client.go |
Documents size-lookup diagnostics. |
internal/strategy/bootstrap/github_preflight_test.go |
Tests authentication, fallback, logging and batching. |
internal/strategy/bootstrap/bootstrap.go |
Implements authenticated preflight and outcome reporting. |
docs/usage.md |
Explains lookup behavior and credential scope. |
client.go |
Updates logger documentation. |
CHANGELOG.md |
Records the preflight changes. |
Review effort: Balanced
|
That bump isn't carried in this PR yet. It changes the shared toolchain and lint pins, so I'm waiting on a decision: port it here, or let this PR go green once #123 lands and Generated by Claude Code |
|
Ported the Go 1.27.2 bump from #123 as 58e81ac (cherry-pick of 8484a5f). It no-ops once #123 lands. Verified locally on go1.27.2:
Generated by Claude Code |
58e81ac to
6e75338
Compare
|
Update: the Go 1.27.2 port (58e81ac) is no longer on this branch. 6e75338 was pushed on top of 068547f, so 58e81ac itself passed every check, Generated by Claude Code |
|
bugbot run |
pjbgf
left a comment
There was a problem hiding this comment.
One low-severity finding from a git-conformance, security and performance review. The rest of the credential scoping (redirect binding, helper origin, TLS-off, log sanitisation) checked out.
Generated by Claude Code
29f63a4 to
91e007c
Compare
|
bugbot run |
|
bugbot run |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit d0e7b78. Configure here.
|
No fix PR exists yet. Verified locally:
Proposed patch: --- a/go.mod
+++ b/go.mod
-toolchain go1.27.2
+toolchain go1.27.3
--- a/mise.toml
+++ b/mise.toml
-go = { version = '1.27.2', ...
+go = { version = '1.27.3', ...This can land on its own against Generated by Claude Code |
The size lookup that selects batched bootstrap for repositories over 1.5 GiB sent no credentials. GitHub answers 404 to an anonymous request for a private repository, so every private repository, whatever its size, was bootstrapped as a single pack, and nothing said so: every failure returned silently and the selection message needed verbose mode. Attach the source credentials to the lookup, which also moves public lookups off the anonymous rate limit, and log every lookup's outcome (HTTP status or error, reported size, chosen mode) through the bootstrap logger, at warning level when the size is unknown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUdEhv4HALJ9DCRUP7GzTw
- Treat github.com as GitHub only on the default port of its scheme, so credentials for a same-named service on another port never reach the GitHub API. - Retry once anonymously when the API refuses the token (401/403), so a public repository is never worse off than before credentials were sent. - Surface a failed lookup as a notice too: the CLI wires no structured logger, so it previously stayed silent without -v. - Treat a reported size of 0 as unknown (GitHub reports 0 until it has computed the size), bound the lookup with a timeout, and sanitize and truncate GitHub's error message before logging it. - Update the BootstrapLogger docs to cover the new record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUdEhv4HALJ9DCRUP7GzTw
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUdEhv4HALJ9DCRUP7GzTw
A token written into the source URL authenticates git through req.URL.User and never becomes HTTPConn.Auth, so the lookup stayed anonymous and a private repository still bootstrapped as one pack. Carry the userinfo password over as basic auth when the connection has no credentials of its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUdEhv4HALJ9DCRUP7GzTw
Helper credentials looked up for a host /info/refs redirected to are stored in HTTPConn.Auth while Endpoint() still reports the configured URL, so the size lookup could send one host's credentials to the GitHub API. Record the URL helper credentials were looked up for and expose EndpointAuth, which returns only credentials bound to the endpoint's own site; the preflight now uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUdEhv4HALJ9DCRUP7GzTw
A 401/403 on the authenticated GitHub size lookup was replaced by the anonymous retry's answer, so SSO enforcement or an exhausted rate limit on a private repository surfaced as a plain 404. Keep the refusal and report it in the log (refused_auth_error) and in the notice. Also: - treat a reported size of 0 as an answer: one-shot at INFO, no notice - send a token written as the URL username alone, as net/http does - note in sameSite's doc that EndpointAuth hands helper credentials to a sibling service on the endpoint's site Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019JjbQK1QFpSHS4yYkrWS5Y Entire-Checkpoint: 01M5AFW25APMXA2M8HD73DA7S7
Adversarial review of the GitHub size preflight found the credential policy split between gitproto.EndpointAuth and bootstrap's sourceCredentials, neither of which looked at where the request goes. HTTPConn.CredentialsFor(dest) replaces both. Each credential is measured from the URL it is bound to (EndpointURL for explicit and URL userinfo, the lookup URL for helper credentials), dest must be same-site with it, and with TLS verification off nothing crosses to another host. So: - helper credentials looked up for a redirect host no longer reach a sibling of that host; - credentials are not sent to the API with TLS verification off; - a URL username without a password is not sent (it may be an account name, which the API would only refuse); - the configured API base no longer receives credentials unless it is on the source's site. Also: - log "authenticated" from the request that produced the answer, so a redirect that stripped the header reads false; - strip C1 controls and bidi formatting from GitHub's message; - notice says the bootstrap "starts with" a single pack, and a failed lookup is no longer printed twice under -v; - one record name for the verbose log and PushLogger; - githubSizeLookup keeps the refused attempt as a nested answer; - the redirect test asserts the redirect target was reached; - docs: a public repo reached only through a credential helper is still looked up anonymously. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019JjbQK1QFpSHS4yYkrWS5Y Entire-Checkpoint: 01M5AHETQ72WQ2NWA9M65233AD
GitHubOwnerRepo passed dot segments through, so a source URL such as https://github.com/../user produced a lookup that path joining resolved to /user, carrying the source credentials to the authenticated-user endpoint. Accept only the characters GitHub allows in owner and repository names, and reject . and .. outright. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUdEhv4HALJ9DCRUP7GzTw
- Skip the lookup for wiki sources: the API has no repository to size. - Report authenticated from whichever headers the authorizer set, not only Authorization. - Drop the CLI flag name from the notice so it reads right for library callers too. - Shorten the CHANGELOG entry and the Examples paragraph; move the credential details into "Credentials and redirects". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015oS6W9qjGMVnrTz5r8EwDL Entire-Checkpoint: 01M5C9ZCW16R8DR2083464JTX7
- Read authenticated off Authorization only again: the redirect guard strips only that header, so implying support for custom auth headers invited a cross-site leak. - Treat a reported size of 0 as unknown: bootstrap runs only with source refs, so 0 means GitHub has not sized the repo yet. It now warns. - Report nothing when the run itself is cancelled. - Clean every error that reaches the log or notice, not only GitHub's message: transport errors carry server-authored text too. - Count a 401/403 as a refused token only when the request still carried it, so a redirect that stripped credentials is not retried or blamed. - Give each attempt its own timeout so a slow refusal cannot starve the anonymous retry. - CredentialsFor returns nil instead of panicking without an endpoint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015oS6W9qjGMVnrTz5r8EwDL Entire-Checkpoint: 01M5CAXBGAJY87C7716VXD6RX3
- Move the preflight to its own file, github_preflight.go. - Move the one-line message cleaner to sanitize.Line and document it in the package's threat model, instead of a second sanitizer in bootstrap. - Set authOrigin for explicit credentials at construction, so CredentialsFor reads one field; share helper-credential adoption in adoptHelperCreds. - Reuse the default-port rule via gitproto.UsesDefaultPort. - logGitHubPreflight branches once and logs with the chosen level. - Tests: no sleeps in the timeout test (assert each attempt's deadline), zero size joins the unusable-body table, shared captureNotices and redirectingAPI helpers, bearer auth through privateRepoAPI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015oS6W9qjGMVnrTz5r8EwDL Entire-Checkpoint: 01M5CBACJHJ3YBH1HH7VZT2BWT
A caller-supplied transport may return a Response with Request unset (net/http itself guards for it), and reading the final request's Authorization header then panicked the whole sync. The answer is still used; authentication is not claimed when its provenance is unknown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015oS6W9qjGMVnrTz5r8EwDL Entire-Checkpoint: 01M5CCCCANQRWX43W2HB2J0153
Reading Response.Request to learn whether the answer was authenticated fails with a caller-supplied transport that leaves it unset: the last fix then reported every answer as unauthenticated, and a refused token lost its anonymous retry. Record each redirected request's Authorization after the client's redirect policy has run instead, which needs nothing from the response. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015oS6W9qjGMVnrTz5r8EwDL Entire-Checkpoint: 01M5CD4YBTTCFDT178G8YNTDX1
mayCarry, EnsureAuthForService and tryHelperRetry each spelled out "TLS verification off and another host, so no credentials". A security rule should not be able to drift between copies; crossesHostUnverified now states it once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015oS6W9qjGMVnrTz5r8EwDL Entire-Checkpoint: 01M5D3RTAGH5944K7TTFQP0FGQ
- The notice names --target-max-pack-bytes again: CLI users are most of its readers, and library callers map it to TargetMaxPackBytes. - docs/usage.md says again how a reported size of 0 is treated. - The sanitize package doc separates raw C1 bytes, which UTF-8 terminals ignore, from UTF-8-encoded C1 characters, which Line strips. - TestLineKeepsMessageOnOneSafeLine no longer splits the Writer test from its doc comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015oS6W9qjGMVnrTz5r8EwDL Entire-Checkpoint: 01M5FEE4HNPGWQTD9XCADVA9ZC
d0e7b78 to
cf885d2
Compare


Problem
When bootstrapping from a
github.comsource without--target-max-pack-bytes, git-sync asks the GitHub REST API (GET /repos/{owner}/{repo}) for the repository size. It switches to 512 MiB batches when the size is 1.5 GiB or more. That lookup sent no credentials. For a private repository GitHub answers 404 to an anonymous caller, so every private repository bootstrapped as a single pack, whatever its size.Nothing reported this:
Public repositories were exposed too: anonymous calls share GitHub's 60 requests/hour limit per egress IP.
Changes
Authenticated lookup
user:token@in the source URL, or credentials the git credential helper supplied when the git server asked for them. A public repository never asks, so with only a helper its lookup stays anonymous. A URL username without a password is not sent: it may be an account name.HTTPConn.CredentialsFor(dest)decides which credentials may go to the API, checking the real destination. Each credential is measured from the URL it is bound to (the endpoint for explicit and URL credentials, the looked-up host for helper ones), the destination must be same-site with it, and with TLS verification off nothing crosses to another host. So helper credentials looked up for a redirect host stay with that host, and an API base on another site gets nothing.GitHubOwnerReponow treatsgithub.comas GitHub only on its scheme's default port. The connection's redirect guard still strips credentials from any hop that leaves the API's site, and the loggedauthenticatedreflects the request that actually produced the answer.Visible outcome
Options.BootstrapLogger. This does not need verbose mode.repository,authenticated,http_status,refused_auth_status,refused_auth_error,size_kborerror,threshold_kb,mode,target_max_pack_bytes.-v(it can still switch to batches if the target rejects the pack). The notice names the token refusal when there was one. Under-vthe failure is printed once, and the verbose log andBootstrapLoggeruse the same record name.Hardening
owner/repo.wiki) get no lookup: the API has nothing to size.Docs
docs/usage.mddocuments the lookup and its credential scope.BootstrapLoggerdoc comments andCHANGELOG.md.Tests
New file
internal/strategy/bootstrap/github_preflight_test.go, written test-first. It covers:Execute);user:token@URL credentials, and a bare URL username not being sent;CredentialsForininternal/gitproto/smarthttp_test.go: explicit, helper and URL credentials, helper credentials bound to a redirect host, and TLS verification off.The preflight tests reach a local TLS server as
https://api.github.com, so the destination check runs as in production.The new tests fail against the previous implementation.
go test ./...,go test -raceon the affected packages andgolangci-lint run(v2.11.4) all pass.Generated by Claude Code
Note
Medium Risk
Touches credential scoping for cross-host API calls and bootstrap batching decisions; mistakes could leak tokens or mis-size packs, but changes are heavily tested and fail closed on redirects/TLS-off.
Overview
Fixes batched bootstrap for large private GitHub repos by sending source credentials on the
GET /repos/{owner}/{repo}size preflight (scoped toapi.github.comvia newHTTPConn.CredentialsFor), with same-site/TLS rules aligned to redirect handling and helper creds bound toauthOrigin.The preflight moves into
github_preflight.gowith structuredBootstrapLogger/PushLoggerrecords, CLI notices on lookup failure, 401/403 → one anonymous retry, stricterGitHubOwnerRepoparsing, andsanitize.Linefor API/error text in logs. Docs and changelog describe the behavior.Reviewed by Cursor Bugbot for commit d0e7b78. Configure here.