Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ Protected tests:
hosted tier, so a repository that had not added its secret yet went green
while running on another vendor's model and CodeBoarding's money, silently.
Any change that reintroduces a credential fallback breaks this test.
Since 7 October 2026 (licensing spec D-16, Svilen's call), a workflow that
names no `llm` is resolved from the provider inputs that are set, and runs on
hosting when none is. That covers only workflows that named nothing; a named
provider still runs on that provider or fails, and the job summary says which
source an unnamed workflow got and why.

## Releases

Expand Down
51 changes: 27 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,7 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: CodeBoarding/CodeBoarding-action@v1
with:
llm: hosted # or a provider name -- see Authentication
- uses: CodeBoarding/CodeBoarding-action@v1 # CodeBoarding hosting; to use your own key, see Authentication
```

Automatic runs review both draft and non-draft pull requests and update one sticky **CodeBoarding review** comment. Opening, reopening, or pushing a commit runs analysis; changing only the draft state does not. A trusted repository owner, member, or collaborator can comment `/codeboarding` to analyze the current PR head again, including on fork PRs; every command creates a new result comment.
Expand Down Expand Up @@ -106,36 +104,41 @@ Every review comment ends with a machine-readable HTML comment, `<!-- codeboardi

## Authentication and providers

The `llm` input is required and says where analysis credentials come from. There are
two answers, and the action never picks one for you:
Wire the key you want to use, and the action uses it. Wire nothing, and it runs on
CodeBoarding hosting:

```yaml
with:
llm: hosted # CodeBoarding's hosted tier, on your plan
- uses: CodeBoarding/CodeBoarding-action@v1 # CodeBoarding hosting, on your plan
```
```yaml
with:
llm: anthropic # your own provider key
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
- uses: CodeBoarding/CodeBoarding-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} # your own key, called directly
```

`hosted` runs through CodeBoarding's proxy and needs `id-token: write`, which
The optional `llm` input names the source outright: `hosted` or a provider. You need it
only when inputs for more than one provider are set, or when you want a missing key to
fail the run rather than fall back to hosting.

Hosting runs through CodeBoarding's proxy and needs `id-token: write`, which
mints short-lived credentials per request and stores no LLM secret in your repository. A
provider key is used directly and needs no OIDC permission.

**An empty value is never a fallback.** If you name a provider and its key is missing --
because the secret does not exist yet, or is misspelt — the run fails in its first
seconds and says which input and which secret to fix. It does not quietly analyze on
CodeBoarding's hosted tier instead. That was the old behaviour, and it meant a repository
could report an Anthropic review that Anthropic never produced.

The same rule makes the combinations explicit rather than order-dependent:
**Check the first line of the job summary after adding a key.** GitHub reads a secret that
does not exist, or is misspelt, as an empty string, so a workflow without `llm` whose only
key is missing runs on hosting. The log's first line and the summary's "Chosen" row say
which source the run used and why. **A named provider never falls back:** with
`llm: anthropic` and the key missing, the run fails in its first seconds and says which
input and which secret to fix.

| Workflow says | Result |
|---|---|
| nothing | refused: `llm` is required |
| `llm: hosted` | CodeBoarding's hosted tier, on the plan of whoever the run is charged to |
| nothing | CodeBoarding hosting, on the plan of whoever the run is charged to |
| `anthropic_api_key` only | Anthropic, directly |
| `anthropic_api_key` + `openai_api_key` | refused: set `llm` to pick one |
| `llm: hosted` | CodeBoarding hosting |
| `llm: hosted` + any provider key | refused: pick one |
| hosting + `model`, `agent_model` or `parsing_model` | refused: hosting chooses its own models |
| `llm: license` | refused: license keys are retired, use `llm: hosted` |
| `llm: anthropic` + `anthropic_api_key` | Anthropic, directly |
| `llm: anthropic`, key empty or absent | refused: names the input and the secret |
Expand Down Expand Up @@ -253,6 +256,8 @@ Precedence is intentionally simple:

Set only `model` when both jobs should use the same model. Set either specialized input only when that job needs a different model. Model identifiers are not secrets and can be stored in GitHub repository variables.

On CodeBoarding hosting, CodeBoarding pays for the tokens and chooses the models. A hosting run that sets any of the three model inputs, or an `AGENT_MODEL` or `PARSING_MODEL` in the job's environment, is refused in its first seconds with the line to remove. A workflow without `llm` that sets a model but whose key secret is missing lands here too, so the refusal also says to check the secret. The model inputs apply when the run uses your own provider key.

## Keep the baseline current

Sync mode commits only Core's persisted incremental-analysis state under `.codeboarding/`:
Expand Down Expand Up @@ -303,7 +308,6 @@ jobs:
- uses: CodeBoarding/CodeBoarding-action@v1
with:
mode: sync
llm: hosted
target_branch: main
force_full: ${{ inputs.force_full || false }}
```
Expand All @@ -324,7 +328,6 @@ permissions:
- uses: CodeBoarding/CodeBoarding-action@v1
with:
mode: sync
llm: hosted
target_branch: main
sync_strategy: pull_request
```
Expand All @@ -338,11 +341,11 @@ With the default `github.token`, the repository or organization must allow GitHu
| Input | Mode | Default | Description |
|---|---|---|---|
| `mode` | both | `review` | `review` or `sync`. |
| `llm` | both | **required** | `hosted` or a provider name. No default. |
| `llm` | both | empty | `hosted` or a provider name. Empty: the provider inputs that are set decide. |
| `<provider>_api_key` | both | empty | That provider's key, e.g. `anthropic_api_key`. See [Providers](#providers). |
| `<provider>_base_url` | both | empty | That provider's endpoint, where it has one. |
| `aws_bedrock_region` | both | empty | Bedrock region. Core defaults to `us-east-1`. |
| `model` | both | empty | Default model for both analysis and parsing. |
| `model` | both | empty | Default model for both analysis and parsing. Own key only: a hosting run that sets it is refused. |
| `agent_model` | both | empty | Analysis-only override for `model`. |
| `parsing_model` | both | empty | Parsing-only override for `model`. |
| `depth_cap` | both | `2` | Positive integer maximum analysis depth, including full-analysis fallbacks, capped by the plan of whoever the run is charged to (3 on Free). Changing it rebuilds incompatible state. |
Expand Down
14 changes: 9 additions & 5 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@ inputs:
required: false
default: 'review'
llm:
description: 'Required. Where analysis credentials come from: hosted or a provider name (anthropic, aws_bedrock, cerebras, deepseek, glm, google, kimi, litellm, ollama, openai, openrouter, orcarouter, vercel).'
required: true
description: 'Optional. Where analysis credentials come from: hosted or a provider name (anthropic, aws_bedrock, cerebras, deepseek, glm, google, kimi, litellm, ollama, openai, openrouter, orcarouter, vercel). Without it, the provider inputs that are set decide: none means hosted, one provider means that provider. A named provider never falls back.'
required: false
default: ''
# Anthropic - selected by llm: anthropic
anthropic_api_key:
description: 'Anthropic API key, used when llm is anthropic. Sets ANTHROPIC_API_KEY.'
Expand Down Expand Up @@ -110,15 +111,15 @@ inputs:
required: false
default: ''
model:
description: 'Optional model used for both analysis and parsing.'
description: 'Optional model used for both analysis and parsing. Not allowed on CodeBoarding hosting, which chooses its own models: the run is refused.'
required: false
default: ''
agent_model:
description: 'Optional analysis-model override. Takes precedence over model.'
description: 'Optional analysis-model override. Takes precedence over model. Not allowed on CodeBoarding hosting.'
required: false
default: ''
parsing_model:
description: 'Optional parsing-model override. Takes precedence over model.'
description: 'Optional parsing-model override. Takes precedence over model. Not allowed on CodeBoarding hosting.'
required: false
default: ''
depth_cap:
Expand Down Expand Up @@ -254,6 +255,9 @@ runs:
env:
ACTION_PATH: ${{ github.action_path }}
CB_IN_LLM: ${{ inputs.llm }}
CB_IN_MODEL: ${{ inputs.model }}
CB_IN_AGENT_MODEL: ${{ inputs.agent_model }}
CB_IN_PARSING_MODEL: ${{ inputs.parsing_model }}
CB_IN_ANTHROPIC_API_KEY: ${{ inputs.anthropic_api_key }}
CB_IN_AWS_BEDROCK_API_KEY: ${{ inputs.aws_bedrock_api_key }}
CB_IN_AWS_BEDROCK_REGION: ${{ inputs.aws_bedrock_region }}
Expand Down
97 changes: 79 additions & 18 deletions scripts/action/credential_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@
that same string as the pull request comment, the error annotation and the job summary --
so the rule and its explanation are written together and cannot drift apart.

One provider, chosen explicitly by the `llm` input, and nothing is ever reached by an
empty string falling through to a default. A misconfigured run fails here -- before the
checkout and the engine install -- naming the input and the secret to fix, rather than
succeeding on someone else's credentials or failing later inside the engine.
One provider per run. A workflow that names it with `llm` gets that provider or a refusal,
never a default. A workflow without `llm` gets whatever its set inputs describe: none means
CodeBoarding hosting, one provider's inputs mean that provider, several are refused. A
misconfigured run fails here -- before the checkout and the engine install -- naming the
input and the secret to fix, rather than failing later inside the engine.

Reads the action's inputs from CB_IN_* environment variables (prefixed so that wiring an
input can never itself set a provider selection variable), and writes the resolved
Expand All @@ -31,6 +32,12 @@
SETTINGS_HINT = "Settings -> Secrets and variables -> Actions"
TABLE = Path(__file__).resolve().parent / "supported-providers.json"

#: The action's model inputs, and the variables the engine reads a model from. On hosting
#: CodeBoarding pays for the tokens, so it chooses the models (spec D-13): a run that names
#: one there is refused. They apply where the run's own provider key pays.
MODEL_INPUTS = ("model", "agent_model", "parsing_model")
MODEL_ENVS = ("AGENT_MODEL", "PARSING_MODEL")


#: Every reason this module can refuse a configuration.
#:
Expand All @@ -41,7 +48,8 @@
#: `test_llm_contract.py` asserts every entry here is exercised.
ERROR_CODES = frozenset(
{
"missing_llm",
"several_provider_keys",
"hosted_with_model",
"unknown_llm",
"missing_provider_key",
"missing_id_token",
Expand Down Expand Up @@ -172,20 +180,56 @@ def _reject_provider_inputs(table: dict, given: dict[str, str]) -> None:
)


def _require_id_token(llm: str, environ: dict[str, str]) -> None:
def _reject_model_choice(environ: dict[str, str], reason: str) -> None:
"""Hosting runs on CodeBoarding's models, so a run there that names one is refused.

Refused rather than ignored: a model named on a hosting run usually means the workflow
meant to use its own key and its secret is missing, and an analysis on a model nobody
asked for would hide that. The job environment counts too, since the engine reads it.
"""
chosen = [f"`{n}`" for n in MODEL_INPUTS if environ.get(f"CB_IN_{n.upper()}", "").strip()]
chosen += [f"`{v}` in the job's environment" for v in MODEL_ENVS if environ.get(v, "").strip()]
if not chosen:
return
what = ", ".join(chosen)
verb = "is" if len(chosen) == 1 else "are"
if reason:
message = (
f"{what} {verb} set, but this run uses CodeBoarding hosting because {reason}, and "
"hosting runs on CodeBoarding's models. If you meant to use your own provider key, "
f"check that its secret exists; otherwise remove {what}."
)
else:
message = (
f"`llm: hosted` runs on CodeBoarding's models, but {what} {verb} set. Remove {what}, "
"or set `llm` to your provider and wire its key to choose a model."
)
raise ConfigError("hosted_with_model", message)


def _require_id_token(llm: str, environ: dict[str, str], reason: str = "") -> None:
# Both, because the relay needs both (oidc_relay.py refuses to start without either)
# and a runner can expose one without the other. Checking only the URL let that case
# through preflight and turned it into a generic failure after the engine install,
# which is the whole thing this check exists to prevent.
if environ.get("ACTIONS_ID_TOKEN_REQUEST_URL") and environ.get("ACTIONS_ID_TOKEN_REQUEST_TOKEN"):
return
# A run that chose hosting because no key is set may have meant to use a key whose
# secret does not exist yet; this is the one refusal that can say so.
if reason:
problem = (
f"This run uses CodeBoarding hosting because {reason}. Hosting authenticates with a "
"GitHub OIDC token, which this job cannot mint. If you meant to use your own provider "
"key, check that its secret exists."
)
else:
problem = f"`llm: {llm}` authenticates with a GitHub OIDC token, which this job cannot mint."
raise ConfigError(
"missing_id_token",
f"`llm: {llm}` authenticates with a GitHub OIDC token, which this job cannot mint. "
"Add `permissions:` with `id-token: write` to the job that uses this action.",
f"{problem} Add `permissions:` with `id-token: write` to the job that uses this action.",
"\n\n".join(
[
f"`llm: {llm}` authenticates with a GitHub OIDC token, which this job cannot mint.",
problem,
f"In `{workflow_path(environ)}`, the job running this action needs:",
"```yaml\n permissions:\n id-token: write\n```",
"No secret is involved: the token is minted per request and never stored.",
Expand Down Expand Up @@ -246,18 +290,30 @@ def resolve(table: dict, environ: dict[str, str]) -> dict:
llm = environ.get("CB_IN_LLM", "").strip().lower()
given = read_inputs(table, environ)

# Without `llm`, the inputs that are set decide (spec D-16), and the plan carries the
# reason so the log and the summary say how the run chose. A named `llm` is never
# second-guessed: it takes the strict paths below exactly as written.
reason = ""
if not llm:
raise ConfigError(
"missing_llm",
"The `llm` input is required and has no default. Set it to `hosted` "
"(CodeBoarding's hosted tier, on your CodeBoarding plan) or one of: "
f"{_provider_list(table)}. See {DOCS}.",
)
providers = sorted({owner_of(table, i) for i in given})
if len(providers) > 1:
raise ConfigError(
"several_provider_keys",
f"Inputs for more than one provider are set ({', '.join(f'`{i}`' for i in sorted(given))}). "
f"Set `llm` to the one this run should use: {', '.join(providers)}.",
)
if providers:
llm = providers[0]
reason = f"only {table['providers'][llm].get('label', llm)}'s inputs are set"
else:
llm = "hosted"
reason = "no provider key is set"

if llm == "hosted":
_reject_provider_inputs(table, given)
_require_id_token(llm, environ)
return {"tier": "hosted", "provider": table["hosted_provider"], "env": {}}
_reject_model_choice(environ, reason)
_require_id_token(llm, environ, reason)
return {"tier": "hosted", "provider": table["hosted_provider"], "env": {}, "reason": reason}

# Keys were retired completely, so the answer that ran on one is refused by name rather
# than as an unknown value: the fix is one word, and the plan it paid for now follows
Expand Down Expand Up @@ -288,7 +344,7 @@ def resolve(table: dict, environ: dict[str, str]) -> dict:
)

env = _resolve_byok(table, name, given, environ)
return {"tier": "byok", "provider": name, "env": env}
return {"tier": "byok", "provider": name, "env": env, "reason": reason}


def _is_endpoint(var: str) -> bool:
Expand Down Expand Up @@ -352,6 +408,8 @@ def reported_provider(plan: dict) -> str:

def plan_headline(table: dict, plan: dict) -> str:
"""The one line the log opens with. Same source as the summary, so they cannot drift."""
if plan.get("reason"):
return f"CodeBoarding is running on {_pays(table, plan)}, because {plan['reason']}."
return f"CodeBoarding is running on {_pays(table, plan)}."


Expand All @@ -366,6 +424,9 @@ def plan_summary(table: dict, plan: dict) -> list[tuple[str, str]]:
if shown:
rows.append(("Provider", f"`{shown}`"))
rows.append(("Credentials", _pays(table, plan)))
# Without `llm`, a secret that does not exist reads as empty and the run goes hosted.
# This row is how someone who meant to use their own key finds out.
rows.append(("Chosen", f"because {plan['reason']}" if plan.get("reason") else "by `llm`"))
# Only where the run was pointed somewhere other than the default, since that is the
# setting most likely to be wrong and least likely to be noticed.
for var, value in sorted(plan["env"].items()):
Expand Down
Loading
Loading