AgentMeasure

Verify AI service charges and delivery with evidence your team can check.

AgentMeasure helps AI service buyers connect applicable billing rules with event records and business requirements. Start with a specific AI support bill or an upcoming acceptance or renewal decision.

Today, the open-source CLI re-counts supported billing exports, organizes findings and records follow-up outcomes. Broader buyer review services are being validated case by case; this is not a universal contract or quality checker. Local CLI commands do not upload data.

Your task	Start here
A specific AI support billing question	Describe the task and check fit, or run the free recount below
An upcoming acceptance or renewal decision	Discuss the decision and evidence needed; project scope is exploratory
A local log anomaly or one metering sample	Free measurement runs — bring a sanitized sample, we run the check
Contribute a rule, counterexample or adapter	Maintainer resources

Website · 中文 · Healthcheck guide · Release notes

Two questions, different next steps

• Does the charge follow the applicable agreement? A supported difference can inform a vendor review. A finding is not an accepted claim or a refund.
• Does delivery meet your business requirements? This can inform acceptance, remediation or renewal. A stricter buyer requirement does not by itself make an existing charge incorrect.

Support operations supply the business facts; finance or procurement owns the spend decision. Missing evidence stays unknown. We separate findings in both directions, evidence gaps, vendor acceptance and actual refunds or used credits.

Since 2026-10-21 the project is entirely free and open source — the paid diagnostics and the commercial fee explorations have been retired. Free measurement runs do not cover enterprise invoice or delivery reviews.

Read the buyer scope and commercial and open-source boundaries.

Re-count a bill in 60 seconds

Save this five-line sample as export.csv (synthetic data, real command):

conversation_id,human_agent_participated,issue_addressed,customer_recontacted_within_window,vendor_billed
c-001,N,Y,N,Y
c-002,N,Y,N,Y
c-003,N,Y,N,Y
c-004,Y,Y,N,Y
c-005,N,Y,Y,Y

pip install agentmeasure
agentmeasure recount --export export.csv --vendor zendesk

The output prints the vendor's rule with its public source, findings in both directions, then the dollars:

Dollars (at USD 2.0 per resolution)
overcharge identified       2.00
undercharge (vendor favour) 0.00
net variance                2.00

No CSV at hand? agentmeasure recount --sample runs the same bundled synthetic export in CLI 0.8.0. In this synthetic example, four rows remain unsettled; $8 is not a verified billable total. The $2 is a rule-priced finding, not recovered money.

Three reproducible showcase cases (overcharge with refused guesses, findings in both directions, the contract lane) live in examples/showcase with frozen expected outputs that CI compares on every change.

Repeat a billing review

For local-only handling, run the CLI on each new period's export. The optional GitHub Actions workflow requires a separate private repository and uploads data and reports to GitHub. Public forks are not private storage. Each period needs a fresh, reviewed export; no vendor data is fetched automatically.

Six vendor rule sets ship today — Intercom Fin, Zendesk AI Agents, HubSpot Breeze, Salesforce Agentforce, Ada and a generic per-resolution baseline — each citing its source and confidence grade (agentmeasure recount --list-vendors). When a recount disagrees with the invoice: agentmeasure prepare maps a native Intercom or Zendesk export onto the canonical columns, agentmeasure dispute builds the Dispute & Recovery Pack, and agentmeasure recovery records what the vendor actually conceded, counted separately from the claim.

A recount is arithmetic, not a verdict: rows without enough evidence land in cannot settle and are removed from the claim, never zeroed. "No finding" is a valid outcome. Vendor recounting is a free CLI capability, not one of the two paid services below.

Start with your own machine (free)

agentmeasure check inspects your local Codex / Claude Code sessions for duplicate records, retry amplification and token anomalies — the same evidence discipline pointed at your own logs. agentmeasure demo runs a bundled synthetic session with no account and no upload. Missing evidence stays unknown; a retry is not automatically a second logical operation.

Python 3.9+. CLI 0.8.0 on PyPI; Provider SDK 0.2.0 on npm, verified 2026-10-15. These packages have separate version numbers. On macOS/Linux use python3 -m pip if needed; on Windows use py -m pip. For permission or “externally managed” errors, use the virtual-environment instructions. With both runtimes present, auto-discovery prefers Codex; use agentmeasure check --runtime claude for Claude Code. No logs in the default seven-day window? Try --all or an explicit --dir and --runtime. A missing input is not a passing check. More recovery steps.

Compare Claude response evidence across files

agentmeasure response-audit first-session.jsonl fork-session.jsonl --json responses.json

This read-only command groups supplied response IDs and shows repeated usage snapshots, conflicts and missing fields. It does not rewrite logs, choose a winning snapshot or compute a session/invoice total. See the response-audit guide.

Free measurement runs

The tools are free and local. If you want a human-run check on your own data, that is free too: bring 20–100 sanitized rows (or a public export) and we run a measurement check, usually within 48 hours of a clean sample. Findings are stated as checkable claims; where your surface cannot prove something, the report says UNPROVABLE instead of guessing.

What we ask — never required: permission to publish the fixture under your handle with the claim boundary you state (nothing customer-identifying; hash or drop identifiers), and if the check was useful, a star on the repo — never a condition.

• Start in issue #6 (public, so the next reader learns from your run), or email tongroy18@gmail.com
• Guide: [docs/provider-onboarding.md]
• Run the same review yourself: docs/review-method/ — intake checks, mapping, commands and report templates behind every free run(docs/provider-onboarding.md) — three entry paths, five steps, and what counts as a successful run

The previously offered $10/$20 paid diagnostics were removed on 2026-10-21; the project is now entirely free and open source.

To prepare a summary locally:

agentmeasure check --json report.json
agentmeasure share report.json --out summary.json
agentmeasure validate summary.json

--out writes immediately on your machine. Inspect the file before you choose to send it. Nothing is automatically uploaded. "No finding" and "not enough evidence" are valid outcomes. Tool bugs can be reported in Issues, with a redacted reproduction.

For tool maintainers and providers

Resource	Use it for
Conformance pack	Check fixtures and reported metrics; PASS / FAIL / UNPROVABLE
Provider SDK	Capture canonical observations from MCP tool handlers: npm install @agentmeasure/mcp
Core specification and AMS-1	Define measurement grains and the evidence needed for claims
Claims index	Inspect dated findings, reproductions and upstream responses
Lab	Existing experimental tooling; no hosted product commitment

The buyer tools above (recount, prepare, dispute, recovery) share one end-to-end walkthrough. They are free CLI capabilities, not one of the two current fixed-price services. Current development priorities are in the roadmap.

Evidence and limits

The September audit report documents the 110-tool review. Individual findings and subsequent corrections are linked in the claims index. Engineering evidence does not establish paid demand, customer savings or a right to a refund. Customer value and repeat purchase remain under validation.

511 Healthcheck unit tests passed on 2026-10-15. See CI for current results and v0.8.0 for the release scope. Example fixtures are not production customer data.

Open-source commitments

• MIT-licensed tools and public rules remain free. Paying does not change a verdict.
• Analysis runs locally. Sharing a sanitized sample is your explicit choice.
• No paid rankings or recommendation slots; no payment custody.

Contributing · Security · Governance · Agent instructions · Whitepaper

Task scope and validation priorities · Developer guide

Why optional paid help?

The public cases and free explanations stay free. A paid review is agreed human work on a remaining decision, not a secret scanner or automatic certification. If existing output answers the question, we recommend self-service. See the paid review scope.