Senior Product Manager for AI and data products in Toronto. 9+ years in product: agentic and RAG search, ML risk models in fintech, and the evaluation bars that keep AI features worth their cost. Open to PM and Senior PM roles on AI, search, data and platform products (Toronto or remote, Canada). Portfolio and résumé: seanmcrae.github.io.
Now: Product Manager, GenAI & ML at Upland Software (keyword search to agentic search). Previously: Technical PM, AI & ML at Canaccede Financial (ML debt-recovery ranking) · Lead Product Manager at CTO Bees Inc. (formerly Data Naya).
Each repo runs end to end without API keys, ships with tests and CI, and includes a product brief (docs/PRODUCT.md): problem, users, scope, success metrics, trade-offs, roadmap. Start with the first two.
| Project | Decision it supports | Proof | Links |
|---|---|---|---|
| ragbench | Which RAG or agentic-search config to fund, on quality, groundedness, latency and cost under a budget | The only significant gain over BM25 (agentic search) costs 2.9x as much per query | Docs · PRODUCT.md |
| llm-gateway | Which model tier each request deserves, within budget, cost cap and latency SLO | 65% cheaper than always-premium on a 300-prompt replay | Docs · PRODUCT.md |
| pocketbrains-app | How far a private, on-device iOS agent can go: multi-step plans with undo, cited Q&A over your notes | Notes Q&A: 96.7% recall@3, 100% citation faithfulness | Docs · PRODUCT.md |
| agent-harness | How much autonomy to give an agent: budgets, guardrails and human approvals, tested in CI | 19/19 scenarios pass with guardrails on; 15/19 with them off | Docs · PRODUCT.md |
| credit-risk-explain | Whether a risk model is fit to ship: calibration, reason codes, fairness slices, model card | 50.7% of defaulters caught in the riskiest 20% of the queue (real UCI data) | Docs · PRODUCT.md |
| workflow-radar | Which AI opportunities to fund first: measured friction, suitability, Monte Carlo ROI | Top quick win pays back in 1.2 months (P50) | Docs · PRODUCT.md |
| prd-to-backlog | Whether a PRD is buildable: a traceable, linted backlog for GitHub Issues, Jira or Linear | Every requirement traced to a story; lint scores 94 to 97 out of 100 | Docs · PRODUCT.md |
| plg-metrics | Whether an A/B result is real: SRM checks, CUPED, power, peeking guardrails | CUPED lift +2.31 pp against a true effect of +2.28 pp | Docs · PRODUCT.md |
Results come from each repo's bundled synthetic or public data; the repo READMEs give the setup and the caveats.
- Define success metrics and evals before building, and gate releases on them. ragbench success metrics · CI fails on a quality regression
- Prefer transparent models and explicit trade-offs over black boxes. credit-risk-explain model card · trade-offs considered
- Track cost to serve alongside quality. llm-gateway cost vs quality results
Built in public from October 2026 with AI-assisted coding; the product decisions, evals and trade-offs are mine.
- pocketbrains-app v0.3.0: honest generalization, tool trimming, RAG eval, docs site · 2026-10-08
- pocketbrains-app v0.2.0: multi-step agent, cited note Q&A, system integrations · 2026-10-08
- v0.1.0 of ragbench, llm-gateway, agent-harness, credit-risk-explain, workflow-radar, prd-to-backlog and plg-metrics · 2026-10-07