From c6b427735fcfba166c1e17799820767c5fe8c3ed Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 29 Sep 2026 22:50:23 +0000 Subject: [PATCH 1/5] =?UTF-8?q?Link=20Phase=203=20eval=20protocol=20to=20t?= =?UTF-8?q?he=20Nov=E2=80=93NYE=20Zenodo=20preprint=20plan.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Point operators at embabel-slm paper-calendar-2026 for venue, citations, and claim freeze while live rates stay measured in this repo. Co-authored-by: jmjava --- docs/evaluation-protocol.md | 8 +++++++- docs/roadmap.md | 5 +++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/evaluation-protocol.md b/docs/evaluation-protocol.md index 6a20b04..197043a 100644 --- a/docs/evaluation-protocol.md +++ b/docs/evaluation-protocol.md @@ -249,9 +249,15 @@ It still may not claim: ## Paper mapping -This protocol is the methods object for a short workshop paper on +This protocol is the methods object for a short workshop / preprint paper on **local SLMs as MCP tools**, not as a replacement model provider. Related MCP-agent benchmarks score tool choice or final answers on hosted servers. The question here is narrower: when a premium agent delegates a bounded coding edit to a private 9B–24B model, which failure layer fires, and how much of that layer is the prompt contract? + +**Nov–Dec 2026 ship plan** (venue, calendar, citations, claim freeze) lives in +the model program repo: +[embabel-slm/docs/paper-calendar-2026.md](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). +Zenodo is the primary DOI path; live rates for that preprint still come from +this repo’s harness and dated notes. diff --git a/docs/roadmap.md b/docs/roadmap.md index d9756fb..c29a64a 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -17,6 +17,11 @@ LAN addresses, or SKUs here. Finish Phase 3 measurement (layered live rates on the committed corpus) and the existing second-host checks before treating Halo as the next lab. +Paper track: Nov–Dec 2026 Zenodo preprint that uses this Phase 3 protocol as +methods — plan in +[embabel-slm paper calendar](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). +Live `pass@1` / `pass@end` and layer rates must be frozen before that DOI. + ## Phase 4 — Halo host profile (future) Use an AMD Ryzen AI Halo-class box as **another private Ollama host**. The From 46958ac8a2dd2c99e06f5721498ab1858220e584 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 29 Sep 2026 22:52:57 +0000 Subject: [PATCH 2/5] Point Phase 3 paper track at the Mar 2027 DOI deadline. Align eval-protocol and roadmap wording with the extended embabel-slm calendar. Co-authored-by: jmjava --- docs/evaluation-protocol.md | 4 ++-- docs/roadmap.md | 7 ++++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/evaluation-protocol.md b/docs/evaluation-protocol.md index 197043a..bf0b73d 100644 --- a/docs/evaluation-protocol.md +++ b/docs/evaluation-protocol.md @@ -256,8 +256,8 @@ hosted servers. The question here is narrower: when a premium agent delegates a bounded coding edit to a private 9B–24B model, which failure layer fires, and how much of that layer is the prompt contract? -**Nov–Dec 2026 ship plan** (venue, calendar, citations, claim freeze) lives in -the model program repo: +**Nov 2026 – Mar 2027 ship plan** (venue, calendar, citations, claim freeze; DOI by +**31 Mar 2027**) lives in the model program repo: [embabel-slm/docs/paper-calendar-2026.md](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). Zenodo is the primary DOI path; live rates for that preprint still come from this repo’s harness and dated notes. diff --git a/docs/roadmap.md b/docs/roadmap.md index c29a64a..b3f12c2 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -17,10 +17,11 @@ LAN addresses, or SKUs here. Finish Phase 3 measurement (layered live rates on the committed corpus) and the existing second-host checks before treating Halo as the next lab. -Paper track: Nov–Dec 2026 Zenodo preprint that uses this Phase 3 protocol as -methods — plan in +Paper track: Nov 2026–Mar 2027 Zenodo preprint (DOI by **31 Mar 2027**) that uses +this Phase 3 protocol as methods — plan in [embabel-slm paper calendar](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). -Live `pass@1` / `pass@end` and layer rates must be frozen before that DOI. +Live `pass@1` / `pass@end` and layer rates should freeze by early January; all +paper numbers freeze by late February. ## Phase 4 — Halo host profile (future) From ebc0bf0ab7adf6c21b561bc3948de773dc43e420 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 29 Sep 2026 22:54:05 +0000 Subject: [PATCH 3/5] Schedule Phase 4 Halo / 395 bring-up for Black Friday week Nov 2026. Keep Phase 3 live rates on the workstation until A13 passes. Co-authored-by: jmjava --- docs/roadmap.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index b3f12c2..ddadedc 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -17,16 +17,26 @@ LAN addresses, or SKUs here. Finish Phase 3 measurement (layered live rates on the committed corpus) and the existing second-host checks before treating Halo as the next lab. +**Halo / Ryzen AI Max+ 395:** planned acquire around **Black Friday week Nov 2026**. +Until then keep live measurement on the workstation Ollama host. Claim Phase 4 +only after the box is up (A13). Paper track coordinates this in +[embabel-slm paper calendar](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). + Paper track: Nov 2026–Mar 2027 Zenodo preprint (DOI by **31 Mar 2027**) that uses this Phase 3 protocol as methods — plan in [embabel-slm paper calendar](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). -Live `pass@1` / `pass@end` and layer rates should freeze by early January; all -paper numbers freeze by late February. +Live `pass@1` / `pass@end` and layer rates should freeze by early January (prefer +Halo once A13 is green; workstation is fine if the box slips); all paper numbers +freeze by late February. + +## Phase 4 — Halo host profile (planned ~Black Friday Nov 2026) -## Phase 4 — Halo host profile (future) +Use an AMD Ryzen AI Max+ 395 / Halo-class box as **another private Ollama host**. +Target arrival: **around Black Friday week November 2026**. The MCP server stays +on the workstation. Premium agents still plan and review. -Use an AMD Ryzen AI Halo-class box as **another private Ollama host**. The -MCP server stays on the workstation. Premium agents still plan and review. +Until the box exists and A13 passes, do not claim this phase — keep Phase 3 live +rates on the workstation host. This is **not** a second product. It is the same bridge with a different `OLLAMA_BASE_URL` (or the same loopback URL behind `ssh -L`). From 622d22f02214ea45ae3cf47445032740a3714524 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 29 Sep 2026 22:55:29 +0000 Subject: [PATCH 4/5] Plan early-Nov clear of downstairs T12 Part A before Halo. Prefer downstairs for live rates until the 395 passes A13. Co-authored-by: jmjava --- docs/roadmap.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index ddadedc..5ec9c55 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -11,23 +11,29 @@ LAN addresses, or SKUs here. | 1 — inference host | Done on the workstation lab | Ollama + starter tags on a private GPU | | 2 — MCP bridge | Done | `local-coding-slm` stdio tools; Cursor / Copilot / Claude adapters | | 3 — measure | Protocol + expanded corpus + harder multi-file suite + MCP apply gate + CI; live rates still informal | Layered scoring; scripted premium routing/review; no auto-classifier yet | -| T12 Part A second NVIDIA host | Blocked on host power | WSL GPU via SSH; see [examples/downstairs-wsl-gpu.md](../examples/downstairs-wsl-gpu.md) | +| T12 Part A second NVIDIA host | Planned clear early Nov 2026 | Downstairs WSL GPU via SSH — was blocked on host power; see [examples/downstairs-wsl-gpu.md](../examples/downstairs-wsl-gpu.md) | | T12 Part B Copilot A8 / Claude A9 | Config ready; operator clicks pending | Same-machine only; [docs/a8-a9-operator-checklist.md](a8-a9-operator-checklist.md) | Finish Phase 3 measurement (layered live rates on the committed corpus) -and the existing second-host checks before treating Halo as the next lab. +and clear the downstairs second-host path (T12 Part A) before treating Halo +as the only next lab. + +**Downstairs (second PC / WSL NVIDIA):** power on and finish SSH forward in +**early Nov 2026** so Part A live rates are not stuck on the workstation while +waiting for the 395. Public-safe notes: +[examples/downstairs-wsl-gpu.md](../examples/downstairs-wsl-gpu.md). **Halo / Ryzen AI Max+ 395:** planned acquire around **Black Friday week Nov 2026**. -Until then keep live measurement on the workstation Ollama host. Claim Phase 4 -only after the box is up (A13). Paper track coordinates this in +Until A13 passes, prefer downstairs for live measurement. Claim Phase 4 only +after the box is up. Paper track coordinates this in [embabel-slm paper calendar](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). Paper track: Nov 2026–Mar 2027 Zenodo preprint (DOI by **31 Mar 2027**) that uses this Phase 3 protocol as methods — plan in [embabel-slm paper calendar](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). Live `pass@1` / `pass@end` and layer rates should freeze by early January (prefer -Halo once A13 is green; workstation is fine if the box slips); all paper numbers -freeze by late February. +Halo once A13 is green; **downstairs** if the box slips; workstation last); +all paper numbers freeze by late February. ## Phase 4 — Halo host profile (planned ~Black Friday Nov 2026) From 03b7f3b4a79284649407fe859754491ba01a8058 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 29 Sep 2026 22:56:56 +0000 Subject: [PATCH 5/5] Mark downstairs T12 Part A as next hardware work before Halo. Phase 3 live rates default to downstairs; do not skip it to chase the 395. Co-authored-by: jmjava --- docs/roadmap.md | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 5ec9c55..589e2ee 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -10,30 +10,26 @@ LAN addresses, or SKUs here. | --- | --- | --- | | 1 — inference host | Done on the workstation lab | Ollama + starter tags on a private GPU | | 2 — MCP bridge | Done | `local-coding-slm` stdio tools; Cursor / Copilot / Claude adapters | -| 3 — measure | Protocol + expanded corpus + harder multi-file suite + MCP apply gate + CI; live rates still informal | Layered scoring; scripted premium routing/review; no auto-classifier yet | -| T12 Part A second NVIDIA host | Planned clear early Nov 2026 | Downstairs WSL GPU via SSH — was blocked on host power; see [examples/downstairs-wsl-gpu.md](../examples/downstairs-wsl-gpu.md) | -| T12 Part B Copilot A8 / Claude A9 | Config ready; operator clicks pending | Same-machine only; [docs/a8-a9-operator-checklist.md](a8-a9-operator-checklist.md) | +| 3 — measure | Protocol + corpus + apply gate + CI; **live rates still need a stable remote GPU** | Layered scoring; next live rows go through downstairs | +| **T12 Part A downstairs** | **Next hardware work** (early Nov 2026) | Second PC / WSL NVIDIA via SSH — was blocked on host power; [examples/downstairs-wsl-gpu.md](../examples/downstairs-wsl-gpu.md) | +| T12 Part B Copilot A8 / Claude A9 | Config ready; operator clicks pending | Same-machine only; do not block on downstairs; [docs/a8-a9-operator-checklist.md](a8-a9-operator-checklist.md) | -Finish Phase 3 measurement (layered live rates on the committed corpus) -and clear the downstairs second-host path (T12 Part A) before treating Halo -as the only next lab. +**Order (do not invert):** finish Phase 3 live measurement on **downstairs** (T12 Part A) **before** treating Halo as the next lab. -**Downstairs (second PC / WSL NVIDIA):** power on and finish SSH forward in -**early Nov 2026** so Part A live rates are not stuck on the workstation while -waiting for the 395. Public-safe notes: +1. **First work — downstairs:** power on, WSL Ollama, `ssh -L` from the IDE workstation, A4-class check, scrubbed live harness / `run_eval.py --live` rows. +2. **Then — keep measuring** Part A on downstairs through November. +3. **Later — Halo / 395** (~Black Friday): bring-up and A13 only after downstairs is a known path (on, or explicitly abandoned). + +Public-safe downstairs notes: [examples/downstairs-wsl-gpu.md](../examples/downstairs-wsl-gpu.md). **Halo / Ryzen AI Max+ 395:** planned acquire around **Black Friday week Nov 2026**. -Until A13 passes, prefer downstairs for live measurement. Claim Phase 4 only -after the box is up. Paper track coordinates this in +Claim Phase 4 only after A13. Until then Phase 3 live rates stay on **downstairs**, not workstation-local Ollama and not Halo. Paper track: [embabel-slm paper calendar](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). -Paper track: Nov 2026–Mar 2027 Zenodo preprint (DOI by **31 Mar 2027**) that uses -this Phase 3 protocol as methods — plan in -[embabel-slm paper calendar](https://github.com/jmjava/embabel-slm/blob/main/docs/paper-calendar-2026.md). -Live `pass@1` / `pass@end` and layer rates should freeze by early January (prefer -Halo once A13 is green; **downstairs** if the box slips; workstation last); -all paper numbers freeze by late February. +Paper track: Nov 2026–Mar 2027 Zenodo preprint (DOI by **31 Mar 2027**). Live +`pass@1` / `pass@end` freeze by early January (**downstairs** default; Halo only if +A13 green); all paper numbers freeze by late February. ## Phase 4 — Halo host profile (planned ~Black Friday Nov 2026) @@ -42,7 +38,7 @@ Target arrival: **around Black Friday week November 2026**. The MCP server stays on the workstation. Premium agents still plan and review. Until the box exists and A13 passes, do not claim this phase — keep Phase 3 live -rates on the workstation host. +rates on the **downstairs** host (T12 Part A). Do not skip downstairs to chase Halo. This is **not** a second product. It is the same bridge with a different `OLLAMA_BASE_URL` (or the same loopback URL behind `ssh -L`).