Implement everything in Stanford/Northeastern's Shepherd — a runtime that makes a live agent run reversible — using Toolpath as the execution trace format.
This page is preserved as-committed so the plan can be judged against what got built. Two details drawn here were later revised by evidence from real runs (the observation transport and the sealing trigger); the as-built page tells that story. Everything else survived contact with reality.
When an agent runs long enough, its trajectory becomes the cheap part and its state becomes the expensive part: the edited files, the installed packages, the running processes, and the provider-side KV cache. When the agent goes wrong at step 10, everything through step 8 was already paid for — and a message log cannot give it back. Restarting re-pays every call; a human rescue does not scale; letting the agent flail burns context.
Shepherd (arXiv:2605.10913) answers by recording execution as a Git-like trace of typed events where every action is a commit, the agent and its environment fork together copy-on-write, and any past state is one checkout away. Reverting is ~5× faster than docker commit, and because the message prefix up to the fork is byte-identical, replay reuses ~95% of the provider's prompt cache. On CooperBench, a supervising meta-agent lifted pair-coding pass rates from 28.8% to 54.7%.
The bet of this plan: Toolpath already is that trace format — a DAG of typed steps with first-class dead ends, actors, and intent, plus projectors that can rebuild any harness's native session files. Shepherd built a private trace and a runtime; we have the open trace and need only the runtime. Building it earns something Shepherd cannot do: forks that cross harnesses, and traces that outlive the run on Pathbase.
Shepherd is four primitives and four demonstrated applications. The primitives: Task (agents as typed values a meta-agent holds and rewrites), Effect (every action a typed event, with intent and outcome separated so interception happens between them, and reversibility tiers for what cannot be undone), Scope (the agent and its environment forked atomically), and Trace (a persistent commit graph where revert is checkout and replay is byte-identical).
The applications, with the paper's numbers as our bars: a runtime supervisor (CooperBench 28.8% → 54.7%), counterfactual replay optimization (beats GEPA and MetaHarness on 4 of 5 datasets at 27–58% less wall-clock), Tree-GRPO (+5.2 points over flat GRPO on Terminal-Bench 2.0), and trajectory compression (68–82% of passing baselines rerun shorter with one hindsight hint). The paper's Lean-mechanized effect calculus is descoped as research garnish.
One reading of the paper's appendix shaped this whole plan: Shepherd's own headline experiment drove an external harness (opencode) over HTTP, with the agent loop on the host and only the sandbox forking. That is structurally identical to how arena already runs claude, codex, and gemini — so parity was never going to require exotic process checkpointing, just live observation, per-step snapshots, and session-truncation-plus-resume.
.trace, .inject(), .handoff(), .discard(), .scope.fork(at=step)base.uri = "toolpath:parent/step" was in the RFC before this project existed; projectors rebuild the native session prefix byte-for-byte, which is the prompt-cache mechanismThe audit that preceded this table found the format needed zero schema changes: the JSONL streaming spec, the fork-shaped base URI, live session watchers, and cross-harness projectors all existed — unexercised — in the toolpath workspace.
Two layers, split by where each language already lives. Rust owns the format and trace plumbing inside the toolpath workspace; Python owns orchestration in a new repo, with arena as a path dependency for sandbox images, tasks, and auth. The host process is the meta-agent's seat; the sandbox is the worker's world; four arrows connect them.
* The amber arrow is the one the build redrew: the plan assumed a host-side tail of the bind-mounted session file. Real runs replaced it with the watcher running inside the sandbox, streaming events over the docker exec stdout pipe — see the as-built page for why.
$ drover supervise --task pair-webhooks-ratelimit \ --workers claude:haiku,claude:haiku --meta claude:sonnet [00:05] worker-a 4 steps reading tests/ errors 0 [02:35] worker-b 31 steps editing src/webhooks.py errors 2 [02:35] meta inject → worker-b guidance: "src/webhooks.py belongs to another agent. Implement rate limiting only in src/ratelimit.py." [04:10] worker-a 22 steps attempted: rm -rf /work/tests intent DENIED [04:10] trace dead-end step recorded · decision: {status: denied} [06:48] worker-a verifier PASS worker-b verifier PASS [06:48] proposals 2 held diffs — review with drover select
$ drover compress --run 20260707T032005-gemini-default-impl-slugify-r2-a09e8423 reading trace 38 steps · verifier FAIL · 1 abandoned file meta-agent fork_step=2 hint "Implement slugify inside textutil.py, the module the tests import. Do not create a new file." forking scope snapshot @ s2 → fresh sandbox resuming prefix served from prompt cache rerun 12 steps · verifier PASS result COMPRESSED 38 → 12 · original attempt preserved as a dead end
The run ID in mock B was real — a Gemini failure from that morning's arena matrix, which implemented a correct-looking slugify in the wrong file and thrashed for 38 steps. The plan's first demo milestone was to make this mock come true.
A meta-agent reads a completed trace, picks a fork step, writes one hint; the rerun must pass the verifier in strictly fewer steps. Chosen first because it exercises the whole fork/replay spine with zero live-supervision complexity, against a corpus that already exists.
Paper: 68–82% of passing baselines compress; mean length roughly halves.
One meta-agent watches N parallel workers' live traces on a ~5-second tick and issues inject, handoff, or discard — the paper's Appendix E protocol, prompt included, adapted to arena pair tasks and then a CooperBench subset.
Paper: pair pass rate 28.8% → 54.7%, closing 91% of the coordination gap.
A proposer edits agent workflows and validates each edit by forking the parent trace at the first affected step and replaying only the suffix, with fix and guard sets gating candidates — harness self-repair grounded in exact replay.
Paper: beats GEPA and MetaHarness on 4 of 5 datasets, 27–58% less wall-clock.
During RL rollouts a meta-agent picks fork points; K sibling branches per point yield per-step advantages flat GRPO cannot see. Extends arena's existing world-model export, then a funded Flat-vs-Tree run on Prime Intellect.
Paper: +5.2 points over flat GRPO on Terminal-Bench 2.0 (Qwen3.5-35B-A3B).
| Phase | Size | Depends on | Proof it works |
|---|---|---|---|
| 1 · live trace | M | — | Live-derived document identical to the post-hoc import of the same run. |
| 2 · scopes | L | 1 | Post-revert workdir byte-identical; forked resume shows cache reads. |
| 3 · control | M | 1 | A destructive tool intent is denied, recorded as a dead-end step; run continues. |
| 4 · meta SDK | M | 2, 3 | Supervisor A/B uplift on arena pair tasks. |
| 5 · compression | S | 2 only | At least one real corpus run compressed: fewer steps, verifier passes. |
| 6 · CRO | M | 2, 5 | Optimized workflow beats the baseline prompt on held-out tasks. |
| 7 · Tree-GRPO | L | 2, budget | Tree beats flat GRPO on a held-out slice at matched compute. |
| 8 · benchmarks | M | rolling | A results table directly comparable to the paper's Tables 3, 8, 10. |
Fork a real failed arena run at a meta-agent-chosen step, resume it with one hint, and watch it pass the verifier in fewer steps — the whole episode one Toolpath graph whose dead end is the original attempt. That artifact is the pitch; everything after it is scale.
Orchestration follows Shepherd into Python because arena, the meta-agent's LLM calls, and prime-rl already live there; Rust keeps the format and trace plumbing.
Portable, unprivileged snapshots first; the paper-grade backend later for system-mutating tasks and the latency table. Prime Intellect's gVisor blocks OverlayFS anyway.
Observe-plus-handoff/discard until their streaming input paths are wired — the paper itself supervised exactly one harness.
Interception becomes a format-level fact rather than runtime-only state — elegant, and a semantic commitment the RFC carries forever.
A funded GRPO run plus full CooperBench replication is research-project territory; subsets gate everything, but the aim-point shapes the calendar.
~/code/empathic/drover; format work in the toolpath workspace; arena minimally touched.Explicitly out of scope from day one: the Lean proof envelope, Windows, the pi harness, an opencode driver, and automatic compensation for irreversible effects — v1 records what cannot be undone, and catching it before it fires is the supervisor's job.