# The workflow manual: how I build software with agent fleets > The complete end-to-end process — docs-first scaffolding, research, one reviewed plan, > decomposition as concurrency control, fleet deployment, and the failure modes behind all of it. - Canonical: https://jedarden.com/guides/workflow/ - Last verified: 2026-07-30 (oldest section review date) - Sections: 14 --- ## What this is **This is the path. Follow these seven stages and you can build serious applications with fleets of agents working in parallel.** Not one chat session — headless workers pulling from a queue of small, well-specified tasks while you do something else. The process is repeatable: starting a new project is mechanical once you have run it twice. It is written as a manual: each stage tells you what to produce, what "done" looks like, and what it hands to the next stage. Work through it in order and the result is a project that many agents can build at once without colliding — which is the whole game, because a fleet is only as useful as the work you can safely hand it. The stages are load-bearing in sequence. Each one produces the artifact the next one consumes, so the order is not a suggestion — a stage skipped is context the downstream workers never receive. ## How to use this Each stage ends with a **gate**: a short list of conditions you check before moving on. They are written to be evaluated, not judged — every item is something you can confirm by looking at a file or running a command. If an item fails, the gate says where to go back to. Work one stage at a time and clear its gate before starting the next. The gates are where this stops being reading and becomes a procedure — and they are also what makes it recoverable, because a project that goes wrong later can be traced to a gate that was passed on optimism. **Note.** The gates get stricter as you go. Stage 02's is a file checklist; stage 06's decides whether your fleet can run wide at all. That is intentional — the cost of being wrong rises the closer you get to dispatching work. **Who this is for.** Anyone running more than one agent at a time, or about to. Most of it also applies to a single interactive session — [section 13](#s13-steal-this) is the short version for that case. ## The stack The specific tools matter less than the shape, and the shape transfers to any agent CLI, any tracker with atomic claims, any dispatcher. But vague guides are useless, so here is exactly what is running:
Hardware and versions this manual describes | Layer | What I run | | --- | --- | | Host | One bare-metal server, accessed only over a mesh VPN — no public ports | | Agent harness | Claude Code, plus adapters for other CLIs | | Tracker | `bead-forge` (`bf`) — repo-local, SQLite live store, JSONL checkpoint | | Orchestrator | NEEDLE — Rust, agent-CLI-agnostic, deterministic outcome handling | | Fleet substrate | `tmux` sessions, one supervisor per worker | | CI | Argo Workflows on a small Kubernetes cluster | | Source hosting | Self-hosted Forgejo, mirrored read-only to GitHub | The tracker and the orchestrator are both mine. That is not a recommendation to write your own — it is a disclosure, because several claims in this manual are about how those two behave, and you should know the author of the claim is also the author of the code. ## What it is not **Caveat.** This is one estate's implementation, not a canonical methodology. Parts of it were deliberately adopted from other people's published workflows, parts were arrived at independently, and parts are convergent — the same conclusion other builders reached separately. [Section 12](#s12-lineage) sorts them, because blurring that line would be a claim I have not earned. It is also not a guide to getting good output from a single prompt. Everything here assumes the bottleneck is *coordination* — keeping many agents pointed at correct, non-overlapping work — rather than the quality of any individual response. ## What it costs Running a fleet is not free, and the dominant cost is not the one people plan for. It is not the volume of successful work — that is predictable and cheap per task. It is a single task that fails repeatedly and gets re-dispatched, which can consume more than the feature it was meant to build. Set a failure threshold before your first run; [section 11](#s11-failure-modes) covers the mechanics. The main lever in the other direction is routing by difficulty: routine, well-specified tasks go to a cheap model tier, and only hard ones — or anything that already failed once — go to a stronger one. In a well-decomposed queue most tasks are routine by construction, so most dispatches should be cheap. _[interactive exhibit: gate-map — see the web version]_ --- ## The whole loop Seven stages, in order: 1. **Scaffold** the repo around documentation 2. **Research** before planning 3. Write **one plan file** 4. **Review** the plan before decomposing it 5. **Decompose** the plan into a dependency-aware work queue 6. **Run** the fleet 7. **Refine** — feed what happened back into the plan Work them in order. Each stage consumes the artifact the previous one emitted, and an agent's context on any given dispatch is assembled from those artifacts — so the sequence is what builds up the thing your workers actually read. ## Artifacts, not conversation The most useful reframe I can offer is this: **nothing is passed between stages as conversation.** Every handoff is a file committed to the repository. That sounds like bureaucracy until you run twenty workers. A worker starting a task has no memory of the session that planned it, no access to the chat where you reasoned about the architecture, and no way to ask a clarifying question that anyone will answer within its lifetime. What it has is the repository. So the repository has to contain the reasoning. **Insight.** The quality of what a fleet writes is bounded by the quality of what it can read. Every stage before "run the fleet" is really one activity: raising that ceiling. ## What each stage gives you **Scaffold** creates the tree agents read before they write, and gives the first research note somewhere to live. Do it before any code. **Research** turns the things you would otherwise guess into written answers the plan can cite. It is also where your acceptance criteria come from. **One plan file** gives every worker the same ground truth, reloaded on every dispatch. One file means there is never a question about which document wins. **Review** is the highest-leverage hour in the process. A queue multiplies plan quality, so an hour spent here is repaid once per task — and the exit condition is concrete: zero undeliverable tasks. **Decompose** produces what agents actually execute, and — the least obvious idea in this manual — sets how much of the work can happen at once. That is [section 06](#s06-decomposition-is-concurrency-control), and it is the stage worth reading twice. **Run** is deliberately the most boring stage: claim, dispatch, handle the outcome by a fixed table. Boring is the goal; determinism is what lets you debug a fleet instead of interrogating it. **Refine** feeds what the fleet learned back into the plan, and returns you to stage 3. This is what keeps the queue describing the system you are actually building. ## What a worker actually receives It is worth being concrete about the thing every stage is feeding, because the abstraction "the agent has context" hides the mechanism. On each dispatch, a worker gets exactly three things: 1. **The task specification** — context, design, acceptance criteria, and the files it owns. 2. **The repository**, at whatever state the working tree is in. 3. **The standing instructions** — the agent-instructions file at the repo root. That is all. No conversation history. No memory of the session that wrote the plan. No access to the reasoning that produced the architecture. If a decision is not in one of those three places, it does not exist as far as the worker is concerned, and the worker will make it again — differently. **Insight.** Everything upstream of dispatch is really one activity: getting decisions out of your head and into one of those three channels. That reframing makes the whole process feel less like documentation discipline and more like what it is — assembling the input to a function you are about to call a hundred times. ## A diagnostic, if you inherit a project mid-flight Not everything arrives having followed the path. If a project is misbehaving, the symptom usually names the stage that was skipped — which tells you where to restart rather than what to debug: | Symptom | Restart at | | --- | --- | | Agents inventing file layouts; `utils/` and `lib/` for the same thing | Scaffold | | A plan of confident assertions, none marked as assumptions | Research | | Workers reasoning from different subsets; irreconcilable commits | One plan file | | Clean, well-formed, confidently wrong work | Review | | Timeouts, or agents overwriting each other | Decomposition | | You can see what the fleet did but not why it was allowed | Outcome handling | | A queue describing a system nobody is building any more | Refinement | The fourth row is the one to internalise. Every other skip produces visible mess you will notice quickly. Skipping review produces work that looks entirely fine until you read it closely, which is why the path puts a gate there. ## The loop is a loop Stage 7 feeds stage 3, not stage 1. You scaffold once and research mostly at the start, but plan → review → decompose → run → refine is a cycle you stay in for the life of the project. Projects that treat the plan as a document written once and then abandoned end up with a queue full of tasks that describe a system nobody is building any more. The plan is a maintained artifact, and maintaining it is a stage, not an afterthought. _[interactive exhibit: arc — see the web version]_ --- ## Repo scaffold — docs are the interface Every repository starts with the same tree, created before any code: ``` / ├── README.md ← purpose and contents └── docs/ ├── notes/ ← features, constraints, decisions unique to this app ├── research/ ← third-party research and source material └── plan/ └── plan.md ← one file: the entire application plan ``` Four directories and two files. Create it before anything else, including on projects you expect to throw away. ## Why docs-first is not ceremony For a human team, documentation is a nice-to-have that competes with shipping. For a fleet, **documentation is the interface**. It is the only channel through which intent reaches a worker. Consider what a worker actually receives on dispatch: a task specification, and whatever the repository contains. It does not receive your reasoning, your architectural preferences, the constraint you mentioned in passing, or the approach you already tried and rejected. If those live only in your head, every worker rediscovers them — differently, and usually wrongly. **Insight.** The scaffold is not documentation *about* the project. It is the project's working memory, and the fleet's only access to it. ## The two notes directories A second notes tree appears at the repository root once agents start working: ``` / ├── docs/notes/ ← curated. Human-shaped, topical, durable. └── notes/ ← generated. One file per task, written by agents. ``` This looks like a mistake and is not one. They hold different things and have different lifespans. `docs/notes/` is knowledge you would hand a new contributor: how this component is meant to work, why the naming is what it is, which constraint is non-negotiable and why. It is curated, edited, and kept true. `notes/` fills with per-task work logs the agents write themselves — `notes/.md`, one per unit of work. It is an execution trail: what the worker understood, what it tried, what it found. Nobody edits it. Let it happen. The trail is genuinely useful when you are reconstructing why an agent did something six weeks ago, and it costs nothing to keep. What matters is that the two trees stay distinct, because the moment curated knowledge and execution logs mix, neither can be trusted. ## The agent-instructions file Alongside the tree, every repo carries an agent-instructions file at the root — `AGENTS.md` or `CLAUDE.md` depending on the harness. This is not the plan. It is the standing rules: how to build, how to test, what the commit conventions are, which commands are forbidden, where the tracker lives. The distinction matters. The plan changes as the project progresses; the standing rules mostly do not. Mixing them means workers re-read a large document to find a small invariant, and invariants that are hard to find are invariants that get violated. **Critical.** Rules in this file must be stated as absolutes, not preferences. "Prefer X" is read by an agent as permission to do Y with a justification. "Never Y — doing so is a failed task" is read as a constraint. The difference in compliance is dramatic and it is entirely about phrasing. ## What goes in the instructions file Concretely, the sections that earn their place: - **Build and test commands.** The exact invocation. Not "run the tests" — the command, including any flags that matter. An agent that guesses will guess wrong on the first repo with a workspace layout. - **Commit conventions.** Style, and any required trailer. If commits must carry a task ID, say so here; it is how work gets traced back later. - **Never-rules.** Never force-push. Never commit build artifacts. Never edit the generated file. Each stated as a prohibition with the consequence attached. - **Where the tracker lives**, and how to claim, comment, and close. - **What to do when blocked** — stop, release, comment. Without this, a blocked agent improvises, and improvisation is where scope creep comes from. **Note.** Keep it short. This file is read on every dispatch, which means every unnecessary paragraph is a tax you pay hundreds of times. If something is only relevant to one part of the project, it belongs in the plan or in the task, not here. ## The README is for orientation, not description The README's job in an agent-run repo is different from its job in a human-run one. It is not a feature list. It is the two paragraphs that let a worker — or you, months later — decide whether the file it is about to edit is even in the right project. Purpose, shape, and where the plan lives. That is enough. Everything else drifts, and a drifted README is worse than a short one because agents believe it. Nothing in this stage requires the code to exist yet. That is the point: the scaffold is what makes it possible for something other than you to write it. _[interactive exhibit: scaffold-tree — see the web version]_ ### Gate — ready for 03 — Research? - The four directories exist: `docs/notes/`, `docs/research/`, `docs/plan/`, and the repo root. - `README.md` states the project's purpose in two paragraphs or fewer. - An agent-instructions file exists at the repo root. - That file names the exact build command and the exact test command — copy-pasteable, not described. - It states the commit convention, including any required trailer. - It lists the never-rules as prohibitions, not preferences. - It states what an agent must do when blocked: stop, release, comment. - The tracker is initialised in the repo. --- ## Research before planning `docs/research/` holds one markdown file per source or per question: an API's real behaviour, a protocol specification, prior art, a paper, a benchmark you ran yourself. The discipline is answering **"what do I not know yet?"** in writing, before the plan commits to answers. A plan written without this stage does not contain fewer assumptions — it contains the same assumptions, unlabelled, indistinguishable from the parts you actually verified. ## Calibrate to risk, not to thoroughness Research depth should track how badly a wrong assumption would hurt. A project wrapping a well-understood REST API needs perhaps three notes. A binary-format decoder — dense specification, a reference implementation as the only ground truth, and failure modes that produce plausible-looking garbage rather than errors — will want five times that. The multiplier is not project size; it is how quietly a wrong assumption fails. **Note.** The test is not "have I researched enough?" It is "which decisions in the plan am I about to make from memory?" Every one of those is a research file you have not written. ## A template that covers most projects For a project of any complexity, these seven questions produce a research set that a plan can be written from. Adapt the nouns; the shape holds: - **Prior art** — what already exists in this space, and what it gets right. - **The thing you are compatible with** — its data model and its contract. - **Storage or format constraints** — what the persisted shape has to survive. - **Behaviour at scale** — how this changes when there are many of whatever the unit is. - **The classic problem in this domain** — the failure mode practitioners already know about, and the standard solutions. This is the highest-value note in the set. - **The over-engineering check** — what happens if you build something simpler. - **What must not break** — existing callers, scripts, or data. The fifth is worth singling out. Nearly every domain has a well-documented failure that is obvious in hindsight and invisible in advance — the thundering herd for work queues, the ABA problem for lock-free structures, clock skew for distributed ordering. Reading about yours before you plan is the difference between designing around it and discovering it under load. ## What a research note contains Keep them short and factual: - **The question.** One line, at the top. - **What I found.** Facts, with sources — a link, a spec section, a command you ran and its output. - **What this means for the design.** One or two sentences. This is the part the plan will cite. - **What is still unknown.** Explicitly. This becomes an open question in the plan rather than a silent assumption. Do not write conclusions you have not earned. A research note that says "unclear — needs a spike" is more useful than one that guesses confidently, because the plan can schedule a spike but cannot detect a guess. ## Four kinds, and they are not interchangeable Most research files fall into one of four shapes, and knowing which you are writing tells you when it is finished: - **Behavioural** — what a dependency actually does, as opposed to what its documentation claims. Finished when you have run it and pasted the output. - **Specification** — the parts of a format or protocol this project touches. Finished when every field you will implement is described. - **Prior art** — what already exists and why you are not using it. Finished when you can state the specific reason, not a vibe. "Too heavy" is not a reason; "requires a server process, and this must run from a git checkout" is. - **Measurement** — a number you took yourself. Finished when the number and the command that produced it are both in the file. The measurement kind is the most undervalued. A single benchmark run before planning has repeatedly changed my architecture, because the assumption I was about to build on turned out to be off by an order of magnitude. _[illustration: Four document cards in a two-by-two grid, labelled BEHAVIOURAL, SPEC, PRIOR ART and MEASURED, each with a check mark beneath it.]_ Four shapes of research note. Knowing which one you are writing is what tells you when it is finished — which is the only reason the categories are worth having. ## Research feeds acceptance criteria Here is the part that closes the loop, and the reason this stage pays for itself twice. Acceptance criteria have to be verifiable by command ([section 06](#s06-decomposition-is-concurrency-control)). Where do those commands come from? Usually from research. The behavioural note that says "this API returns 429 with a `Retry-After` header under these conditions, here is the curl that reproduces it" *is* the acceptance criterion for the retry task, already written. **Insight.** If you find yourself unable to write a command-verifiable acceptance criterion for a task, that is usually not a decomposition problem. It is research you skipped, surfacing two stages later. ## When the honest answer is "spike it" Sometimes the research cannot be completed by reading. The right output then is a research note that says so, and a task in the plan to go find out — timeboxed, with a stated question and a stated deliverable. That is very different from an assumption. A spike is scheduled, bounded, and produces a written answer that the plan can then depend on. An assumption is unscheduled, unbounded, and produces a plan that is wrong in a way nobody is looking for. ## Research is also how you avoid rewriting the plan The failure mode this stage prevents is not "the plan is wrong." It is "the plan is wrong in a way that is only discovered in phase 3, after phases 1 and 2 have been built on top of it." By then the queue holds tasks that assume the wrong thing, agents have committed code that encodes it, and fixing it means unwinding real work. Research is cheap precisely because it happens before anything depends on it. ### Gate — ready for 04 — The plan file? - Every decision the plan is about to make is backed by a research note, or is listed as an open question. There is no third category. - Each note states what it means for the design — not just what you found. - Anything you could not answer by reading is written up as a timeboxed spike with a stated question and deliverable. - At least one note contains a command you ran and its actual output. - You can point at the note that will supply each of the plan's first-phase acceptance criteria. --- ## The one plan file `docs/plan/plan.md` — one file, the complete plan. Not a directory of design documents. Not a wiki. One file. Every good plan I have written converged on the same skeleton, and I now start from it deliberately rather than rediscovering it each time. ## Why one file The plan is ground truth that agents reload on every dispatch. That single fact drives the constraint. Split it across five documents and three things go wrong. Workers load whichever one the task happened to reference, so different agents reason from different subsets. Nobody can tell which document is authoritative when two disagree. And you, three weeks later, cannot either. **Note.** A plan that grows past what fits in one file is usually a sign the project should be two projects. When it genuinely is not, the right move is a scoped sub-plan that names its parent and states which of the parent's constraints it inherits — not a fragmented set of peers. ## Hard constraints — the section that does the work Most of a plan is descriptive. One section is not: the constraints block, which is a contract with every agent that will ever read it. Write it as absolutes. The heading I use is literally *"Hard constraints and invariants (violating any of these is a failed task)"*, and the framing is deliberate. An agent reading "prefer conventional commits" treats it as a default it may override with a good reason. An agent reading "commit style follows the existing history: `feat(scope): …`" treats it as a fact about the world. What belongs there: - **Never-rules.** Never force-push. Never commit build artifacts. Never change an existing URL without a redirect. Each one stated as a prohibition, not a preference. - **Pipeline facts agents cannot discover.** The build image has no Python; the deploy is webhook-driven and takes up to ten minutes; CI pods are deleted on completion so logs must be streamed live. An agent that does not know these will waste a dispatch learning them. - **Safety rules for anything published.** What must never appear in output — credentials, internal hostnames, infrastructure specifics. - **The worker protocol.** How to claim, what "done" means, what to do when blocked. Especially what to do when blocked: *stop, release the task, comment describing the blocker.* Without that clause, a blocked agent improvises. **Critical.** The single highest-value line in any plan I have written is the instruction for what to do when instructions conflict. Agents will encounter contradictions you did not anticipate. Telling them explicitly to stop and surface it — rather than resolving it themselves — converts a silent wrong turn into a visible question. ## Current state, with a date Agents cannot see your machine. Anything they would otherwise guess belongs in a "current state" section, with a verification date attached. This is the part people skip and then wonder why agents hallucinate architecture. If the plan does not say the project has no sitemap integration, has eleven notes in a content collection, and renders its portfolio entirely client-side, then every task that touches those areas begins with an agent inventing an answer. Date it. A current-state section without a date rots invisibly. ## Put the number in the plan If the project is driven by a measurable target — decode rate against a reference implementation, requests handled per dollar, indexed pages — state the metric in the plan explicitly. Autonomous loops work dramatically better chasing a number than chasing a vibe. "Improve performance" produces motion; "p95 under 200ms, measured by this command" produces progress you can verify without reading the diff. ## Open questions are for genuinely open things The closing section of every plan is open questions. It has one rule: **it contains only genuinely deferred items, never decisions an early phase secretly depends on.** A question sitting in that list that Phase 1 needs answered is not a deferred decision — it is a plan defect wearing a disguise, and it will surface as an agent either blocking or guessing. Finding these is most of what the next stage does. Mark decided questions in place rather than deleting them. *"DECIDED 2026-07-04: MIT, with this nuance…"* preserves why, which matters when someone asks six months later. ## Architecture decisions as a table One section that earns its space out of proportion to its length: a table of decisions with their rationale. | # | Decision | Rationale | | --- | --- | --- | | D1 | Server-render the portfolio; keep animation as enhancement | Biggest indexing defect; data is already structured | | D2 | Generate the sitemap from the integration; delete the hand-written one | Auto-includes every new page forever | The point is not documentation. It is **removing per-task judgement calls.** Without D1 written down, every task that touches rendering re-litigates it, and different workers reach different answers on different days. With it, the decision is a fact they inherit. A useful heuristic: any time you find yourself explaining the same preference to a second agent, it belongs in this table. ## Order phases by dependency, not by importance Phases should be ordered so that each one only depends on things already built. That sounds obvious and is routinely violated, usually by putting the exciting work first. Two rules that keep it honest: - **Within a phase, tasks are independent unless an edge says otherwise.** If you cannot say that truthfully, the phase is really two phases. - **A phase that depends on an unmade decision is not schedulable.** Move the decision earlier or move the phase later. There is no third option, and pretending otherwise is what produces blocked agents. ## Sub-plans, when one file genuinely is not enough Occasionally a deliverable is large enough to need its own plan — a substantial feature with its own phases, constraints, and open questions. The failure mode to avoid is a set of peer documents with no stated relationship. What works instead: a scoped sub-plan that **names its parent, states which of the parent's constraints it inherits, and says explicitly which parent item it supersedes.** The parent then carries one line pointing at it. That keeps a single entry point. A worker reading the parent either finds everything it needs or finds an unambiguous pointer — never two documents with equal claim to being authoritative. **Note.** This manual's own plan is a sub-plan of the site's program plan, structured exactly that way, because I hit this problem while writing it. ## The verification playbook The last thing in the plan is how to know it worked: the actual commands, with expected outputs. ```bash # Portfolio is visible in HTML, not just after JS runs curl -s https://example.com/ | grep -c 'ProjectName' # 404 behaves like a 404 curl -s -o /dev/null -w '%{http_code}\n' https://example.com/does-not-exist ``` This does two jobs. It tells agents how to self-verify, and it stops "done" from being a matter of opinion. A task whose acceptance criterion is a command either passes or does not. _[interactive exhibit: plan-skeleton — see the web version]_ ### Gate — ready for 05 — Review? - `docs/plan/plan.md` exists and is the only plan document. - It has all seven parts: overview, hard constraints, current state, architecture decisions, phases, open questions, verification playbook. - Every constraint is phrased as an absolute — "never X", not "prefer Y". - The current-state section carries a verification date. - Every phase depends only on things earlier phases produce. - Every task in every phase names the files it will touch. - If the project has a measurable target, the metric and the command that measures it are both stated. --- ## Refinement — gating the plan before you decompose This is the stage almost nobody documents, and it is the one with the highest return. The reason is arithmetic. A defect in a plan paragraph does not stay one defect. It becomes five wrong tasks, each dispatched to an agent that executes it faithfully, each producing commits you will later have to unwind — and each consuming a model call you paid for. **Insight.** Work queues multiply plan quality in both directions. The same mechanism that lets twenty workers implement a good plan quickly lets them implement a bad one just as quickly. ## Gate it, do not just read it Re-reading your own plan finds almost nothing. You wrote it; the assumptions that are invisible to you were invisible when you wrote them. What works is a structured pass with an explicit checklist, run by something that is not you. Three passes, in order: 1. **Structural review.** Checklist-driven: are there unstated assumptions? Is every data model defined before a phase depends on it? Does any phase depend on a decision still sitting in open questions? Are acceptance criteria verifiable by command, or are they opinions? 2. **Adversarial gap hunt.** A separate pass whose only job is to find what is missing — not to evaluate what is present. These find different things, which is why they are separate passes rather than one. 3. **Fix and re-run.** Edit the plan text, then run both passes again. Repeat until a pass comes back clean. The third step is the one people skip. A review that finds eleven defects and is never re-run has told you the plan had eleven defects, not that it now has zero. ## What to expect from the passes A plan of moderate size typically runs three revisions: - **First draft.** Complete-looking. Expect roughly ten defects — task ambiguities, missing data models, dependency edges encoded wrongly. - **After the first fix pass.** Expect a handful of *residual* defects: some the first review missed, some introduced by the fixes themselves. - **After the second.** Clean. The residual count is the reason the re-run is not optional. A single pass feels thorough and reliably leaves a plan that is still wrong in several places, each of which becomes a task. Record the verdict as a number, not an impression. **"Zero undeliverable tasks"** is checkable; "looks good" is not, and the difference matters because this is the gate the rest of the method depends on. **Note.** "Cold read" matters. The reviewer should not have participated in writing the plan, and should be told to evaluate deliverability — can an agent execute this task as written, with only the repository for context? — rather than whether the ideas are good. ## What the review is actually looking for In rough order of how often they show up: - **Tasks that are not deliverable as written.** The most common defect by a wide margin. Usually a task that assumes context the agent will not have. - **Phases that depend on undecided things.** An open question that Phase 1 needs. Either decide it or move the dependent work later. - **Acceptance criteria that are opinions.** "Works correctly" cannot be validated. "`cargo test decoder::` passes and the CLI round-trips the sample file" can. - **Missing data models.** A plan describing behaviour over a structure it never defines. Every worker invents its own version. - **Dependency edges that are wrong in both directions** — declared where none exists, missing where one does. The second kind is what produces two agents in one file; see [section 06](#s06-decomposition-is-concurrency-control). - **Constraints stated as preferences.** Covered in [section 04](#s04-the-one-plan-file), and worth re-checking here because it is easy to write "should" without noticing. ## How to actually run it The mechanics matter, because "have someone review it" is the kind of advice that never gets followed. What works for me is dispatching the review as its own task, to a fresh session, with an explicit brief: ``` You have not seen this plan before. Do not evaluate whether the ideas are good. For each task in §5, answer one question: could an agent execute this as written, with only this repository for context? If no, say precisely what is missing — a file path, a data model, a decision, a command. Do not propose alternative designs. Output: a numbered list of undeliverable tasks with the specific gap in each. ``` Three things that brief is doing deliberately: - **"You have not seen this plan"** puts the reviewer in the position of the worker who will execute it. That is the perspective that finds missing context. - **"Do not evaluate whether the ideas are good"** stops the review turning into a design discussion, which is a different and much less useful conversation. - **"Do not propose alternative designs"** keeps the output actionable. You want defects, not a redesign. **Note.** The gap hunt gets a different brief: *what is missing entirely?* — sections that should exist, decisions nobody made, failure modes unconsidered. Running both briefs in one pass produces a worse version of each, because the mindsets are genuinely different. ## What good output looks like A useful review finding is specific enough to fix without a conversation: > **Task P3.4 is undeliverable.** It says "add a `/now/` page drawn from > current focus areas" but never says where the focus areas come from. The repo > contains no such list. An agent would either invent content or block. Compare to a useless one — *"P3.4 could be clearer"* — which costs a round trip to turn into the sentence above. If your reviews are producing the second kind, the brief is the problem, not the reviewer. ## When to stop Stop when a full review pass returns nothing new, and the open-questions section contains only items no phase depends on. Not when the plan feels good. Feeling good about a plan is uncorrelated with the plan being deliverable, which is the entire reason this stage is a gate with an exit condition rather than a judgement call. _[interactive exhibit: defect-burndown — see the web version]_ ### Gate — ready for 06 — Decomposition? - A full structural review pass returned nothing new. - A separate gap-hunt pass returned nothing new. - The count of undeliverable tasks is zero, and you can say that as a number rather than an impression. - The open-questions section contains nothing that any phase depends on. - Every acceptance criterion in the plan is a command, not a description. - The reviewer was not the author. --- ## Decomposition is concurrency control This is the stage that decides how big your fleet can be. The rule: > **Every task declares the files it owns. Tasks with disjoint file sets run > concurrently; tasks that overlap are serialised by a dependency edge. Cutting > the work is therefore the same act as planning the concurrency.** That is the mechanism, and it is the whole reason many agents can share one repository safely. Most writing about fleets treats decomposition as a sizing problem — make tasks small enough to finish. Sizing matters, but the *shape* of the cut is what sets how many agents can work at once, and no amount of orchestration downstream can recover a bad cut. Get this stage right and the rest of the path is mechanical. ## Start with a genesis task One root tracking issue, titled after the project, holding a reference to the plan and a checklist of phases. Every phase's tasks block it; it closes when the project ships. It is not project management theatre. It is a fixed point — the thing you check progress against when the queue has ninety items and you have lost the plot. It also gives every task an unambiguous answer to "what is this for", which is the context an agent needs and the plan alone does not provide per-task. ## Pick a decomposition pattern Three are in regular use here: | Pattern | When | | --- | --- | | **Up-front batch** — create every task for every phase now | The plan survived review cleanly and phases are well specified. Expect to create the whole queue in one commit. | | **Phased hierarchy** — fully decompose Phase 1; later phases get one placeholder each, expanded when reached | The default. Later phases depend on outcomes you cannot specify yet. | | **Just-in-time** — a loop creates two to five tasks per iteration from the gap between plan and reality | Metric-driven projects, where the next task depends on current measurements. | ## Size for completion Oversized tasks are the leading cause of worker timeouts. Size to these thresholds: - **Keep together** if it is a single file or module, under ~3,000 characters, with fewer than eight acceptance criteria. - **Split** if it exceeds ~4,000 characters or ten acceptance criteria. - **Split** if it mixes concerns — CLI *and* execution *and* recovery. - **Split** if it contains sequential phases: "first X, then Y, then Z" is three tasks wearing one title. - The canonical shape for a component is a three-part split: setup and scaffolding, core implementation, then edge cases and cleanup. First two high priority, third can wait. Splitting oversized tasks buys two things at once: timeout risk drops, and the amount of work that can proceed in parallel goes up, because smaller tasks name fewer files each. **Insight.** Sizing and partitioning are the same operation seen from two angles. A task small enough to finish reliably is usually also a task that names few enough files to run beside its neighbours. That is not a coincidence: both properties come from the task having one concern. ## Write the full specification at creation time Every task description follows one shape: ``` ## Context — why this exists; link the plan section ## Design — approach, files to touch, constraints ## Acceptance Criteria — each one verifiable by a command ## Notes — gotchas, references ``` "Verifiable by a command" is load-bearing. Agents close their own tasks, so the acceptance criteria are what they validate against. "Works correctly" is useless. `cargo test decoder::` passing is not. ## Now the part that matters: declare the files Add one more line to every task: **the files it owns.** ``` Owns: src/api/upload.ts, src/api/upload.test.ts ``` Everything a task writes must be in that list. Reading anything is fine. Writing outside the list is a failed task — the agent stops and comments rather than taking the file. **Insight.** Two tasks with disjoint `Owns` sets can run at the same time, safely, forever. Two tasks whose `Owns` sets intersect cannot — and no amount of orchestrator cleverness changes that, because they are editing the same bytes. This is why decomposition *is* concurrency control. The `Owns` lines, taken together, are a partition of the repository's file surface. A clean partition means the fleet runs wide. A partition where every task reaches into the router, or the shared types module, or the config file, means the fleet is one worker wearing a costume. ## One feature, cut three ways Take "add file upload" and watch the concurrency change without the feature changing at all. **By layer** — the instinctive cut, one task per architectural tier: ``` A Upload API endpoint Owns: api/router.ts, api/upload.ts B Storage adapter Owns: api/router.ts, storage/s3.ts C Upload form UI Owns: api/router.ts, ui/UploadForm.tsx ``` Three tasks, and `api/router.ts` in all three because each layer registers a route. Peak concurrency: **one**. Three workers would take turns or destroy each other's work. **By file surface** — same feature, cut so nothing overlaps: ``` A Register upload route Owns: api/router.ts B Upload handler Owns: api/upload.ts after: A C Storage adapter Owns: storage/s3.ts after: A D Upload form UI Owns: ui/UploadForm.tsx after: A ``` Four tasks instead of three, one extra dependency edge, and now B, C and D run **concurrently**. The extra task is not overhead — it is the thing that bought the parallelism. **One big task** — everything in a single unit. No overlap, because there is nothing to overlap with. Safe, and exactly as fast as one worker, which is the cost people forget to count. **Insight.** The middle cut is more tasks and *less* total wall-clock. That is the whole argument in one example: the number of tasks is not the cost, the shape of their file sets is. ## Check the partition before you launch This takes seconds and has saved me hours. Before starting a fleet, list the ready tasks and look for a file claimed twice: ```bash # Every file named by more than one ready task ready --json \ | jq -r '.[] | .id as $i | .owns[] | "\(.) \($i)"' \ | sort | awk '{f[$1]=f[$1]" "$2; c[$1]++} END{for(k in c) if(c[k]>1) print k":"f[k]}' ``` Anything it prints is either a missing dependency edge or a task that should be split. Both are cheap to fix now and expensive to discover after two agents have been in the same file for twenty minutes. ## Dependency edges are the serialisation primitive When two tasks genuinely need the same file, do not try to coordinate them at runtime. Serialise them with a dependency edge and let the queue enforce it. The usual fix is better than the usual instinct. If three tasks all need to register a route in `router.ts`, the instinct is to let them coordinate. The fix is to make route registration *its own task* that the other three depend on. Now four tasks have disjoint file sets, three of them run concurrently, and nothing had to negotiate. **Critical.** Only add an edge if the later task genuinely cannot start first — then check the graph for cycles before launching anything. Circular dependencies do not error. They silently starve the fleet — the queue reports plenty of open tasks, all mutually blocked, and workers idle against it indefinitely. ## Why the partition is the safety mechanism Workers on one repository share a working tree. The partition is what keeps them out of each other's way — and it is doing real work, because the failure it prevents is not the one people expect. Two agents editing one file is **not** a merge conflict. Git never sees two versions to reconcile. One agent commits; the other, in the normal course of its own work, runs `git checkout` or `git reset --hard` and discards the first agent's uncommitted changes. No error, no conflict marker. The losing task simply produced nothing, and its worker reports success, because from inside that session nothing went wrong. This is why the `Owns` line is not bookkeeping. It is the declaration that makes the above impossible. **Caveat.** Know where the guarantee comes from. Claims are atomic — one worker per task, enforced by a single database transaction — and a per-workspace lock serialises commit-trailer injection. The working-tree separation, though, comes from the partition itself: every worker on a repository is dispatched into the same directory, with no per-worker worktree or clone. **The discipline is the mechanism**, which is exactly why this stage is worth the care. A worktree per worker would move that guarantee into the tooling, and it is the intended direction. Until it lands, a correctly-authored `Owns` set is what makes in-repo concurrency safe — so treat the partition check below as part of launching, not as an optional review. So the rule to carry out of this section: **before you launch, confirm that every concurrently-runnable task has a disjoint `Owns` set.** It takes seconds, it is the one check that cannot be done after the fact, and it is what turns a pile of tasks into a fleet. _[interactive exhibit: partition — see the web version]_ ### Gate — ready for 07 — The queue? - A genesis task exists and every phase task blocks it. - Every task is under ~3,000 characters with fewer than eight acceptance criteria. - Every acceptance criterion is a command that passes or fails. - Every task carries an `Owns:` line listing the files it may write. - No two concurrently-runnable tasks share a file in their `Owns` sets — checked with the command above, not by eye. - The dependency graph has no cycles. - Tasks that must share a file are serialised by an explicit edge. --- ## The queue — atomic claims and production data The tracker is repo-local: tasks live inside the repository, next to the code. The queue travels with a clone and survives any server. That property is worth more than it sounds — it means there is no tracker to be down, no API to rate limit you, and no separate thing to back up. Two properties actually matter for fleets: claims must be atomic, and the store must be treated as production data. ## Read-then-write claiming does not work The obvious implementation of "claim a task" is two steps: list the open tasks, then mark one as yours. This is a race, and it is not a subtle one. ``` Worker A: list ready tasks → [task-123, task-124] Worker B: list ready tasks → [task-123, task-124] Worker C: list ready tasks → [task-123, task-124] Worker A: mark task-123 in_progress → ok Worker B: mark task-123 in_progress → also ok, silently Worker C: mark task-123 in_progress → also ok, silently ``` Whether the later writes fail depends entirely on whether the tracker enforces exclusivity. Most do not — they just overwrite the assignee field. Adding a read-back to verify does not close it — it only narrows the window. Label-based locking schemes do not close it either: they are the same read-then-write race with more steps. At twenty workers, phantom claims are not an edge case, they are the normal state. **Critical.** If your queue cannot claim in a single atomic operation, your effective fleet size is one. Everything above that is workers burning tokens on work another worker is already doing. ## The database file is the coordination primitive The fix is that a claim must be one transaction: select the next eligible task and mark it claimed, atomically, with the database enforcing exclusivity. SQLite does this with `BEGIN IMMEDIATE`. The write lock serialises the transaction; the second worker to arrive blocks for a millisecond or two, then picks the next available task. No phantom claims, no retry storm, and no server — the file itself is the lock. There is a second lock worth knowing about: a per-workspace advisory lock around claiming, which stops a large fleet from stampeding the same database at the same instant. That is a throughput optimisation rather than a correctness one; the transaction is what makes it correct. **Note.** Note the boundary. Atomic claiming guarantees *one worker per task*. It says nothing about two tasks touching one file — that is [section 06](#s06-decomposition-is-concurrency-control)'s problem, and no amount of claim hardening addresses it. These are different races and conflating them is how people end up surprised. ## Agents own closure Do not parse agent output to decide whether a task is finished. Output formats drift, diagnostics contaminate the parse, and a worker that reports success in an unexpected phrasing leaves a task claimed forever. Instead, make closure part of the agent's instructions: implement, commit with the task ID as a commit trailer, push, validate against the acceptance criteria, then either close the task with a reason or release it with a blocked label and an explanation. The orchestrator verifies exactly one thing: **that a commit exists since dispatch.** That is a cheap tripwire and it catches the failure mode that matters — an agent that reports success while changing nothing. ## Why the tracker lives in the repository Putting the queue inside the repo rather than in a hosted service buys four things, and I did not appreciate the fourth until I had run this for a while: 1. **It travels with a clone.** Check out the repo on another machine and the work queue is there. 2. **There is nothing to be down.** No API, no rate limit, no auth token to rotate, no outage that stops a fleet. 3. **History is unified.** The task that produced a commit and the commit itself are in the same log, so `git log` answers "why does this code exist?" 4. **Agents already have access.** A worker can read the queue with the same tools it reads the code. No credential provisioning, no MCP server, no separate permission model — which removes an entire category of setup that otherwise has to be solved once per worker. The cost is that the queue is only as available as the checkout, and two machines working the same project have two queues until someone pushes. For a single-operator estate that trade has been overwhelmingly worth it. ## The commit trailer Every commit an agent makes carries the task ID as a trailer: ``` feat(upload): add S3 storage adapter Implements the multipart path with retry on 5xx. Task: bf-4k2p ``` That one line is what makes the whole thing traceable in both directions: from a commit to the task that specified it, and from a task to everything it changed. Six months later, `git log --grep` is how you answer "what was this for" without reconstructing anything. It is also what the orchestrator's tripwire looks for. Which brings us to the limits of that tripwire. ## What "verify a commit exists" catches, and what it misses The check is deliberately crude. It is worth being precise about its boundaries, because a tripwire you overestimate is worse than none. **It catches:** an agent that reports success having changed nothing. This is the common failure and the reason the check exists. **It does not catch:** an agent that commits something wrong, commits something unrelated, or commits a partial implementation and closes anyway. All three happen, and none of them are detectable by "did the tree change". **Caveat.** Nothing in the outcome handling verifies that the work is *correct*. That job belongs entirely to the acceptance criteria being commands — which is why [section 06](#s06-decomposition-is-concurrency-control) treats "verifiable by a command" as load-bearing rather than a style preference. The commit check is a floor, not a quality gate. ## Treat the queue like production data The tracker here has two representations: a live SQLite database, and a git-tracked JSONL checkpoint. The database is authoritative; the checkpoint is written by an explicit flush. That split has one sharp edge. Repair tooling rebuilds the database *from* the checkpoint, so running a repair before flushing destroys every task created since the last flush — silently, because from the tool's perspective it did exactly what it was asked. **Critical.** The safe order is always: **flush first, verify integrity, repair only if needed.** Never repair against a stale checkpoint. The queue is the fleet's memory, and losing it is not losing a database — it is losing the plan. ```bash # 1. checkpoint the live store sync --flush-only # 2. a real integrity check, not the tool's own opinion sqlite3 .beads/beads.db "PRAGMA integrity_check;" # 3. only now, and only if step 2 complained doctor --repair ``` The tool has since grown a guard that refuses to repair over unflushed work, but the ordering is still the habit worth having — the guard is a backstop, not a substitute for knowing which representation is authoritative. _[interactive exhibit: engine-room — see the web version]_ ### Gate — ready for 08 — Sizing the fleet? - Two workers attempting the same task results in exactly one claim. Test it rather than assuming it. - Claiming is a single database operation, not a read followed by a write. - Agents close their own tasks and the orchestrator verifies a commit exists since dispatch. - Commits carry the task ID as a trailer. - You know which representation of the queue is authoritative, and which is the checkpoint. - You have flushed the live store to its checkpoint at least once, and know to do so before any repair. --- ## Scaling in two dimensions A fleet grows along two independent axes, and they behave nothing alike. **Depth** is workers inside one repository. **Width** is repositories in flight at once. Most confusion about how large a fleet can get comes from treating these as one number. ## Depth is bounded by the partition Adding a second worker to a repository is safe exactly to the degree that the tasks they will pick up have disjoint file sets. That is not a property of the orchestrator, the model, or the machine — it is a property of how you cut the work, which is why [section 06](#s06-decomposition-is-concurrency-control) comes first. Two shapes, at opposite extremes: - **A content repository**, where every task owns one markdown file and nothing else. The partition is close to perfect. Depth scales until you run out of ready tasks. - **A refactor touching a shared module**, where every task inevitably reaches into the same file. The partition is nearly degenerate. Depth caps at one or two no matter how many workers you start. The failure at depth is not slowness. It is the quiet destruction described in section 06: work that was done, then discarded, with both workers reporting success. **Insight.** Past a certain overlap ratio, adding workers makes throughput *worse*, not just flatter. Each new worker raises the chance of colliding with every worker already running. ## Depth is also bounded by the host There is a second ceiling on depth that has nothing to do with the partition, and it bites earlier than people expect. Every worker costs something just by existing. Measured across a twelve-worker fleet mid-execution, an idle supervisor process held **roughly 500 MB** resident before dispatching anything at all, and each agent subprocess it launched added another 230–400 MB on top of that. The first number is the one that matters: nothing gates it, and it scales linearly with worker count whether those workers are executing or sitting against an empty queue. That arithmetic sets the ceiling. Forty workers is some twenty gigabytes of resident memory before a single agent starts — on a machine that then has to hold the agents too. Fleet size is bounded by the floor, not by the peak. **Insight.** Worker count has to be bounded by the host's capacity for the fleet's overhead, not by the number of available tasks. The overhead is per-worker and constant; it does not shrink because the tasks are small. So depth has two independent ceilings — the partition's overlap ratio and the machine's headroom — and you hit whichever is lower. A perfectly partitioned repository still cannot absorb more workers than the host has memory to hold idle. ## Width is bounded by having enough different work Width has no shared working tree, so it has none of depth's failure modes. Two repositories cannot corrupt each other's checkouts. What bounds width is supply. Each repository in flight needs a ready queue — a plan, decomposed, with unblocked tasks in it. Nine repositories with three ready tasks each keep a fleet fed; nine repositories where eight have empty queues just means eight idle workers and a lot of polling. This is why **spreading wide across repositories is what lets a fleet take on diverse work.** Depth gets one thing done faster. Width is how you get many different things moving at once, and in practice it is where a fleet's real capacity comes from — because most estates have far more breadth of pending work than they have depth of parallelisable work inside any single repository. ## Which axis to reach for A rough decision procedure: 1. **Is the work in one repo?** If the tasks partition cleanly — one file each, no shared surface — go deep. This is the content-repo case and it is the happy path. 2. **Does every task touch the same file?** Do not go deep. Either restructure the decomposition so the shared file becomes its own task the others depend on, or accept depth 1 and go wide instead. 3. **Do you have many repos with ready queues?** Go wide. This scales further than depth and has a far more forgiving failure mode — the worst case is an idle worker, not lost work. **Caveat.** The two axes multiply, but only nominally. Total workers is depth × width; useful work is depth-adjusted-for-overlap × width. Reporting the first number as capacity is how people end up believing they are running twenty agents when they are getting the throughput of six. ## Measuring your own overlap ratio The exhibit above uses a parameter — file overlap in the partition — which is easy to state and easy to treat as a vibe. It is measurable. Take the ready tasks, count how many distinct files they name in total, and count how many are named more than once. The second number over the first is your overlap ratio, and it is the number that decides how far depth scales: ```bash ready --json | jq -r '.[].owns[]' | sort | uniq -c \ | awk '{t++; if ($1>1) c++} END {printf "%d/%d files contended (%.0f%%)\n", c, t, 100*c/t}' ``` Under about five percent, go as deep as the host allows. Over about a third, depth is not going to help and the effort belongs in re-cutting the tasks. **Note.** This measures the *ready* set, not the whole queue, and that is deliberate. Tasks blocked behind dependency edges cannot collide with anything — the edge is already doing its job. Only what can run right now can conflict right now. ## The estate view Once both axes are understood, capacity planning stops being about workers and becomes about queues. The question is no longer "how many agents should I run?" It is "how many repositories currently have ready, well-partitioned work?" — because that product, not the worker count, is what actually determines throughput. In practice this means the constraint usually binds upstream of the fleet entirely. Most days the limit is not compute and not agents; it is how many projects have a reviewed plan decomposed into ready tasks. Which puts the bottleneck back on the stages in sections 03 through 06, where it belongs. ## What this changes about planning Once depth is understood as a property of the decomposition, it stops being a runtime tuning knob and becomes a planning decision. The question "how many workers should I run on this repo?" has no answer in isolation. The answerable version is: "given how I cut these tasks, how many can run concurrently?" — and that is determined before a single worker starts, in the `Owns` lines you wrote during decomposition. _[interactive exhibit: scale — see the web version]_ ### Gate — ready for 09 — Deployment? - You have measured the overlap ratio of the ready set, not estimated it. - You have chosen depth or width for this run, and can state why. - Your intended worker count is below the host's sustainable ceiling. - If going deep: the overlap ratio is low enough that the workers you plan to start will all be productive. - If going wide: every repository you plan to include has ready, unblocked tasks in it. --- ## Fleet deployment The orchestrator's job is deliberately boring: pick the highest-priority unblocked task, claim it atomically, build a prompt from the task specification, invoke an agent CLI, and handle the outcome by an explicit table. ``` success → verify a commit exists → close failure → release → retry timeout → release → defer repeated failure → escalate, stop retrying ``` Determinism in that loop is what makes a fleet debuggable. When something goes wrong at twenty workers, you need to be able to say what the system did, not guess at what it decided. ## Adapters, and the invoke-template contract The orchestrator does not know what an LLM is. It knows how to run a command. Each agent CLI is described by an adapter: a template string with a few substitutions. ``` cd {workspace} && --model {model} < {prompt_file} ``` The available substitutions are `{workspace}`, `{prompt_file}`, `{bead_id}`, and `{model}`. That is the entire contract. Adapters ship for the major CLIs, and adding one for a tool nobody has heard of is a config entry, not a code change. **Insight.** Making the agent CLI a configuration detail rather than a dependency is the single decision that has aged best. Models and CLIs have churned repeatedly; the dispatcher has not had to care. **Caveat.** Note what `{workspace}` is: the repository path, passed through verbatim. There is no per-worker suffix, clone, or derived worktree. Every worker on a repository is dispatched into the same directory — which is exactly the constraint [section 06](#s06-decomposition-is-concurrency-control) exists to manage. ## What actually goes into a dispatch The prompt a worker receives is assembled, not written. Its parts:
1. **The standing instructions** from the repo root — build commands, commit conventions, never-rules. 2. **The task specification** verbatim — context, design, acceptance criteria, owned files. 3. **The protocol**: implement, commit with the task ID as a trailer, push, validate against each acceptance criterion, then close with a reason or release with a blocker comment. 4. **The stop conditions**: what to do when blocked, when instructions conflict, or when the work would require touching a file the task does not own. Part four is the one people leave out, and it is the difference between an agent that surfaces a problem and one that improvises around it. An agent with no stated stop condition will always find a way to proceed, because proceeding is what it is for. ## The outcome table, in full Every dispatch ends in exactly one of these, and the handling is fixed: | Outcome | Handling | | --- | --- | | Agent closed the task, commit exists | Accept. Move on. | | Agent closed the task, **no commit since dispatch** | Reject the close, reopen, flag. This is the tripwire. | | Agent released with a blocker comment | Leave blocked. It needs a human or a dependency. | | Non-zero exit, no close | Release, increment failure count, retry if under the threshold | | Timeout | Release and defer. Usually means the task is too big. | | Failure count at threshold | Stop retrying. Escalate as a task. | The value is not in any individual row — it is that the table is exhaustive and fixed. When a fleet does something surprising, you can point at the row that produced it. A dispatcher that decides case by case gives you nothing to point at. ## Route by difficulty, not by preference Cost control is mostly one rule: **routine, well-specified tasks go to a cheap model tier; gnarly ones, and anything that already failed once, go to a stronger one.** That alone changes the economics of running twenty workers more than any other tuning available. Most tasks in a well-decomposed queue are routine by construction — that is what decomposition was for — so most dispatches should be cheap. With per-task velocity statistics you can go further and route by observed model performance per task type. That is a refinement. The cheap-by-default rule is where the money is. ## Four dispatch modes Dispatch is a spectrum, not a switch, and matching the mode to the work is most of running a fleet well. They ladder by how much oversight each one needs. An **interactive session** is one conversation you watch continuously — the right mode for exploratory work, and for debugging the tasks themselves. A **supervised batch** fans a dozen tasks out and you review per batch: a phase you want eyes on. An **autonomous fleet** is where the volume happens — workers looping claim, execute, close for hours, with your attention arriving per incident rather than per task. A **metric marathon** is a single persistent loop chasing one number from the plan, and the number is the only thing you supervise. The one rule that is not obvious: **never run a metric marathon and fleet workers on the same repository at once.** They fight over working-tree state, for exactly the reasons in section 06 — the marathon loop holds uncommitted work across iterations, and a fleet worker's checkout will eat it. ## Stagger the launch Starting every worker at once produces a thundering herd at the least useful moment. All of them scan for work simultaneously, all of them compute the same highest-priority task, all of them attempt to claim it, and the database serves one winner and a pile of losers — then they all retry in lockstep. A one-to-two second stagger between launches removes it entirely. The workers arrive at the queue at different moments, see different frontiers, and spread across the available work instead of stacking on its head. **Note.** Atomic claiming makes the herd *correct* — exactly one worker wins — but it does not make it *efficient*. Correctness and contention are separate problems, and the stagger is what addresses the second. ## Workers exiting is not workers failing A worker that has drained its workspace stops on an idle timeout and exits. That is the designed behaviour, and it looks identical to a crash in any dashboard that counts running processes. The operational consequence is that a fleet is not self-sustaining: as workspaces empty, the fleet shrinks. Either something redistributes workers to workspaces that still have ready tasks, or an operator relaunches them. Assuming a fleet stays the size you started it at is how you end up with two workers and a queue you thought was being worked. ## Where workers actually live Each worker is a supervisor process holding a terminal session, dispatching an agent CLI in a loop. That is a deliberately unglamorous substrate, and it has one property that matters: you can attach to a worker and watch it work. Debugging a fleet is mostly reading what an agent actually saw. Being able to attach to a live session and scroll back beats any amount of structured logging for that particular job — though structured telemetry is what tells you *which* session to attach to. **Critical.** Stopping a fleet is not one command, and getting it wrong leaves orphans. Stopping the terminal sessions kills the visible part while leaving supervisor processes and in-flight agent dispatches running — which then hold claims on tasks nobody is working. Know the full stop sequence for your setup and verify with a process listing, not with the tool's own status output. _[interactive exhibit: dispatch-ladder — see the web version]_ ### Gate — ready for 10 — Running it? - An adapter is configured and a single task dispatches end to end. - Every outcome in the table has a defined handling, including repeated failure. - A failure threshold exists and stops dispatch rather than retrying forever. - Launches are staggered. - Model routing sends routine tasks to the cheap tier. - You have run the full stop sequence once and confirmed with a process listing that nothing was left running. --- ## The operator's day The steady state is pleasantly dull. A queue draining. Commits with task-ID trailers accumulating. Workers going idle and exiting when nothing is ready. If you are watching a fleet closely for hours, either something is wrong or you have not yet built the habit of trusting the outcome table. The interesting question is not what to watch continuously — it is which signals deserve interruption. ## What is worth an alert Three things: - **A task that has failed more than twice.** Not the first failure; that is normal. The third means something structural, and it will keep failing forever if nothing stops it. - **A queue that is not draining while workers are alive.** Either everything is blocked, or workers are looking in the wrong place. - **Spend rate departing from the task-completion rate.** The signature of work being redone. Everything else can wait for a scheduled look. _[illustration: An operator standing with folded arms beside a panel holding three warning rows, next to a column of six small grey blocks carrying no marks.]_ Three signals earn an interruption. Everything else is the grey column — real, worth reading eventually, and not worth a notification. ## "The fleet is starved" is usually false This is the most common false alarm I get, and I have learned to distrust it by default. Nearly every starvation alert I have investigated had ready tasks sitting in the queue. The alert was real; the diagnosis was not. The actual causes, in rough order of frequency: - A worker querying from the wrong working directory, so it saw an empty queue that was not the queue. - Diagnostic output contaminating parsed command output, so a populated queue parsed as zero items. - A dependency cycle — plenty of open tasks, all mutually blocked, none ready. The second cause is worth understanding, because the mechanism is invisible and the symptom is perfectly misleading. A helper writes diagnostic output to stdout. That is harmless everywhere except inside a command substitution whose stdout *is* the return value — a JSON list of claimable tasks. The debug text lands in the middle of the JSON, parsing fails, and the failure reads as an empty candidate list. Full queue, correct query, and the worker concludes there is nothing to do, with every layer reporting success. Any diagnostic written by a function that is ever called in a subshell belongs on stderr. Check that before you check anything else. **Critical.** Re-check the ready queue yourself before believing any starvation alert. Run the query, from the right directory, and read the output. This takes thirty seconds and has been right more often than the alert has. **Note.** A worker exiting when its workspace runs dry is **normal**, not a crash. Workers finish the available work and stop on an idle timeout. Mistaking that for a failure leads to relaunching workers into an empty queue and then investigating why they exit again. ## Reading a worker session When something does need investigating, the artifact worth reading is the session itself — what the agent saw and what it did, in order. Three things I look for, roughly in this order:
1. **What context did it actually get?** Nine times in ten the agent behaved sensibly given wrong or missing input. Check the task specification before questioning the reasoning. 2. **Where did it start improvising?** There is usually a specific turn where it stopped following the task and started solving a problem it invented. That turn is the defect — normally a stop condition the task never stated. 3. **What did it think "done" meant?** If it validated against something other than the acceptance criteria, the criteria were not commands. Notice that all three diagnoses point upstream — at the task, not the agent. That has been true often enough that I now treat "the model did something stupid" as a hypothesis of last resort rather than first. ## Daily versus weekly Not everything deserves the same cadence, and conflating them is how operating a fleet turns into a full-time job: **Daily, and it should take two minutes:** are workers alive, is the queue draining, did anything escalate. If all three are fine, stop looking. **Weekly, and it deserves real attention:** the pass below. **Insight.** The daily check is deliberately not a review. Its only job is to detect that the weekly review needs to happen early. Treating every day as a review day is how people conclude fleets are exhausting to run — the whole point is that most days require nothing. ## Telemetry that earns its place Structured telemetry exported in standard formats is worth the setup, but not for dashboards. It is worth it for one question: *which session do I attach to?* The useful signals are per-dispatch — model, token usage, duration, outcome — and what you actually do with them is spot the outlier, then go read that worker's session directly. Aggregate charts of a fleet tell you remarkably little; the distribution's tail tells you everything. Per-task velocity statistics are the exception. Those accumulate into something genuinely useful: which task types are slow, which model handles which shape of work, where the estimates were wrong. ## The weekly pass Once a week, not continuously: 1. **Read the escalations.** Anything that became a human-labelled task. 2. **Look at the failure list.** Tasks that failed and were retried — are they the same underlying problem wearing different task IDs? 3. **Check the plan against reality.** What did the fleet discover that the plan does not know? That is stage 7, and it is the step that keeps the queue from describing a system nobody is building any more. 4. **Check spend against completion.** A rising ratio means rework. **Note.** The temptation is to automate this pass. I have not, because every time I have looked at it manually I have found something the automation would not have known to check — usually a plan assumption that quietly stopped being true. ### Gate — ready for 11 — Troubleshooting? - You can answer all three daily questions in under two minutes: are workers alive, is the queue draining, did anything escalate. - Escalations arrive as tasks, not as messages in a channel you have to remember to read. - You can attach to a running worker and read what the agent actually saw. - A weekly review is scheduled, and its output is written down somewhere the plan can absorb. --- ## Failure modes Follow the path and most of this section never reaches you — each stage above exists partly to prevent one of these. But a manual that only describes the happy path leaves you unable to recognise the unhappy one, so here is what the steps are protecting against, and what to do if something gets through. Read it once now and once after your first fleet run. It will mean more the second time. ## Runaway retries A task that keeps failing will be re-dispatched forever by a naive loop. There is no natural stopping point: the task is open, it is unblocked, it is high-priority, so the dispatcher picks it again. Left alone, this is unbounded. A single task can absorb hundreds of dispatches over a day and hundreds of dollars, and still be open at the end of it. **Critical.** Circuit-break explicitly. After about three failures, stop dispatching, split the task into smaller children, and block the original until they land. "It will eventually succeed" is not a retry policy; it is an unbounded spend authorisation. The deeper lesson is that a task failing repeatedly is almost never a transient problem. It is a task that is too large, underspecified, or blocked on something the plan does not mention. Retrying does not address any of those. ## Recursive self-decomposition **Symptom:** the task count grows exponentially. Many tasks with near-identical titles, several generations deep. **Cause:** automatic splitting without a termination condition that actually fires. A task judged too large is split into children; the children are eligible for the same evaluation; each splits again. Six generations of five children each is over five thousand tasks. **Fix:** two guards, and both must fail closed. - Mark generated children with a label that excludes them from re-evaluation, or track generation depth and stop at a maximum. - Verify the guard reads a field that is actually populated. A depth guard that reads its label from a source returning nothing computes depth zero forever and never fires. **Critical.** A safety check that can be satisfied by empty input is not a check. Treat "no data" as "do not proceed", never as "constraint not violated." This is the single most valuable rule in this section, because a guard that fails open is indistinguishable from a guard that works until the day it matters. **Note.** Count failure modes, not guards. Several guards reading the same field are one guard wearing several names — they will fail together, while appearing to be defence in depth. ## Silent orphans A worker dies mid-dispatch — OOM, a killed session, a supervisor that exited without cleaning up. The task stays marked claimed, assigned to a worker that no longer exists. Nothing is working on it, and nothing will, because from the queue's perspective it is in progress. The fix is a reaper: detect claims held by processes that are gone, release them back to ready. The thing that makes this hard is that "the worker is gone" and "the worker is slow" look identical from outside, so the check has to be against the process table rather than against elapsed time alone. Related, and worth checking for specifically: stopping a fleet incorrectly produces exactly this state at scale. Killing terminal sessions leaves supervisor processes and in-flight dispatches alive, holding claims. Verify a stop with a process listing, not with the orchestrator's own status output. ## Escalation belongs in the queue Some things an agent genuinely cannot decide: a product judgement, a credential that must be created by a human, an ambiguity where guessing has real consequences. The wrong answer is a side channel — a message that gets lost, or a log line nobody reads. The right answer is that **escalation is a queue item.** The task gets a `human` label, blocks its dependents, and pages me. The queue itself is the escalation channel, so there is exactly one place where the state of the project lives. **Insight.** Anything that would otherwise become out-of-band state should become a task. This is the same instinct as "artifacts, not conversation" from section 01, applied to exceptions rather than to handoffs. ## Do not let the agent grade its own homework unsupervised Agents close their own tasks — that is [section 07](#s07-atomic-claims)'s design, and it is right, because parsing output to infer success failed worse. But self-closure needs a tripwire. The orchestrator verifies that a commit exists since dispatch. That is a low bar deliberately: it is cheap, it is unambiguous, and it catches the specific failure that self-closure invites — an agent that validates enthusiastically against acceptance criteria while having changed nothing. Acceptance criteria that are commands are what make the rest of it work. An agent can convince itself that "works correctly" is satisfied. It cannot convince itself that a failing test passed. ## Queue corruption Covered in section 07 and repeated here because it is the one that loses the most: the live store is authoritative and the checkpoint is a snapshot. Repair rebuilds from the checkpoint. Repairing before flushing destroys everything created since the last flush. Flush first. Always. ## Resource exhaustion looks like model failure A worker that gets OOM-killed mid-dispatch does not report an out-of-memory error. It reports nothing — the process is gone. What you observe is a task that was claimed, produced no commit, and is now held by a worker that does not exist. The tell is correlation: several workers failing in the same window, on unrelated tasks, in unrelated repositories. Model quality does not fluctuate by wall-clock minute. Memory pressure does. Worth checking before blaming the agent: memory headroom per worker, and whether the fleet size respects the host ceiling from [section 08](#s08-two-dimensions). ## Self-modification is its own category If your fleet's tooling is itself in a repository the fleet works on, a bad change can take out every worker at once — the orchestrator deploys a broken binary and the whole fleet stops, including the workers that would have fixed it. The mitigations are the ordinary ones from deployment engineering, and they apply unchanged: stage the rollout so one worker gets a new version first, keep the previous version recoverable, and never let an automated path replace the thing that runs the automated path without a gate. **Insight.** A fleet that maintains its own tooling has a bootstrapping problem the moment the tooling breaks. Keep at least one path to recovery that does not depend on the fleet — usually just you, a terminal, and a known-good binary. ## A triage order that works When something is wrong and you do not yet know what, this order has found it fastest for me:
1. **Is there ready work?** Run the ready query yourself, from the right directory. Most alarms die here. 2. **Are the workers alive?** Process table, not the tool's status output. 3. **Is one task failing repeatedly?** Check failure counts before reading any logs — a runaway retry explains most cost and log-volume anomalies. 4. **Did anything commit?** If claims are moving but the tree is not changing, you have agents reporting success without doing work. 5. **Only now, read a session.** Pick the outlier by duration or token count and read what the agent actually saw. Steps one through four are cheap and mechanical. Step five is expensive and requires judgement, which is exactly why it is last. ## The compounding failure The worst incidents are not any single mode — they are two interacting. A task too large to finish (sizing) fails repeatedly (no circuit breaker), each failure leaves a partial working tree (no isolation), which causes the *next* task in that repo to fail for unrelated-looking reasons (contamination), which triggers its own retries. By the time it surfaces it looks like the model got worse. It did not. Four independent gaps lined up. That is the argument for why the boring stages come first. Sizing, review, and partition discipline are not process for its own sake — they are what stops the failure modes in this section from finding each other. _[interactive exhibit: failure-scenes — see the web version]_ --- ## Lineage — what I adopted and what I arrived at This workflow is not a fork of one source and not an independent invention. Two of its stages came from someone else's published methodology, and the rest were worked out here. Sorting them is worth a section, because a manual that blurs that line is asking you to trust it on everything else. **Note.** "Arrived at here" means *not taken from anyone* — it is a claim about how this process was built, not a claim of being first. Several of these are practices other people have surely reached on their own, and some are old ideas from outside agent work. None of that changes the honest answer to "where did you get this?" ## Adopted from Jeffrey Emanuel Two stages, taken deliberately because they were right: **The single plan file** ([section 04](#s04-the-one-plan-file)). One document, complete, reloaded on every dispatch. I had been splitting design across several documents and paying for it in exactly the way that stage describes — workers reasoning from different subsets, and no answer to which one wins. **Gating the plan before decomposition** ([section 05](#s05-gating-the-plan)). Review the plan to a checkable exit condition, and only then cut tasks. This is the highest-leverage stage in the whole path, and I did not work it out; I adopted it. Those two are load-bearing. Removing either would take a large piece of the method with it, and pretending they were mine would be a strange thing to do about the parts that work best. ## Arrived at here The rest was built in this estate, mostly by hitting a wall and then designing around it: - **The docs-first scaffold** ([02](#s02-repo-scaffold)) — the fixed tree, and the split between curated notes and the agent-written execution trail. - **Research before planning** ([03](#s03-research-before-planning)) — one note per question, calibrated to how quietly a wrong assumption fails. - **The genesis task** ([06](#s06-decomposition-is-concurrency-control)) — one root tracking issue holding the phase checklist, as a fixed point to check progress against. - **Task sizing rules** ([06](#s06-decomposition-is-concurrency-control)) — the character and criteria thresholds, and the three-part split for a component. - **Agent-owned closure with a commit tripwire** ([07](#s07-atomic-claims)) — the agent validates and closes; the orchestrator verifies only that a commit exists. Both halves came from watching output parsing fail. - **The design-time file partition** ([06](#s06-decomposition-is-concurrency-control)) — `Owns` lines as the concurrency plan. This is the idea I would defend hardest, and it is the one this manual is built around. - **Two-axis scaling** ([08](#s08-two-dimensions)) — depth bounded by the partition and by the host, width bounded by ready queues. _[illustration: A provenance map: two large cards under the heading ADOPTED on the left, and five smaller cards under the heading ARRIVED AT on the right, separated by a dividing line.]_ The count is not the point — the two on the left are load-bearing, and removing either would take a large piece of the method with it. Sorting them is what makes the rest of the manual worth trusting. ## Atomic claiming, and where it came from One item deserves its own note, because "arrived at here" is true but incomplete. **Atomic claiming** ([section 07](#s07-atomic-claims)) was designed in response to a specific gap: the tracker I was running at the time claimed work with a client-side read-then-write sequence, which is a race. At twenty workers it produced phantom claims constantly. The single-transaction claim, and the tracker built around it, exist because that behaviour needed replacing. So the mechanism is mine and the *problem statement* was handed to me by the tool I was already using. Work queues are old and well-studied ground — the contribution here is not the technique, it is treating atomic claiming as the threshold property that separates several agents from a fleet. ## Where this diverges Three differences from the methodology the two adopted stages come from. Each is an engineering judgement, and each is argued in full elsewhere in this manual: **Coordination happens at design time, not at runtime.** The canonical approach gives agents a messaging channel to coordinate through. There is no inter-agent channel here at all. The file partition does that work before dispatch, so overlapping tasks are serialised by a dependency edge rather than negotiated by messages while running. **The tracker is a local implementation.** It stays compatible with the tool it replaces and adds the atomic claim, because a fleet without one is a fleet of one. **The isolation caveat is stated rather than implied away.** Working-tree separation comes from the partition, not from per-worker worktrees. [Section 06](#s06-decomposition-is-concurrency-control) says so directly, and names worktree isolation as the intended direction rather than suggesting it already exists. **Insight.** None of these are disagreements about goals. They are different answers to the same question — how do you stop many agents from stepping on each other — and the answer here is "decide it when you cut the work" rather than "let them sort it out while running." --- ## Steal this without the tooling None of this requires my tracker, my orchestrator, or a fleet. Most of it improves a single interactive session, and all of it transfers to any agent CLI with any queue. In rough order of return per unit of effort: ## 1. Docs-first scaffold Create `docs/research/`, `docs/plan/`, and an agent-instructions file before you write code. Agents are bounded by what they can read, and this costs a minute. Works with: one session, no tooling at all. ## 2. Review the plan before decomposing it The highest-leverage hour available. Run a structured pass, then a separate adversarial gap hunt, then fix and re-run until clean — and use something other than yourself, because you cannot see your own assumptions. Exit condition: zero undeliverable tasks, not "looks good." Works with: one session. Ask a fresh session to cold-read the plan for deliverability. ## 3. Small tasks with command-verifiable acceptance criteria One concern, one module, under ~3,000 characters, fewer than eight criteria, every criterion a command that passes or fails. This is the single biggest quality lever in the whole method, and it is entirely free. It works because it removes the agent's discretion about whether it is done. Works with: one session. It is just how you phrase the request. ## 4. Declare the files each task owns Even with one agent, writing `Owns: src/foo.ts, src/foo.test.ts` sharpens the task — it forces you to decide the blast radius before starting, which surfaces badly-scoped work early. With more than one agent it stops being a nicety and becomes the concurrency plan. See [section 06](#s06-decomposition-is-concurrency-control). Works with: one session, as a scoping discipline. Load-bearing at two or more. ## 5. Make the agent close its own work, and verify a commit exists Have the agent validate against the criteria and state its result, then check the one thing it cannot fake: that something actually changed. Works with: one session — `git log` after the fact is the same check. ## 6. Atomic claims, if you ever run more than one worker Everything above is optional polish for a single session. This one is not optional the moment there are two workers: if your queue cannot claim in one atomic operation, your effective fleet size is one regardless of how many processes you start. **Insight.** The first five transfer down to a single session. The sixth is the threshold — it is what separates "several agents" from "a fleet," and it is a property of the queue, not of the agents. ## A starting checklist If you want to try this on the next thing you build, in order:
- [ ] Create `docs/research/`, `docs/plan/`, and an instructions file before any code - [ ] Write one research note per question you would otherwise guess at - [ ] Write `plan.md` with a hard-constraints section phrased as absolutes - [ ] Have a fresh session cold-read it for deliverability; fix; re-run until clean - [ ] Cut tasks: one concern, under ~3,000 characters, fewer than eight criteria - [ ] Every criterion a command; every task an `Owns:` line - [ ] Check that concurrently-runnable tasks have disjoint `Owns` sets - [ ] Run it — and verify a commit exists per completed task The first four take an afternoon and account for most of the benefit. The last four are what let you add a second worker without it costing you. _[illustration: One person at a single laptop beside a checklist card, its upper items checked off in red and its lower items still empty.]_ One person, one session, no fleet. The checked half is the afternoon that pays for itself; the rest is only worth doing if you ever add a second worker. ## Why this keeps working Tools will keep changing and the models certainly will, but the shape has outlasted every upgrade so far: *one reviewed plan, decomposed into small verifiable tasks with declared file ownership, processed by workers with deterministic outcome handling.* It holds because none of it depends on the model being good at anything in particular. It depends on the work being specified well enough that finishing it is unambiguous, and cut cleanly enough that finishing several at once is safe. Those are properties of your plan and your task list, not of the agent — which is why the path invests so much before the first dispatch. The one idea to carry away is section 06's, because it is the least obvious and the most load-bearing: **how you cut the work is how you cut the concurrency.** Everything else here is technique. That one is structure — and it is what makes the difference between running an agent and running a fleet. Start with the checklist above. The first project takes an afternoon of setup; every one after that starts from a template you already know works.