From idea to agent fleet: how I build applications with LLM coding agents

I build most of my software with LLM coding agents now — not one chat session, but fleets of headless workers grinding through a queue of small, well-specified tasks while I do something else. Dozens of repos have been built this way, from an RDAP-backed domain checker to a pure-Rust GRIB2 decoder to a Telegram bridge for Claude Code.

The workflow that emerged has a specific shape, and the shape matters more than the tools. Every stage exists because skipping it burned me: projects that skipped research got thin plans, projects that skipped plan review shipped the plan’s defects multiplied by the number of agents, and oversized tasks were the number-one cause of worker timeouts. This guide walks the whole arc — idea → repo → plan → work queue → fleet — with the real numbers and the real failure modes.

My stack is Claude Code as the agent harness, beads-style git-native issue tracking (I run bead-forge, my implementation built for concurrent fleets), and NEEDLE, a Rust orchestrator that dispatches queue items to any LLM CLI. But the workflow transfers: any agent CLI, any queue with atomic claims, any dispatcher.

The shape of the workflow

1. Scaffold the repo around documentation
2. Research before planning
3. Write one plan file
4. Review the plan before decomposing it
5. Decompose the plan into a dependency-aware work queue
6. Run the fleet
7. Watch it, and know the failure modes

1. Scaffold the repo around documentation

Every repo starts with the same tree, created before any code:

<repo>/
├── README.md          ← purpose and contents
└── docs/
    ├── notes/         ← features, constraints, decisions unique to this app
    ├── research/      ← third-party research and source material
    └── plan/
        └── plan.md    ← single file: the entire application plan

Docs-first isn’t ceremony — for agents, documentation is the interface. Workers load the plan and notes as context on every dispatch; the quality of what they read bounds the quality of what they write.

One thing to expect: a second notes tree appears at the repo root once agents start working. docs/notes/ holds curated, human-shaped topical notes; notes/ fills with per-task work logs the agents write themselves (notes/<task-id>.md). Let it happen — the execution trail is genuinely useful when you’re reconstructing why an agent did something — but keep the two trees distinct. Curated knowledge you’d hand a new contributor goes in docs/notes/.

2. Research before planning

docs/research/, one markdown file per source or question: an API’s real behavior, a protocol spec, prior art, a paper. The discipline is answering “what do I not know yet?” in writing before the plan commits to answers.

Calibrate to risk. A project wrapping a well-understood API needed three research docs; a binary-format decoder accumulated fifteen. The one project where I skipped this stage ended up with the thinnest plan and the most ad-hoc development — the correlation is not subtle.

3. Write one plan file

docs/plan/plan.md — one file, the complete plan. Every good plan I’ve written converges on the same skeleton:

## Overview
## Design Principles / Non-Goals
## Architecture
## Components            (numbered, one subsection each)
## Data Models           (always a dedicated section)
## Implementation Phases (Phase 1..N, concrete deliverables each)
## Open Questions        (the universal closing section)

Why one file: the plan is the ground truth agents reload every iteration. Split it across five documents and workers (and you, three weeks later) can’t tell which is authoritative. If the project is driven by a measurable target — “decode rate against the reference implementation”, “requests handled per dollar” — put the metric in the plan explicitly. Autonomous loops work dramatically better chasing a number than chasing a vibe.

4. Review the plan before you decompose it

This is the highest-leverage hour in the whole workflow. A defect in a plan paragraph becomes five wrong tasks, each dispatched to an agent that executes it faithfully, each producing commits you’ll have to unwind. Work queues multiply plan quality — in both directions.

So gate it: run a structured review (I use a checklist-driven pass looking for unstated assumptions, missing data models, phases that depend on unmade decisions), then an explicit gap hunt, then fix the plan text and re-run until clean. The projects where I closed all plan gaps before creating tasks had the smoothest runs; the “Open Questions” section at the end should contain only genuinely deferred items, never decisions Phase 1 secretly depends on.

5. Decompose the plan into a work queue

Now convert the plan into tracker issues — I’ll call them tasks here; in my tooling they’re “beads” in a git-native SQLite/JSONL tracker that lives inside the repo, which means the queue travels with the code and survives any server.

Start with a genesis task: one root tracking issue titled after the project, holding the plan reference and a phase checklist. Every phase’s tasks block it; it closes when the project ships. It’s the fixed point you check progress against.

Pick a decomposition pattern. Three are in production use here:

Size the tasks. From my orchestrator’s telemetry, oversized tasks are the top cause of worker timeouts (roughly a fifth of completed tasks had timeout escalations before I learned this). The rules that fixed it:

• One concern, one file or module, description under ~3,000 characters, fewer than 8 acceptance criteria.
• Split anything with mixed concerns (CLI + engine + recovery) or internal sequential steps.
• The canonical split for a component: (1) skeleton/setup, (2) core implementation, (3) edge cases and polish — the first two high priority, the third can wait.

Write the full spec at creation time. Every task description follows:

## Context      — why this exists; link the plan section
## Design       — approach, files to touch, constraints
## Acceptance Criteria — each one verifiable by a command
## Notes        — gotchas, references

“Verifiable by a command” is the load-bearing phrase. Agents close their own tasks (more below), so acceptance criteria are what they validate against. “Works correctly” is useless; “cargo test decoder:: passes and the CLI round-trips sample.grib2” is executable.

Wire dependencies honestly. Only add an edge if the later task genuinely cannot start first — then check the dependency tree for cycles before launching anything. Circular dependencies don’t error; they silently starve the fleet. I lost days to workers idling on a queue that reported plenty of open tasks, all mutually blocked.

6. Run the fleet

Dispatch is a spectrum, and matching the mode to the work matters:

• Interactive session — exploratory work, or debugging the tasks themselves.
• Supervised batch — a dozen tasks fanned out to sub-agents while you watch. Good for a phase you want eyes on.
• Autonomous fleet — headless workers looping claim → execute → close for hours. This is where the volume happens.
• Metric marathon — one persistent loop chasing the plan’s north-star number. Never run this and fleet workers on the same repo simultaneously; they fight over working-tree state.

The orchestrator’s job is deliberately boring: pick the highest-priority unblocked task, claim it, build a prompt from the task spec, run the agent CLI, handle the outcome by an explicit table (success → verify → close; failure → release and retry; timeout → release and defer; repeated failure → escalate). Determinism in the loop is what makes a fleet debuggable.

Three rules that took incidents to learn:

Claims must be atomic. “List open tasks, then mark one claimed” is a race: with twenty workers I watched four claim the same task simultaneously, and label-based locking schemes all eventually lied. The fix is a claim that’s a single database transaction. If your queue can’t do that, your fleet size is effectively one.

One worker per repository. Workers on the same repo share a working tree; two agents with uncommitted state collide in ways that look like model stupidity but are just dirty checkouts. Scale across repos, not within one. (Worktree-per-worker isolation works too, at some setup cost.)

Agents own closure. My orchestrator originally parsed agent output to decide whether to close tasks — and failed silently, orphaning work. Now the prompt makes the agent do it: implement, commit with the task ID as a commit trailer, push, validate against the acceptance criteria, then close the task with a reason (“Implemented X in src/y.rs; tests clean”) or release it with a blocked label and an explanation. The orchestrator only verifies that a commit exists since dispatch — a cheap tripwire that catches agents that claim success while changing nothing.

On cost: route by task difficulty. Routine, well-specified tasks go to a cheap model tier; gnarly ones and anything that failed once go to a stronger tier. With per-task velocity stats you can even route by observed model performance per task type — but the cheap-by-default rule alone changes the economics of running twenty workers.

7. Watch it, and know the failure modes

The steady state is pleasantly dull: a queue draining, commits with task-ID trailers accumulating, workers going idle and exiting when nothing’s ready. The exceptions are where operational knowledge lives:

• “The fleet is starved” is usually false. Nearly every starvation alert I investigated had ready tasks in the queue — the real cause was a worker querying from the wrong directory, diagnostics corrupting parsed output, or a dependency cycle. Re-check the ready queue yourself before believing any alert.
• Break failure loops manually. A task that keeps failing will be redispatched forever by a naive loop — my record is one task dispatched 310 times before I noticed. Circuit-break it: after ~3 failures, stop, split the task into smaller children, and block the original until they land.
• Escalate to a human as a task. Things agents genuinely can’t decide become tracker items with a human label that page me (a Telegram bridge, in my case) and block their dependents. The queue itself is the escalation channel — no side-channel state.
• Treat the queue database like production data. Back it up, checkpoint it to the git-tracked export before running any repair tooling, and never let “repair” run against a stale checkpoint. I once wiped the fresh tasks in seven workspaces at once by running repairs in the wrong order. The queue is the fleet’s memory; losing it is losing the plot.

Steal this even without the tooling

If you take nothing else, these five transfer to any agent setup — including a single Claude Code session:

1. Docs-first scaffold. Agents are only as good as what they can read.
2. Review the plan before decomposing it. Fleets multiply plan defects.
3. Small tasks with command-verifiable acceptance criteria. The single biggest quality lever.
4. Agent-owned closure with validation. Make the agent prove its work against the criteria, and verify a commit exists.
5. Atomic claims and honest dependencies if you ever run more than one worker.

The tools will keep changing — the models certainly will — but “one reviewed plan, decomposed into small verifiable tasks, processed by workers with deterministic outcome handling” has survived every model upgrade and every incident so far. That’s the part I’d bet on.