Workflow First, Agent Second: Choosing the Least Autonomy That Works

In an agent, the model decides what happens next. In a workflow, your code decides, and the model fills in steps. That one distinction settles most architecture questions about LLM systems, and it’s worth settling before choosing a framework.

Anthropic’s Building effective agents draws the line this way and recommends the simplest option that works. It’s good advice, and teams skip it often, because “agent” sounds like the more capable choice. It’s the more autonomous choice. Autonomy costs you predictability, testability and money, and it widens the set of actions a bug can reach.

A quick test: can you draw the flowchart?

Before writing code, try to draw the process as a flowchart with a fixed set of boxes.

• If you can, even with branches, build a workflow. The model classifies, extracts, drafts or checks; your code routes.
• If the number and order of steps depends on what the model discovers (debugging a failing test, researching an open question, working through an unfamiliar codebase), you have an agent problem.

Most business processes pass the flowchart test: triaging support tickets, extracting fields from invoices, drafting a reply for approval, summarising a document into a template. People build them as agents anyway and then spend weeks adding guardrails to stop the agent doing things a workflow would never have attempted.

The five workflow patterns

These cover most production LLM systems that aren’t open-ended agents.

Prompt chaining. A fixed sequence of calls, each consuming the previous output, with programmatic checks between steps. Use it when a task splits cleanly: outline, check the outline, then write. The checks between steps are where the reliability comes from.

Routing. One call classifies the input, and code sends it to a specialised prompt or model. It also lets you send easy inputs to a smaller, cheaper model. Most of the value is in the router’s label set, so design it from real traffic.

Parallelization. Run independent calls at once and combine the results. That can mean splitting a document into sections, or voting: asking the same question several times and taking the majority or the strictest answer. Voting is a cheap way to make a classifier more conservative.

Orchestrator–workers. One call breaks the task into subtasks at runtime, workers handle them, and a final step combines the results. This is the first pattern where the model decides the shape of the work, so it sits closest to agent territory. Cap the number of subtasks in code.

Evaluator–optimizer. One call produces, another critiques against explicit criteria, and the loop repeats up to a fixed limit. It works when you can write the criteria down, as with a style guide, a schema or a test suite.

Enforcing fan-out and retry limits bounds call count. Cost also needs token limits, and latency needs timeouts; a bounded number of calls alone bounds neither. Your code sees every intermediate result, so each step can be unit-tested with recorded inputs.

What changes when you do need an agent

Sometimes the flowchart can’t be drawn, and an agent (a loop where the model picks tools until it decides it’s done) is the right tool. Coding assistants are the obvious case. When you choose one, several things change at once.

Cost becomes a distribution rather than a number. A workflow costs roughly N calls. An agent costs whatever the loop takes. Anthropic reported that its multi-agent research system used about 15× the tokens of a chat interaction, and single agents about 4×. Put hard limits in code: maximum iterations, maximum tokens per session, and wall-clock time.

Testing moves from outputs to trajectories. You can’t assert the exact sequence of tool calls, because it changes run to run. You can assert properties: the agent never called delete_*, it called verify before submit, it finished within 20 steps, and the final state matches. That’s the approach in the evals post.

Permissions become the real boundary. In a workflow your code decides which actions are possible at each step. In an agent, tools may be exposed on several turns unless the harness narrows them by state, and the model decides. Whatever the agent’s credentials allow is, in practice, what the agent can do, including after a prompt injection. Scope the credentials to the smallest job, and keep irreversible actions behind a separate approval step.

Observability is no longer optional. When something goes wrong in a workflow, you know which step failed. In an agent you need the full trace (each model call, tool call, input and output) to understand what happened at all.

A hybrid is often the right answer

The choice isn’t binary. A common, sturdy shape is a workflow with one agentic step:

ticket arrives
  -> classify (model, fixed labels)             # routing
  -> if label == "billing_dispute":
       investigate (agent, read-only tools,     # the only open-ended part
                    max 15 steps, 60s budget)
  -> draft reply (model, template)              # chaining
  -> human approves                             # gate
  -> send (code)                                # the only write action

The agent gets read-only tools and a budget. The one action with side effects (sending the reply) belongs to code, behind a human. The open-ended reasoning sits where it’s needed and nowhere else.

Where this stops

This is a way to choose an architecture, not a guarantee that a workflow will work. Workflows fail too: a router with bad labels sends everything to the wrong prompt, and a chain with no checks between steps compounds errors. The advantage is that those failures happen at known points you can test and measure. Start there, and add autonomy only where a real task needed it and a workflow couldn’t handle it.