RAG, loops, graphs: four decisions, not four products

Published 22 July 2026 · updated 16 August 2026

One supplier offers “an assistant that answers from your documents”. A second talks about RAG. A third sells a “multi-agent graph”. A fourth no longer writes prompts, but loops. All four describe real work, and nothing lets you compare them: the words don’t sit at the same level. They name four decisions that get taken one after the other. And naming the decisions is enough to make the offers comparable.

Four practical layers, not a universal standard

I use this taxonomy to compare offers. Other teams put memory, tools or orchestration under different labels; what matters is to make the responsibilities explicit.

The context: what the model has in front of it when it answers. RAG lives here, and so does context engineering
The loop: what the system does after the answer. Verify, correct, try again, and go round until a stated condition stops the cycle
The graph: who is allowed to run next. Sequence, parallel, a mandatory pass through a human
The harness: everything else. Tools, permissions, memory, the log, recovery after an incident

The rest of the article takes the four decisions one at a time.

RAG: the most overused term, and still a real challenge

The principle takes two steps: find the useful passages in your documents, then hand the passages to the model with the question. The demonstration on the Applied AI page works this way, and it shows the extracts it read before answering.

Retrieval has improved a lot, but it stays a problem: extraction, chunking, metadata, query construction, ranking, permissions and evaluation all affect quality. Once that baseline works, much of the difficulty turns administrative: your documents live in a mailbox, in a document system, in the accounting package, and each place has its own permissions. Who may see what, which version prevails, what happens to an answer when someone corrects the source the next day. An assistant that answers from an HR document doesn’t have a model problem. It has a permissions problem.

The loop: on evidence, never on confidence

The author should not be the only judge

A model that rereads its own output may improve it, but it also shares the blind spots of the first answer. For output that matters, separate production and evaluation where you can, then add deterministic checks or human review. A second model doesn’t guarantee truth either.

The judge needs a source of truth

For code, the tests. For a text, the original document. If nobody can name the source of truth, the system has no verification, however confident the vocabulary sounds.

So the practical rule: don’t loop on confidence alone; loop on evidence. “The agent believes it has finished” isn’t a stopping condition; “the amounts match the statement” is. And the rule lives in the code, not in the prompt: a model that has already failed three times readily persuades itself that the fourth version is good enough.

When durable execution becomes useful

Three needs justify a workflow engine, a state machine or an execution graph, without requiring one particular representation:

Surviving a stop: the server restarts and the work resumes where it stopped, instead of paying for every call again
Waiting for approval in the middle: execution halts, waits three days for a signature, then continues
Replaying afterwards: “why was this invoice rejected on 12 March?” needs the state of the system at that moment, not the final answer

The reverse matters just as much: formal orchestration earns its keep when steps and relationships recur. Until you understand the process, a simple loop may explore better than a premature plan. And loops, state machines and graphs can live together in one system.

The graph: breadth, not judgement

A graph is the plan of the work, drawn: the tasks, and who waits for what. Two words carry the whole vocabulary. A box is a node, meaning one bounded task with something going in and something coming out. An arrow between two boxes means the second task needs what the first produced, so it waits. Nodes do the work, arrows carry the results. The budget rule fits in one sentence: a graph gives you breadth, not judgement. When each step needs the result of the step before it, running them in parallel only adds calls. When the work splits into tasks that never read each other, parallel pays immediately.

The test costs an evening: draw your chain, and for each “and then”, ask whether the next task really needs the result of the one before. Many chains then turn out to carry arrows nobody needs.

What makes a task usable as a node is a contract: one bounded job, a defined input, a defined output shape. A node whose output is free prose is a node only a human can read, and the next task is left guessing where the answer starts. A fixed output shape is what lets one node consume another with nobody in the middle.

The width has a price, and the clearest public number comes from Bun. Its creator ported 535,496 lines of Zig to Rust in eleven days with roughly fifty coordinated workflows, peaking near sixty-four agents at once, for an estimated $165,000 of model usage, and every change passed two independent reviews before it was kept. The objection arrived with the result: the creator of Zig called the output unreviewed slop. Both halves belong in the same sentence. A fleet reaches a scale no single context holds, and it spends the budget in the background whether or not the task deserved it.

The anchors a graph needs

A finished graph is where the loop’s failure returns wearing better clothes. Paired checkers, audit nodes, meta-nodes tuning the other nodes, and every one of them reading a report produced by the same system that produced the numbers. The audit agrees with the finance figures, both having come from the same place. Everything is consistent and nothing is verified.

That version fails later than a single loop, costs more on the way down, and shows more green lights while it does it.

The shape of the graph delivers no truth on its own. A graph needs anchors, meaning nodes nobody can argue with: a test that actually ran rather than one that should pass, money that arrived in the account, a customer who renewed.

Some rules also have to be frozen, in particular the ones an optimiser would be tempted to weaken, which are the ones it bends in order to win. A graph is only as honest as the parts of it that refuse to move.

Two different graphs under the same word

The execution graph above describes the work. The knowledge graph describes your data: every piece of information becomes an explicit relationship between two entities, “this contract binds this supplier to this municipality”. GraphRAG names that family, and Microsoft’s implementation is the open reference in it.

The difference shows in the shape of the questions. Vector search excels at finding what was written on a subject. Hybrid search, metadata and aggregation can also answer some questions that span documents.

A graph becomes attractive when the relationships have to stay explicit and recurring, and so queryable: which contracts has a supplier won over five years, and who sat on the committee each time? The answer spans several sources. A knowledge graph materialises it, at the cost of extraction, normalisation, schema design, provenance and maintenance.

The paper LLM-assisted Knowledge Graph Engineering reports experiments that used ChatGPT to help build knowledge graphs; it doesn’t establish the ongoing cost of every graph system. Compare the total cost and the answer quality on the real questions. There’s no universal hundred-to-one ratio between vector retrieval and a graph.

Where your signature goes

The one decision you can’t delegate remains. Human approval sits before the irreversible: sending, paying, posting an entry. Put it everywhere and you are the bottleneck; put it nowhere and nobody is watching. For a municipality the formula already exists: it’s the milestone of the HERMES method, the moment the authority that pays authorises the next step, or doesn’t.

The vocabulary will change again before the end of the year; the four decisions will stay. The parts that do the work are the least spectacular ones, and they never fail loudly. They fail the day an error passes in silence, forty times in a row.

A minimal decision record

For each use case, write down the shape of the question, the evidence you require, the cost of an error, how often the relationships recur, the latency, the volume, the permissions, the recovery and the observability. Start with the simplest architecture that reaches the measured threshold. Add a loop, durable state or a graph only when the test shows what it buys.

The status of this taxonomy

I reviewed the four layers on 14 August 2026 and added the Bun figures on 26 August 2026. The four layers are a proposed taxonomy. Measure cost and performance on the actual corpus, the actual questions and the actual permissions.