rban.devFull-stack engineer

RAG systems fail at the workflow edges before they fail at retrieval

Apr 6, 20266 min readBy Rahul Ban

I built a self-healing deployment pipeline for an AI agent last month. The idea was simple: after every deploy, it detects regressions, triages whether the change caused them, and kicks off an agent to open a PR with a fix. No manual intervention needed until review time.

What surprised me was what broke. The regressions we caught weren't retrieval quality drops. They were workflow edge failures. The system would find the right document, then fail to route it. Or time out on a handoff between components. Or silently swallow an error and return a plausible-sounding nothing.

I usually care more about retrieval metrics. Recall@k, precision, MRR. Those numbers feel concrete. But now I care more about what happens after retrieval succeeds. That's where the system actually delivers value, and that's where it fails in production.

The Real Problem

Measuring retrieval quality without workflow observability is like measuring typing speed for a programmer who can't debug. You can type the perfect code, but if you can't trace why it fails, you're just fast at building broken things.

The tension is between shipping AI features that look good in demos versus building systems that are trustworthy at 2 AM when they break. It's the difference between optimizing for a notebook demo and optimizing for a pager that doesn't go off.

The human cost is real. Debugging black boxes eats cycles. Maintaining systems with clear edges is work, but it's predictable work. You know what you're dealing with.

Where Teams Usually Get It Wrong

Most teams optimize for the demo. They spend weeks tuning retrieval, building fancy rerankers, and crafting perfect prompts. The eval suite passes. The notebook looks great.

Then it hits production. A user asks a question that triggers a code path the demo didn't cover. The retrieval works. The context is perfect. But the workflow fails because of a timeout, a routing error, or a state mismatch.

The failure mode is opaque. Was it the model? The tool call? The handoff? You don't know because you only instrumented retrieval. Now you're debugging a black box at 2 AM, and your team's trust in the system evaporates.

I've been here. The part I do not trust yet is any system where the only metrics are retrieval-focused. Those are vanity metrics for production systems.

What Actually Breaks

The failures I saw fell into three buckets:

Routing failures. The system retrieves the right document, but the agent fails to route it to the correct downstream tool. The context is perfect, but the workflow logic is wrong. This looks like a retrieval failure from the outside, but it's not.

Handoff timeouts. A component finishes its work, but the next component is busy or stuck. The handoff times out. The system doesn't crash, but it doesn't progress either. It just hangs, consuming resources and user patience.

Silent error swallowing. An exception is caught, logged somewhere invisible, and the system returns a default response. The user gets something, but it's wrong. The logs don't show the error because the error handling itself is broken.

These failures share a pattern: they happen at the edges between components. Retrieval is a component. The agent loop is a component. The tool calls are components. The edges between them are where the system fails.

A Better Working Shape

I keep coming back to boundaries. Clear, explicit boundaries between components. The Model Context Protocol specification shows how to define these boundaries. It treats context as a first-class citizen, with explicit contracts between providers and consumers.

The practical question for me is: where do you draw the lines? I draw them at every handoff. Between retrieval and the agent. Between the agent and tools. Between the workflow and external services. Each boundary has a contract. Each contract has a timeout. Each timeout has a fallback.

LangGraph's approach to workflow orchestration treats state transitions as first-class citizens. You don't just call functions. You transition state. The state is inspectable. The transitions are explicit. When something fails, you know exactly where you were in the workflow.

This is where my earlier post on AI systems needing edges becomes practical. Edges aren't just conceptual. They're timeouts, contracts, and state machines. They're the difference between a system that fails silently and one that fails noisily in a way you can fix.

The tradeoff is upfront design versus debugging time. You can spend weeks designing boundaries, or you can spend weeks debugging black boxes. I'll take the design time. It's cheaper.

Building for the 2 AM Page

Operational trust comes from predictable behavior. Not perfect behavior—predictable behavior. You need to know what will happen when things break.

The self-healing pipeline I built follows patterns from LangChain's production agents work. It doesn't try to fix everything automatically. It detects regressions, triages them, and proposes fixes. A human reviews the PR. The automation stops before it gets clever.

This is the key: automation that augments human judgment, not replaces it. The system can say "this workflow edge failed 10 times in a row, here's a proposed fix." But it can't decide whether the fix is correct. That's the human's job.

Azure AI Foundry's deployment patterns emphasize this. They don't just deploy models. They deploy systems with monitoring, fallback policies, and rollback strategies. The model is the easy part. The system around it is what matters.

The advantage is trust. When your team knows the system will fail in predictable ways, they trust it more. They use it more. They build on it. That's the real upside.

What to Watch in Practice

Here's what I look for now:

Define boundaries first. Before you write retrieval code, define the contracts between components. What does each component expect? What does it return? What happens on timeout? Write it down. Make it testable.

Instrument state transitions. Every handoff should be logged. Not just "tool called," but "state changed from X to Y." When it fails, you know where you were. This is more useful than any retrieval metric.

Test failure injection. Your eval suite should include workflow failures. What happens when retrieval times out? When a tool throws an exception? When the agent loop runs too long? If you can't answer these, your system isn't ready for production.

Measure workflow completion, not just retrieval. Track the full journey: query → retrieval → reasoning → action → response. Where does it drop off? That's your failure point.

The part I do not trust yet is fully automated fixes without human review. The self-healing pipeline can propose fixes, but a human must approve them. I would rather have a system that pages me with a clear error than one that silently "fixes" itself into a worse state.

What I do not trust yet is any system where the only observability is retrieval metrics. Those are table stakes. The real observability is in the workflow edges.

Closing

RAG systems fail at the workflow edges before they fail at retrieval. That's the pattern I see. That's the pattern I'm building for.

The shift is from demo polish to operational trust. From optimizing for the notebook to optimizing for the pager. From black boxes to clear edges.

I care more about what happens after retrieval succeeds. That's where the value is. That's where the system either delivers or fails.

Build systems with explicit boundaries. Make state transitions inspectable. Automate recovery, but keep humans in the loop. That's how you earn trust. That's how you ship something that works at 2 AM.

The rest is just typing.

Resources Worth Reading

Related Reading