Stop Chasing Perfect Prompts: Build Systems Instead

In 2024, we spent an embarrassing amount of time on a single scoring instruction. The Job Change Intent Scorer accepted an optional field, new_company_hint, passed in from a webhook payload. The system prompt mentioned the field existed. It did not say what to do with it. The reasoning model treated it as weak background noise instead of strong corroborating evidence. Confidence scores sat at 0.2 to 0.3 when they should have cleared 0.5. We added four lines: what the hint represents, how to cross-reference it against web search results, how a confirmed match shifts the threshold, and what to do when no hint arrives. Scores corrected immediately.

That incident crystallized something we had been circling for months. The problem was never the phrasing of the instruction. The problem was that we had no system for specifying scoring rules. We were writing instructions the way you'd write a note to a colleague, trusting that shared context would fill the gaps. LLMs do not share your context. They infer nothing from field names. Every rule has to be spelled out explicitly, and the only way to catch the gaps is to build a process that forces you to enumerate them before you ship.

This is the shift McKinsey documented in their State of AI in 2024 report: organizations are moving beyond experimentation with individual AI applications to building integrated AI systems and workflows that deliver sustained business value. The practitioners who got there first stopped asking "how do I write a better instruction?" and started asking "how do I build a process that catches bad instructions before they reach production?"

The early phase of working with LLMs in n8n pipelines felt like a vocabulary problem. Get the wording right, get the output you want. We iterated on phrasing. We tested synonyms. We moved sentences around. Some of it worked, in the sense that a specific input would produce a specific output. None of it was reliable across variations.

Three failure modes kept appearing.

First: polite instructions do not function as system constraints. Telling a reasoning model to "prefer" a format, or to "try to" stay within a token budget, produces inconsistent behavior. The model treats preference language as optional. When we needed a hard constraint, we had to write it as a hard constraint, with explicit consequences for violation described in the same instruction block. Soft language is not a system.

Second: output variance is real and it compounds. We ran the same pipeline with the same input across multiple sessions and got different confidence scores. Not wildly different, but different enough to matter for downstream routing logic. A system that routes leads based on a 0.5 threshold behaves differently when the same lead scores 0.48 on Tuesday and 0.53 on Thursday. Chasing the "right" instruction does not fix this. Building a feedback loop that catches and flags variance does.

Third, and this one cost us two hours: synthetic test data passes pipelines that real data breaks. We used synthetic IDs during development. They moved through every node cleanly. When we switched to real records, the write step failed. We blamed the wrong service for two hours before finding the actual cause. The lesson is not about test data hygiene in the abstract. It is that a system without real-data validation at every stage will surface bugs in production that your instruction-tweaking phase never touched.

The honest caveat here: building systems takes longer upfront. If you need a one-off answer from an LLM today, a well-crafted single instruction is faster than designing a pipeline. Systems thinking pays off when you need the same process to run reliably across different inputs, different team members, or different weeks. For genuinely one-time tasks, the overhead is not worth it. Know which situation you are in before you commit to either approach.

The shift from instruction-tuning to system design is not philosophical. It shows up in specific, concrete decisions.

Enumerate every rule the model needs to apply. The new_company_hint incident was not a prompt quality problem. It was a specification problem. We had not written down what the field meant, how it should interact with other evidence, or what the scoring threshold should be when it was present versus absent. Once we wrote those four lines, the behavior corrected. The instruction was not clever. It was complete. Completeness is a system property, not a writing skill.

Build the error surface before you build the happy path. One of the most useful things we did across our n8n pipelines was add dead letter queues before we considered the feature complete. Not as an afterthought. Not as a "we'll add it later" item. The queue is what tells you where your system is actually failing. Without it, you see clean runs in testing and mysterious failures in production. With it, you see a 41% dead letter rate drop to 11% after a single parser fix, because the queue made the failure visible. That number came from fixing JSON extraction depth in one pipeline. The fix took an afternoon. Finding it without the queue would have taken weeks.

Separate configuration from logic. We retrofitted a single configuration loader across nine products after building the first few with hardcoded values. Every time we needed to change a threshold, a model parameter, or an API endpoint, we were editing logic files. That is a system design failure. Configuration belongs in one place. Logic reads from it. This is not a novel idea in software engineering, but it took us longer than it should have to apply it consistently to LLM pipelines.

Chain outputs, not instructions. The biggest productivity gain we found was not in any single instruction. It was in designing pipelines where the output of one stage becomes the structured input of the next, with explicit contracts between them. When a web search result feeds a scoring node, the scoring node should not have to guess what format the search result is in. The contract specifies it. This is what ForgeWorkflows calls agentic logic: not a single model doing everything, but discrete components with defined handoff points. Each component can be tested, replaced, and improved independently.

The concerns veteran PMs raise about AI-first program management often trace back to this exact gap. When AI behavior is unpredictable, it is usually because the system has no contracts between stages, no validation at handoff points, and no mechanism for catching failures before they propagate. The answer is not better instructions at the start. It is better structure throughout.

Measure actual costs, not theoretical ones. We found that memory-based rate estimates were wrong by 30 to 50 percent when we measured actual pipeline runs. Web search tokens ran at roughly twice our theoretical estimates. The search fee itself was only about one-third of the real cost; tokens were two-thirds. None of this is visible if you are optimizing a single instruction in isolation. It only becomes visible when you instrument a system and measure what actually happens.

This connects to a broader point about the current state of AI tooling as of mid-2026: the platforms have matured enough that instrumentation is possible. n8n exposes execution data. APIs return token counts. Dead letter queues capture failure payloads. The infrastructure for system-level measurement exists. The practitioners who are pulling ahead are the ones using it.

Start with the scoring rubric, not the instruction text. Every time we built a pipeline that involved an LLM making a judgment, we should have written the scoring rubric first, as a separate document, before writing a single line of the system instruction. The rubric forces you to enumerate every case: what counts as strong evidence, what counts as weak evidence, what happens when evidence conflicts, what the default behavior is when a field is absent. The instruction is then a translation of the rubric into a form the model can consume. We did this in the wrong order on several builds and paid for it in debugging time.

Treat the first real-data run as a separate test phase, not a deployment. Synthetic data is useful for structural testing. It is not useful for catching the edge cases that real records introduce: missing fields, rebranded companies, contacts with no activity history, webhook payloads with unexpected nesting. We would now build a dedicated real-data validation stage into every pipeline before considering it ready for production use. The two hours we lost to synthetic ID failures would have been thirty minutes with this in place.

Build the feedback loop before you optimize anything. The temptation is to get the happy path working and then add observability later. We would reverse that order. Instrument first. Measure baseline behavior. Then optimize against the measurement. Without a baseline, you do not know whether your changes are improvements or just different. The 41% to 11% dead letter improvement was only possible because we had a queue measuring the failure rate. Without it, we would have been guessing.