Prompt Templates & Chaining

A prompt template is not just a string with blanks. It is a reusable contract that defines the model role, task instructions, allowed inputs, examples, output schema, and failure behavior. The named placeholders should be explicit, such as customer_note, policy_context, or target_audience, so reviewers can see which data enters the prompt and where it appears.

Production templates should have ids and versions, for example support_triage:v3. The version should identify the prompt text, examples, model family assumptions, parameter defaults, and evaluation set. When quality changes, the team can compare versions instead of guessing which wording shipped to users.

The most common template bug is mixing untrusted user input into the same region as instructions. If a customer note is inserted directly after a sentence like Follow these rules, the model may treat malicious or accidental instructions inside the note as part of the task. The safer pattern is to keep stable instructions in the system or instruction section and place user-controlled text in a labeled data section.

Escaping and labeling help. Convert or escape delimiters that appear inside user text, wrap data in a clearly labeled block, and instruct the model that the block is data, not commands. This is not a complete prompt injection defense, but it prevents the application from accidentally promoting user text into the instruction layer.

Few-shot examples are often part of the template rather than ad hoc text added by a caller. A classification template might include one clear positive example, one negative example, and one ambiguous example where the correct answer is unknown. An extraction template might show exactly how missing fields should be represented.

Examples should be treated as maintained assets. They consume tokens, they can bias the model, and they can become stale when policy changes. A useful registry records why each example exists and keeps an evaluation case that proves the example still matters.

A prompt chain breaks work into sequential model calls. Instead of asking one prompt to read a long complaint, infer product area, extract facts, draft a response, and check tone, the application might run extract -> classify -> draft -> validate. Each step has a narrower prompt and a smaller success condition.

This improves debuggability because the application can inspect the intermediate state. If the final answer is wrong, you can see whether extraction failed, routing chose the wrong specialist, or the drafting prompt ignored the state. The tradeoff is more orchestration, more network round trips, and more places for retries or validation failures.

Sequential chains are the simplest pattern: output from step one becomes input to step two. Routing chains classify the request first, then dispatch to a specialized prompt for billing, technical support, legal review, or another branch. Map-reduce chains run the same prompt over chunks, then combine the partial outputs. Iterative refinement chains generate an answer, critique it, and revise it until a quality threshold or retry limit is reached.

These patterns are framework-independent. LangChain-style libraries, prompt-flow tools, and custom orchestration code all express the same underlying graph: prompt templates, model calls, validators, state, and transitions.

A fixed chain has a known graph before the request starts. It may branch, but the branches are predefined. An agent goes further: it can choose tools, decide the next step based on observations, maintain memory, and loop until a goal or budget limit is reached.

The boundary matters in interviews. Prompt chaining is usually easier to test because the possible paths are constrained. Agents are more flexible but require stronger guardrails, budgets, tool permissions, and observability. If your chain starts planning its own steps, calling tools, and deciding when it is done, connect the design to agent lessons and discuss agent safety explicitly.

A prompt registry stores more than raw text. It should track template id, semantic version, owner, allowed variables, examples, model compatibility, parameter defaults, eval set, changelog, and rollout status. This turns prompt changes into reviewable artifacts rather than invisible string edits.

The registry also enables controlled rollout. A team can run offline evals, shadow the new version on real traffic, send a small percentage of users to the new prompt, compare quality and cost, and roll back if validation failures increase.

A single large prompt is attractive because it has one model call, one context, and lower orchestration complexity. It can be the right design for short, low-risk transformations where intermediate state does not matter. The downside is that debugging is hard: extraction, reasoning, style, and validation are all tangled together.

A chain adds overhead but gives checkpoints. You can validate extraction before drafting, route before generation, and retry only the failed step. This is usually better for automation, regulated workflows, long documents, or any feature where a wrong intermediate assumption can poison the final answer.

Passing the full transcript from one step to the next is easy, but it increases tokens and preserves irrelevant noise. A better pattern is to pass a compact state object: extracted facts, selected route, citations, confidence, and validation status. The next prompt should receive only the state it needs plus any required source text.

Structured state also makes caching and replay easier. If the extraction state for a document chunk is stable, downstream prompts can be re-run without paying for extraction again.

Chains can fail quietly because each step makes the next step sound more authoritative. If extraction invents a field, the drafting step may confidently use it. Validators should check syntax, schema, allowed labels, citation presence, policy constraints, and uncertainty behavior before the output becomes input for another model call.

For important workflows, validation should include semantic checks too: does the summary cite the right chunk, does the answer use only approved fields, and does confidence drop when evidence is missing. Some checks are deterministic; others may use a smaller critique model, but those critique calls need their own budgets and stop rules.

Many small calls can increase wall-clock latency because each call has network overhead, queue time, and generation time. They can also increase total tokens if each step repeats instructions and context. Parallel map steps can reduce wall-clock time for long documents, but they may increase peak rate-limit usage.

Caching helps when a step is deterministic or repeated: classification for the same input, chunk summaries for unchanged documents, rendered templates for stable configuration, or final outputs for identical requests. Cache keys should include template version, model version, parameters, normalized input, and relevant safety policy version so stale prompt behavior is not reused accidentally.

Measure latency, input tokens, output tokens, cache hits, retries, validation failures, route distribution, and model errors for each step. A chain-level average hides the real bottleneck. Step-level logs let teams find whether a regression came from routing, extraction, drafting, validation, or a prompt rollout.

Retry only failures that are likely transient or repairable. A malformed JSON response may get one repair attempt; a policy refusal should not loop forever. Chain steps that write to external systems must be idempotent or separated from model generation so a retry does not create duplicate side effects.

Templates should keep instructions and untrusted data separate, but that is only one layer. Use server-side authorization, tool allowlists, retrieval filtering, output validation, and audit logs. A chain can amplify injection if early steps summarize malicious text as if it were trusted policy, so each step must preserve source labels and trust boundaries.

A four-step chain can hit provider rate limits faster than one call, especially when map steps run in parallel. Enforce per-request token budgets, maximum branches, maximum refinement rounds, timeout budgets, and graceful degradation. Budget failures should return a controlled fallback rather than a partial internal trace.

This example uses named placeholders, a tiny template registry, safe interpolation into data sections, and a fixed two-step chain. The first prompt extracts state from a customer note. The second prompt drafts a response from the validated extraction output. The mock model calls make the orchestration visible without requiring provider credentials.

A refinement loop can improve quality, but only if it has a measurable stop condition. This example stops when the critique says the draft is acceptable or when the maximum number of rounds is reached.