Glossary · Foundations

Prompt Engineering

Prompt engineering is the practice of writing instructions to LLMs to get reliable, structured output. Definition, techniques, and when to stop optimizing.

By Kadin Nestler · May 28, 2026 · Updated May 28, 2026
On this page

Core techniques

  • Role and context: tell the model who it is and what the task is in plain language.
  • Few-shot examples: show 2-5 input/output pairs so the model imitates the pattern.
  • Chain-of-thought: instruct the model to think step by step before answering. Improves reasoning accuracy on math, logic, and multi-step tasks.
  • Structured output: specify JSON schemas, XML tags, or markdown sections the model must produce.
  • Guardrails: explicit "if you do not know, say so" and "do not return information outside these documents."

Anthropic-specific prompt patterns

Anthropic's prompt engineering guide recommends XML tags (<document>, <instructions>, <example>) to delimit sections — Claude was trained to respect them. Long-context prompts work better when the question is at the end after the context, not the beginning. Prefilling the assistant turn ("Sure, here is the JSON: {") aggressively constrains the output format.

When to stop prompt engineering and switch tactics

  • If the prompt is over 3K tokens and still unreliable, you probably need RAG, not a longer prompt.
  • If the same prompt fails on 20% of inputs and edge cases are predictable, you need an evaluation harness and a fine-tune, not more clever wording.
  • If you need millisecond latency, you need a smaller model and a simpler prompt, not a more sophisticated one.
  • If the task requires real-time data, you need tool use, not a better prompt.

The 2026 reality

Frontier models in 2026 follow plain instructions well enough that elaborate prompt tricks matter less than they did in 2023. The leverage has shifted to evaluation (testing prompts against held-out examples), versioning (storing prompts like code), and observability (logging every prompt and output). Promptlayer, LangSmith, and Anthropic's own prompt console manage this lifecycle.

What it means for your business

If your vendor talks about prompt engineering as their secret sauce, push back. Prompts are the easy part. Reliable evals, version control, and monitoring are the hard parts and the ones that actually keep an AI workflow running in production.

  • Large Language Model (LLM) — A Large Language Model is a transformer-based neural network trained on trillions of tokens to predict the next token. Definition, key models, and business use.
  • AI Evaluation — AI evaluation is how you measure whether an AI system actually works. Definition, methods, and why evals are the bottleneck in production AI.
  • Fine-Tuning — Fine-tuning adapts a pre-trained LLM to a narrow task by training it further on labeled examples. Definition, cost, and when it beats prompting.
  • Claude Sonnet — Claude Sonnet is Anthropic's balanced model — strong reasoning, lower cost than Opus, faster latency. Definition, pricing, and use cases.
  • Tool Use — Tool use is when an LLM calls external APIs, databases, or code on its own. Definition, function calling, and how it powers AI agents.
← Back to the full glossary