Phase 03 · Talking to models: the API layer Core

System prompts

The standing instructions that define the model's job, personality, rules and boundaries - set by you, invisible to the user.

In plain terms

It's the employee handbook you hand the model before its shift: 'You are a support agent for Acme. Be brief. Never discuss competitors. If unsure, say so.' The customer never sees the handbook, but every answer is shaped by it.

Why it matters

User messages change every turn; your product's identity shouldn't. The system prompt separates what the app is (yours, stable, trusted more by the model) from what the user says (variable, untrusted).

How it works

A well-built system prompt covers: role ('You are...'), task and audience, tone, output format, hard rules ('never...', 'always...'), how to handle unknowns and edge cases, and a few examples if format matters. Models are trained to weight system instructions above user ones - though not absolutely (-> prompt injection).

When you use it

Every production feature has one. Even one sentence ('You extract dates from emails and reply with only an ISO date') dramatically stabilizes output.

Common mistakes

  • Writing 'You are a helpful assistant' and stopping - the defaults are generic because the instructions were.
  • One 3,000-word wall of contradictory rules - models follow clear, short, ordered rules far better.
  • Putting per-request data in the system prompt (breaks prompt caching; keep it stable).

Best practices

  • Tell it what TO do, not only what to avoid; positive instructions beat prohibitions.
  • Specify the failure mode: 'If the answer isn't in the document, say I don't know' - this one line kills most hallucinations.
  • Version your system prompts like code; a one-word change can shift behavior.

Try it yourself

Build the same Q&A call with three system prompts - none, one sentence, one paragraph with rules and examples. Feed all three the same tricky question and compare.

Resources

Related topics

Chat & Messages APIs: roles and turns The standard request format for every LLM: a list of messages, each with a role (system / user / assistant), sent fresh on every call. Role, context & constraints The workhorse prompt structure: who the model is (role), what it needs to know (context), and the exact boundaries of an acceptable answer (constraints). Prompt injection and AI security Prompt injection is malicious instruction text hidden inside data the model reads. Prompt caching Providers cache the unchanged prefix of your prompt so repeated calls reprocess it at ~10% of the price and much lower latency. LangChain core: messages, prompts and chains LangChain is a framework for composing LLM apps from Chat Model calls, Prompt Template objects, messages, parsers, tools and retrievers.