Phase 04 · Prompt engineering Core

Chain of thought & reasoning models

Getting the model to work through a problem step-by-step before answering - either by prompting it to, or by using models trained to reason internally.

In plain terms

Ask someone '17 x 24?' and demand an instant answer: they'll guess. Give them scratch paper and they'll be right. LLMs are the same - 'think step by step, then answer' gives the model scratch paper, and accuracy on math, logic and multi-step problems jumps.

Why it matters

Models generate one token at a time with fixed compute per token; a hard problem answered in one leap exceeds that budget. Reasoning tokens ARE the model's computation - more relevant tokens before the answer means more thinking. This insight was so powerful it got baked into whole model families (OpenAI o-series, Claude extended thinking, DeepSeek-R1) that generate hidden reasoning before every reply.

How it works

Classic version: instruct 'Think step by step inside <thinking> tags, then give the final answer in <answer> tags' - then parse only the answer. Reasoning models automate this: you pay for invisible 'thinking tokens' and get the polished conclusion; you often control a thinking budget (none / low / high). Either way the tradeoff is identical: accuracy vs. latency and cost.

When you use it

Math, logic, planning, code debugging, evaluating tradeoffs, multi-step extraction. Skip it for lookups, casual chat and simple classification - you'd pay thinking tax for nothing.

Common mistakes

  • Using an expensive reasoning model for everything, including 'what's your refund policy'.
  • Asking for step-by-step but showing users the whole ramble - separate reasoning from the final answer.
  • Trusting the reasoning text as a faithful account of the model's process - it's useful, not guaranteed introspection.

Best practices

  • Structure it: reasoning in one tagged block, answer in another, parse the answer only.
  • For reasoning models, start with low thinking budgets and raise until quality plateaus.
  • Add 'check your answer against the constraints before finalizing' - cheap self-verification helps.

Try it yourself

Give a mid-tier model a tricky logic puzzle with 'answer with just the final answer', then again with step-by-step thinking. Then try a reasoning model. Compare all three on accuracy, latency and token cost.

Resources

Related topics

Zero-shot & few-shot prompting Zero-shot = just ask. Few-shot = show 2-5 worked examples in the prompt so the model imitates the pattern instead of guessing your intent. What is an agent, really An agent is an LLM wrapped in a loop that can choose actions and use tools. Choosing the right model Picking a model is balancing four axes - quality, cost, latency, context size - for your specific task, not finding 'the best model'.