Prompt Engineering That Actually Works

The right mindset

Prompt engineering has a reputation for secret tricks: special phrases, threats, tips, magic words. Most of that is folklore. Modern models are trained to follow instructions, so the biggest gains come from something less exciting: communicating clearly.

A useful test: imagine handing your prompt to a smart new colleague who knows nothing about your project. Would they have enough information to do an excellent job? If they'd need to ask five clarifying questions, so does the model — except the model can't ask; it guesses.

1. Be specific about the task and the goal

Vague requests get generic answers. State what you want, who it's for, and what "good" looks like.

Write about our new feature.

Write a 150-word announcement of our new "Export to PDF"
feature for our customer newsletter. Readers are small-business
owners who aren't technical. Focus on the time it saves them,
use a friendly tone, and end with one sentence telling them
where to find it (File → Export).

It also helps to explain why. "Keep it short because people read this on their phones" gives the model a principle it can apply to decisions you didn't anticipate, which a bare "keep it short" doesn't.

2. Provide the context

The model only knows what's in its training data and in your prompt. Include the relevant facts, documents, constraints and background directly. If you want it to answer from a document, paste the document. If there's a style guide, include it.

When you include a lot of material, put the long documents first and your actual question or instructions after them — this tends to produce better results with long inputs.

3. Show examples (few-shot prompting)

Examples are often the single most effective way to communicate format, tone and judgment. Describing a style is hard; showing it is easy.

Classify each customer message as BILLING, BUG, FEATURE_REQUEST or OTHER.

Message: "I was charged twice this month."
Category: BILLING

Message: "The app crashes when I upload a photo."
Category: BUG

Message: "It would be great if I could export to Excel."
Category: FEATURE_REQUEST

Message: "My invoice shows the wrong company name."
Category:

Tips for good examples:

• Make them varied. If all examples look alike, the model may copy surface details (length, wording) rather than the underlying pattern.
• Include edge cases — the tricky inputs where judgment matters most.
• Keep them correct. The model will imitate mistakes in your examples too.

4. Structure your prompt

Long prompts that mix instructions, data and examples in one block are easy to misread. Separate the parts clearly. XML-style tags, Markdown headings or triple quotes all work:

You are reviewing a contract for a small business owner.

<contract>
{contract_text}
</contract>

<instructions>
1. List any clauses about automatic renewal or cancellation fees.
2. Flag anything unusual compared with a standard services agreement.
3. Explain each point in plain English, without legal jargon.
</instructions>

Clear boundaries also reduce the risk of the model mistaking content inside a document for instructions to follow.

5. Leave room for reasoning

For multi-step problems — math, analysis, planning, tricky judgments — models do better when they work through the problem before committing to an answer. This is known as chain-of-thought prompting.

Many current models have built-in "thinking" or "reasoning" modes that do this automatically; if yours does, turn it on for hard tasks rather than prompting for it. Otherwise, ask explicitly:

Before answering, think through the problem step by step:
identify the relevant facts, consider alternatives, then give
your final answer on a separate line starting with "Answer:".

Asking for the final answer in a predictable place makes it easy for your code to extract it while still allowing the reasoning.

6. Specify the output format

If a program will read the output, say exactly what shape it must have — and give an example.

Extract the following fields from the email and return only JSON,
with no extra text:

{"customer_name": string, "order_id": string or null,
 "issue_summary": string (max 20 words), "urgent": boolean}

For production use, many model APIs offer structured outputs or JSON modes that guarantee valid output against a schema — prefer those over prompt instructions alone when available.

Two more formatting tips:

• Say what to do, not only what to avoid. "Write in flowing paragraphs" works better than "don't use bullet points".
• Your prompt's style leaks into the answer. A prompt full of Markdown tends to get Markdown back; a plain prose prompt tends to get prose.

7. Use roles and system prompts — sensibly

Most chat APIs have a system prompt: standing instructions that apply to the whole conversation. It's the right place for the assistant's purpose, audience, tone, rules and background knowledge.

Assigning a role ("You are an experienced tax accountant explaining things to first-time freelancers") can help set expertise level and tone. But a role is not a magic spell — it works because it implies context. The more concrete detail you add about the audience and goal, the better.

8. Allow "I don't know"

Models try to be helpful, and without permission to say otherwise they may produce a plausible-sounding answer when they lack the information. Explicitly allow uncertainty:

Answer using only the provided documents. If the documents don't
contain the answer, say "I couldn't find that in the documents"
rather than guessing.

Asking the model to quote the relevant passage before answering makes answers easier to verify, and reduces made-up details further.

9. Break big tasks into steps

One giant prompt that researches, analyzes, writes and formats in a single pass is hard to debug. Splitting it into a chain of smaller prompts — extract → analyze → draft → review — makes each step simpler, lets you check intermediate results, and lets you use different settings (or models) for different steps.

10. Iterate like an engineer

The real skill is not writing the perfect prompt on the first try — it's improving prompts systematically:

1. Collect test inputs — 10–50 realistic examples, including hard and unusual cases.
2. Define success — what makes an output good or bad, as concretely as possible.
3. Run, read and score the outputs. Actually read them; patterns in failures tell you what's missing.
4. Change one thing at a time and re-run the whole set, so you notice when a fix for one case breaks another.

This turns prompting from guesswork into measurable progress. We cover it in depth in Evaluating LLM Applications.

Common mistakes

• Assuming shared context — the model doesn't know your company, project or preferences unless you say so.
• Contradictory instructions — "be thorough" and "be brief" with no guidance on how to balance them.
• Shouting — ALL CAPS and "CRITICAL!!!" rarely help with modern models and can make them over-apply a rule. Calm, clear instructions work better.
• Prompt bloat — piling on rules after every failure until the prompt contradicts itself. Periodically rewrite from scratch.
• Testing on one example — a prompt that works once may fail on the next ten inputs.

Key takeaways

• Clear communication beats tricks: say what you want, for whom, and why.
• Give the model the context it can't know on its own.
• Examples are the most powerful way to show format and judgment.
• Structure long prompts with clear sections or tags.
• Let the model reason on hard problems; define output formats precisely.
• Allow "I don't know" to reduce made-up answers.
• Improve prompts with a test set, not by gut feeling.