What is a System Prompt?

What is it?

Chat-style model APIs typically accept messages with different roles. The usual ones are:

• System (some APIs call it developer): standing instructions from whoever builds the application.
• User: what the end user types.
• Assistant: the model's previous replies.
• Tool: results returned from tools (Function Calling (Tool Calling) Explained).

The exact role names and how strongly each is weighted vary by provider, so consult the documentation of the model you use. The idea is constant: a system prompt is the "job description" the model carries into every turn.

Why does it matter?

The same underlying model can behave like a terse code reviewer, a patient tutor or a strict data-extraction engine, purely based on the system prompt. For an application it is the cheapest and fastest place to control behaviour: tone, scope ("only answer questions about our product"), output format, escalation rules, and how to handle missing information.

How does it work?

The system prompt is simply text placed at the start of the context, so it:

• uses tokens on every request (What are Tokens in AI?, What is a Context Window?);
• is followed on a best-effort basis. Models are trained to prioritise it, but they can still drift or be talked around;
• can be inferred or extracted by a determined user in some setups, so it should not hold secrets.

Good system prompts are concrete and ordered by importance:

1. Role and audience.
2. What the assistant should and should not do.
3. How to use supplied information (e.g. "answer only from the provided documents; otherwise say you don't know").
4. Output format and tone.
5. What to do in edge cases (unclear request, out-of-scope request, sensitive topic).

Example

You are the support assistant for Acme Billing.
- Answer only questions about Acme invoices, plans and refunds.
- Use only the policy excerpts provided in <policy> tags. If the answer is not there,
  say you are not sure and offer to connect the user to a human agent.
- Never ask for full card numbers.
- Keep answers under 120 words, in a friendly, plain tone.

Each rule maps to a behaviour you can test: out-of-scope questions, missing policy, a user pasting a card number, and answer length.

When should I use it?

Use a system prompt for anything that should hold for the whole session or product: persona, scope, format, safety rules, and tool-use guidance. Use the user message for the specific task. Version-control system prompts, and re-run your test cases (How to Evaluate AI Systems and Outputs) after each change.

Common mistakes

• Putting secrets in it (API keys, private rules). Assume it can leak.
• Treating it as a firewall. Instructions can be overridden by crafted input; enforce permissions in code. See What is Prompt Injection?.
• Making it enormous and contradictory. Long, conflicting rules reduce compliance and cost tokens on every call.
• Never testing edge cases. Test refusals, missing data and adversarial inputs.

Related concepts