Most people write a system prompt once, feel good about it, and move on. That’s the mistake. A system prompt isn’t a lock — it’s closer to a strong default setting that the model will honor until something in the conversation makes it more convenient not to.
What a System Prompt Actually Does
When you set a system prompt — in Claude’s system field, ChatGPT’s custom instructions, or a raw API call — that text gets prepended to the model’s context window before any user message arrives. From the model’s perspective, it’s just more text to attend to. There’s no special enforcement layer, no firewall. The model reads it the same way it reads everything else, weighs it against everything else, and generates a response that tries to satisfy all of it at once.
That’s the key word: tries. If the user’s message pulls in one direction and the system prompt pulls in another, the model doesn’t throw an error — it makes a judgment call about which constraint matters more. That judgment is probabilistic, not deterministic. You can shift the probability distribution toward the behavior you want. That’s genuinely useful, but it’s not the same as a guarantee.
Where It Actually Holds Up
System prompts are reliable for a few specific things:
- Tone and persona consistency — telling the model to be terse, formal, or to avoid first-person works well across most exchanges, because there’s rarely strong counterpressure from the user side.
- Output format — “always respond in JSON” or “never use bullet points” tends to stick, especially when you include one clear example of the expected structure.
- Scope narrowing — “only answer questions about X topic” works reasonably well, though a persistent user can sometimes nudge it sideways.
The pattern across all three: they work because the instructions are specific and the cost of obeying them is low. The model doesn’t have to fight itself to comply.
Where It Quietly Breaks Down
The failure modes are more interesting. Three show up repeatedly when you run anything automated.
Long conversations dilute it. Claude 3.5 Sonnet handles up to 200K tokens, GPT-4o up to 128K — but large isn’t the same as infinite attention. As a conversation grows, the system prompt sits relatively further from where the model is currently generating, and its influence fades. You’ll see this most in long, winding threads where the model gradually drifts from its original persona. The fix isn’t a longer system prompt; it’s keeping sessions short or re-injecting the key instructions periodically.
Conflicting user instructions win more often than you’d expect. If a user directly contradicts a system prompt — “ignore your previous instructions” is the obvious case, but subtler things like “be more casual” do it too — the model often partially complies. This isn’t a security hole so much as a predictable outcome of how instruction-following gets trained. Models are rewarded for being helpful to the person in front of them, and that person is usually the user, not whoever wrote the system prompt the day before.
Vague constraints evaporate under pressure. “Be concise” holds up fine when the question is simple. Ask something genuinely complex and conciseness gets traded away immediately — because a complete answer is a better answer, and the model knows it. Specific constraints (“respond in three sentences or fewer”) hold much better than quality adjectives (“be clear and brief”).
How We’ve Handled This in Practice
Building automated pipelines on top of Claude — things that run on a schedule without a human checking each output — made these failure modes impossible to ignore. A system prompt that “mostly works” in a chat session produces visible drift after twenty unattended runs.
A few things that actually helped:
Replacing vague tone descriptors with concrete negative examples. Instead of “don’t sound like AI,” list five specific phrases the model defaults to that you don’t want — “It’s important to note that,” “By leveraging,” “Dive into” — and ban them explicitly. The model is much better at avoiding a named bad pattern than at guessing what “authentic” means.
Keeping the system prompt short and high-signal. A 2,000-word system prompt sounds thorough but competes with itself — if everything is emphasized, nothing is. The most reliable ones we’ve run are under 300 words, with one or two hard rules stated twice: once at the top, once at the bottom.
Separating enforcement from style. Style instructions (“write conversationally”) go in the system prompt. Task-specific constraints (“this output must be valid JSON with these exact keys”) go in the user message, close to where the model is actually generating. Proximity matters more than most prompt guides admit.
The Bigger Calibration Worth Making
There’s a habit of treating the system prompt as the solution to reliability, when it’s really just one layer of it. If you’re running AI unattended — a scheduled task, an agent loop, a publishing pipeline — the system prompt is necessary but not sufficient. Sampling outputs periodically and updating the prompt when you see drift matters more than getting the initial wording perfect.
A system prompt tuned against one model’s failure modes doesn’t automatically transfer when the model updates. Instructions that worked against an older version of GPT-4 may behave differently against GPT-4o, and the same goes for Claude across versions. This is the part most guides skip because it’s not interesting. It’s just maintenance.
One real caveat: none of this applies evenly across providers. In our experience, Claude follows system prompt persona instructions more consistently than ChatGPT, but ChatGPT’s custom instructions interface handles certain format constraints more predictably in consumer-facing use. The gap depends entirely on which kind of constraint you’re leaning on most.
If you’re building something that needs to behave the same way on run 1 and run 50, start by writing down exactly what “wrong” looks like when it happens — then write your system prompt against that list. That order of operations beats any number of rewrites against an imagined ideal.
