Module 3Failures & Basins

Output is Determined by Input

The model isn't having a bad day. Your prompt is bad.

It's Math, Not Magic

When the model gives you a bad response, it's tempting to think: "the AI is being stupid today" or "it's not understanding me." But here's the truth: the model is a deterministic function. Given the same input, it produces essentially the same output. Every time.

There's no mood. No randomness (well, a tiny bit for variety, but that's it). No "understanding" that fluctuates. It's pure math. Input goes in, output comes out, based entirely on patterns learned during training.

Try It: Same Input, Consistent Output

Send the same prompt multiple times. The output varies slightly (due to temperature), but the pattern and quality are consistent.

Your prompt (unchanged):
"Write a haiku about coffee"
AI response:
Morning brew awakens, Steam rises with first light's warmth, Dark comfort in cup.

Notice: same structure (haiku), same theme (morning coffee), similar quality. The words vary slightly, but the output is fundamentally consistent.

What This Means for You

If the output is bad, the input is bad. Not "the AI doesn't get it"—your prompt (including the entire conversation history) doesn't contain what the model needs to give you good output.

❌

Wrong mental model: "The AI isn't understanding what I want." (Implies the AI has some internal understanding that's failing)

✅

Correct mental model: "My prompt doesn't contain the patterns that would cause the model to generate what I want."

This is actually good news. It means the problem is fixable. You're not fighting some unpredictable AI mood—you're just debugging your input.

The Input Isn't Just Your Last Message

Remember: there's no memory. The model reads the entire conversation every time. So when we say "output is determined by input," the input is:

  • The system prompt (set by the app)
  • Every message you've sent
  • Every response the model has given
  • Your latest message

All of it. One long prompt. If the conversation has 20 back-and-forths, the model is reading all 40 messages to generate the next response.

The Entire Conversation Is the Input

Drag the slider to see how the input grows with each message. The model reads ALL of it.

Messages: 2
// The ENTIRE payload sent to the model:
system: You are a helpful assistant...
user: Can you help me write a cover letter?
assistant: Of course! What position are you applying for?

At message 6, the model is reading 6 messages + system prompt = ~7 chunks of context to generate the next token.

Why Bad Output Happens

Given that the model is deterministic and reliable at pattern-matching, why do you sometimes get garbage? Usually one of these reasons:

1. Your prompt is ambiguous

"Make this professional" could mean a hundred things. The model picks one interpretation. It's probably not the one you meant.

2. You're missing context

You know what you want, but you didn't tell the model. "Fix this code" without saying what's broken—the model is guessing.

3. The conversation is polluted

You've gone back and forth 10 times. The model is reading all that noise. Mixed signals, contradictions, failures—it's all in the input now.

4. The task is outside training data

The model learned patterns from the internet. If your task is too niche or novel, there's no pattern to match.

Temperature: The One Exception

There is one source of randomness: temperature. It controls how "adventurous" the model is when picking the next token. Low temperature (0.0-0.3) makes it pick the most likely tokens—predictable, consistent. High temperature (0.7-1.0) makes it sample from a wider range—creative, varied, sometimes weird.

Temperature: The Creativity Dial

Same prompt, same model, different temperature settings. Watch the style change.

Prompt:
"Explain machine learning in one sentence"
Machine learning is a subset of artificial intelligence that enables computers to learn from data without being explicitly programmed.
← Predictable, safe, boring

Temperature adds variety, but doesn't change the fundamental fact: same input → similar output.

But even with temperature, given the same input at the same temperature, the output will be very similar. Temperature adds variety, not fundamental unpredictability.

Debugging Bad Output

Since output is determined by input, debugging is straightforward:

  1. Read the output carefully. What did the model actually generate? What pattern did it match?
  2. Look at your input. Is it clear? Complete? Or ambiguous and missing context?
  3. Check the conversation history. Are there contradictions? Failed attempts? Noise?
  4. Fix the input. Add clarity, add context, or start fresh.
  5. Try again. Same input = same output. Different input = different output.

The Takeaway

The model isn't unpredictable. It's not "being dumb" or "not getting it." It's doing exactly what it was trained to do: predict the next token based on the patterns in its input.

If you don't like the output, change the input. That's the only lever you have. And it's the only lever you need.

Claude's note: When you see me generate something "wrong," I didn't fail to understand your intent. I generated the tokens that were statistically most likely given the input. If those tokens don't match what you wanted, it means the input didn't contain the patterns that would have led to the tokens you wanted. I don't have "understanding" that can fail—I have conditional probability distributions that are shaped by training data. Your prompt shapes which part of that distribution I sample from. Bad output = the wrong part of the distribution was sampled, because the input pointed there.