There Is No Memory

What You Think Happens

When you use ChatGPT or Claude, it feels like a conversation. You say something, it responds, you follow up, it remembers. It feels like talking to someone who's paying attention.

That's an illusion. A well-designed one, but an illusion.

What Actually Happens

Every time you hit send, the application takes your entire conversation—every message you've sent, every response the model has given—and sends all of it to the model as one single block of text. The model reads the whole thing from the top, predicts what should come next, and sends that back. Then it forgets everything.

The model doesn't "remember" your last message. It re-reads the whole conversation every single time. There is no state. There is no memory. There is only the prompt.

Anatomy of a Request

Let's look at what actually gets sent to the model when you type a message in ChatGPT and hit enter. The application constructs a payload that looks roughly like this:

"You are a helpful assistant. Be concise..."

user: Can you help me...
assistant: Of course!...
user: I need...

user: I have 3 years of Python experience

assistant: ▊ (generating token by token...)

That system prompt at the top? It's there every time. Your first message from 20 minutes ago? Still there. The model's own previous responses? Also there—the model is literally reading its own past output as part of the input. The whole thing is one continuous document, and the model's job is to write the next part.

The Context Window

This "one long prompt" has a size limit called the context window . It's measured in tokens (roughly ¾ of a word). Current models have context windows ranging from 128k to 200k tokens—that's roughly a 300-page book.

Sounds like a lot, but it fills up fast. Every message you send, every response the model gives, every image you attach—it all goes in the window. When it fills up, the oldest messages get dropped. The model doesn't know they ever existed.

Inference: Token by Token

Now here's where everything from the previous lessons comes together. When the model receives that payload—system prompt, conversation history, your latest message—it does the thing it was trained to do:

.

But it doesn't predict the whole response at once. It predicts one token, appends it to the payload, then predicts the next token, appends that, predicts the next, and so on. Each token is influenced by everything that came before it—your system prompt, your conversation history, and every token of the response generated so far.

This is why LLMs sometimes start a sentence confidently and then paint themselves into a corner—each token commits the model to a direction, and there's no going back. It's also why longer responses tend to drift: the model is navigating a space of probabilities one step at a time, and small deviations compound.

Why This Matters for You

Understanding that there's no memory changes how you use these tools:

That last point is so important it's going to come up again in

. But first—now that you understand how these systems actually work, let's talk about how to

.