Context and memory

The model only sees what you send

Each request includes everything the model can use: instructions, tool definitions, retrieved text, conversation history, and the new message. Together they form the context window.

The model has no memory between requests. Anything it should remember has to be sent again.

1. Treat the window as a budget

Every token costs time and money, and the window has a limit. Long, noisy context can also make answers worse, because the important part is harder to find.

2. Keep the history short

In long conversations, keep the recent turns and replace older ones with a short summary. Keep decisions and open questions, and drop small talk.

3. Put stable parts first

Instructions and tool definitions rarely change. Keeping them at the start lets many APIs cache them, which lowers cost and latency for repeated requests.

4. Memory is retrieval over your own notes

Long term memory usually means saving useful facts, such as a user’s preferences, and retrieving the relevant ones in later sessions. Decide what is worth saving, and let users see and delete it.

When is it useful?

Context management matters as soon as conversations get long, documents get large, or costs start to grow. For short, single requests, send what is needed and keep it simple.