Tokens, Context Windows & Temperature

These three concepts are the invisible machinery behind every AI interaction. You do not need to memorize the technical details, but understanding what they are -- and how they affect your results -- will make you a dramatically better AI user. Think of this lesson as learning the controls of a car before you start driving.

A token is a piece of text -- roughly 3/4 of a word in English. The sentence "I love artificial intelligence" is about 5 tokens. Longer or more unusual words get split into multiple tokens, while short common words are often a single token.

Why tokens matter in practice:

• Cost: API pricing is per token (input + output). A lengthy conversation can cost real money if you are using the API directly.
• Speed: More tokens means slower responses. If you have ever noticed the AI taking longer on a complicated request, token count is usually the reason.
• Limits: Every model has a maximum token limit for a single interaction. Hit the ceiling, and the AI starts "forgetting" the start of your conversation.

Quick estimates to keep in your head:

• 1 token is roughly 4 characters in English
• 100 tokens is roughly 75 words
• 1,000 tokens is roughly 750 words (about 1.5 pages of a document)
• A typical back-and-forth conversation uses 2,000-5,000 tokens

The context window is the total number of tokens the AI can "see" at once -- your entire conversation history (every message you sent and every response the AI gave) all has to fit inside this window.

When you exceed the context window, the AI literally cannot see the beginning of your conversation anymore. It is like having a conversation where the other person gradually forgets everything you said earlier. You might reference something from your first message, and the AI will have no idea what you are talking about.

Temperature controls the randomness of the AI's word choices. It is a number that typically ranges from 0.0 to 2.0, and it fundamentally changes the character of the output.

• 0.0: Deterministic -- same input gives the same output every time. Best for facts, code, and data extraction where you want consistency.
• 0.7: Balanced -- the default setting for most chat tools. Good for general use where you want helpful, varied responses.
• 1.0+: Creative -- more varied and unexpected word choices. Good for brainstorming, creative fiction, generating alternative phrasings.
• 2.0: Maximum randomness -- often produces incoherent or bizarre output. Rarely useful in practice.

Most chat interfaces (ChatGPT, Claude, Gemini) do not let you set temperature directly -- they use a sensible default behind the scenes. But if you use the API or tools like Cursor or the OpenAI Playground, you will see temperature as a setting you can adjust.