What Are Tokens?
Think of tokens as the teeny-tiny building blocks of text that help AI models (like ChatGPT or Claude) understand and generate language. Instead of reading whole words, AI breaks everything down into little chunks of text—usually just a few characters or part of a word.
It’s kind of like chopping veggies for a soup—smaller pieces help everything cook evenly! For example, the word “language” might be split into two tokens: “lan” and “guage.”
Why should you care? Well, tokens are the basic units AI uses to process your text, and even pricing for most AI tools is based on how many tokens you use. The longer your text or prompt (yes, even extra spaces or formatting count!), the more tokens you’re using.
What Is a Context Window?
Now, imagine you’re juggling a busy dinner prep and trying to remember every step of the recipe without peeking—that’s exactly what a context window is for AI.
In simple terms, the context window is the AI model’s “short-term memory.” It’s the maximum number of tokens it can handle at one time, and it includes everything you type (your prompt) plus the AI’s response.
If the context window is small, the AI might “forget” earlier parts of the conversation or text. But with larger context windows (like the ones in newer models), it’s like having a friend who remembers every detail—even the extra seasoning you added last time.
| Major LLMs – Training Tokens & Context Window Sizes | ||
| Model Family | Training Tokens (Approx.) | Context Window (Max Tokens) |
| Llama 4 (Meta) | Up to 40T tokens (Behemoth) | Scout: 10M, Maverick/Behemoth: 1M |
| Llama 3 / 3.1 (Meta) | ~15T tokens | Up to 128K tokens |
| Mistral Large/Medium | Not disclosed | Up to 128K tokens |
| GPT-4.1 / GPT-4o (OpenAI) | Not disclosed | 128K (GPT‑4o), ~1M (GPT‑4.1) |
| Claude 4 (Anthropic) | Not disclosed | 200K default, up to 1M (enterprise) |
| Gemini 2.5 (Google) | Not disclosed | 128K, scalable to 1M (Pro) |
| DeepSeek R1 / R2 | Estimated multi‑trillion scale | 128K–1M tokens |