Topic 10 · LLMs, Generative AI & Safety

How Large Language Models Work

Interactive explorations of tokenisation, next-token prediction, temperature sampling, and hallucination — the core mechanisms behind GPT, Claude, and Gemini.

What is a Token?

LLMs don't read characters or words — they read tokens. A token is roughly 3–4 characters. The model's vocabulary is typically 50,000–100,000 tokens.

Token cost calculator

API pricing is per token. See how your text translates to cost at current frontier model prices.

Key tokenisation facts
• 1 token ≈ 4 characters or ¾ of an English word
• 100 tokens ≈ 75 words (1 paragraph)
• GPT-4 context: 128,000 tokens (~96,000 words)
• Rare words are split into multiple tokens
• Code is denser: 1 token ≈ 2.6 chars