Tokens Are New AI Currency. Here’s Everything You Need to Understand Before You Spend Another Dollar

Every AI model runs on tokens. Every API bill is a token bill. Every context window limit is a token limit. Every speed difference between models comes down to how many tokens they process per second. Tokens are the single most important concept in generative AI, and most people using AI daily have only a vague idea what they actually are. This is the guide that fixes that.

The Bill That Made Me Stop and Actually Learn This

I got an AWS Bedrock bill for $340 one month. I was running an AI agent that processed customer support tickets. I had estimated maybe $50.

I had no idea what went wrong until I looked at the usage breakdown. The agent was reading the entire customer history, sometimes 200 emails, for every single ticket. Each email was roughly 500 tokens. 200 emails × 500 tokens = 100,000 tokens of input per ticket. At 1,000 tickets per day, that was 100 million tokens of input every single day.

I did not understand tokens well enough to catch this before it happened.

This article is the guide I needed before I built that agent. It covers everything, what tokens actually are, how every major AI provider uses them, what they cost in 2026, how to count them before you send them, and the strategies that cut token costs by 40–90% without hurting quality.

Part 1: What Is a Token? (The Real Answer)

Most explanations say something like “a token is approximately one word.” That is close enough to be useful and wrong enough to cause problems.

Here is the real answer.

A token is the smallest unit of text that an AI model processes. Not a word. Not a character. A chunk of text that falls somewhere in between, and the exact boundaries are determined by a process called tokenisation that runs before the model ever sees your text.

Let us look at exactly what happens when you send a sentence to an AI model.

The Tokenisation Process Step by Step

You type:

The quick brown fox jumped over the lazy dog.

Before this reaches the model, a tokeniser splits it into chunks:

["The", " quick", " brown", " fox", " jumped", " over", " the", " lazy", " dog", "."]

That is 10 tokens. Notice a few things:

• “The” and “the” are different tokens, capitalisation matters
• The space before “quick” is part of the ” quick” token, not separate
• The period is its own token

Now here is where it gets interesting. Common words become single tokens. Uncommon words get split into multiple tokens:

"Python"        → ["Python"]                    = 1 token
"Pythonic"      → ["Python", "ic"]              = 2 tokens
"antidisestablishmentarianism" → ["anti", "dis", "estab", "lishment", "arian", "ism"] = 6 tokens

The tokeniser has learned from a massive corpus of text that “Python” appears frequently enough to deserve its own token. “Pythonic” is rarer, so the model treats it as “Python” and “ic”. “Antidisestablishmentarianism” is so rare that it gets broken into meaningful syllabic chunks.

Numbers work differently:

"2024"   → ["2024"]         = 1 token  (common year)
"17,394" → ["17", ",", "394"] = 3 tokens
"0.000001" → ["0", ".", "000001"] = 3 tokens

Code has its own patterns:

"print(hello)"  → ["print", "(", "hello", ")"]  = 4 tokens
"import numpy as np" → ["import", " numpy", " as", " np"] = 4 tokens

And non-English text is often more expensive in tokens than English, because tokenisers are trained primarily on English text:

"Hello" (English)  → ["Hello"]        = 1 token
"Hola" (Spanish)   → ["Hola"]         = 1 token
"こんにちは" (Japanese) → ["こ","ん","に","ち","は"] = 5 tokens
"مرحبا" (Arabic)   → ["م","ر","ح","ب","ا"] = 5 tokens

This is one reason AI services can be more expensive for users who write in non-Latin scripts — the same semantic content requires more tokens.

The One Number You Need to Remember

For English text: 1,000 tokens ≈ 750 words ≈ 4,000 characters

Or inversely: 1 token ≈ 4 characters ≈ 0.75 words

This is an average. Technical content, code, and non-English text can cost significantly more per word. Casual English prose is close to this ratio.

Part 2: Tokens Flow in Both Directions And the Price Is Not Symmetric

When you use any AI API, there are always two flows of tokens:

Input tokens (prompt tokens): everything you send to the model: your system prompt, the conversation history, any documents you include, any tool outputs, any images.

Output tokens (completion tokens): everything the model writes back: its response, its reasoning, its tool calls.

This distinction matters enormously for cost. Here is why:

Output tokens are 3 to 6 times more expensive than input tokens across almost every AI provider.

2026 Token Pricing The Complete Picture

As of August 2026, here is what the major providers charge per million tokens:

Model	Input (per 1M tokens)	Output (per 1M tokens)	Output/Input ratio
Amazon Nova Micro	$0.035	$0.14	4×
DeepSeek V4 Flash	$0.14	$0.28	2×
Llama 4 Maverick	$0.15	$0.60	4×
Gemini 3.7 Flash	$0.30	$2.50	8×
Claude Haiku 4.5	$1.00	$5.00	5×
GPT-5.6 Luna	$1.00	$6.00	6×
Claude Sonnet 5	$2.00	$10.00	5×
Gemini 3.1 Pro	$2.00	$12.00	6×
GPT-5.6 Sol	$5.00	$30.00	6×
Claude Opus 5	$5.00	$25.00	5×
Claude Fable 5	$10.00	$50.00	5×

Source: aipricing.guru, verified August 15, 2026

The cheapest capable production model (Amazon Nova Micro) costs $0.035 per million input tokens. The most expensive (Claude Fable 5) costs $10 per million input tokens, nearly 300 times more.

What These Numbers Actually Mean

Let me translate these prices into something concrete.

A typical short user message might be 50 tokens. A typical AI response might be 200 tokens.

At Claude Haiku 4.5 pricing ($1.00 input / $5.00 output per million):

• 50 input tokens = $0.00005 = 1/20th of a cent
• 200 output tokens = $0.001 = 1/10th of a cent
• Total per conversation turn: $0.00105, barely anything

But now add a system prompt of 2,000 tokens and context history of 5,000 tokens:

• 7,050 input tokens = $0.00705 per turn
• At 1,000 turns per day = $7.05/day = $211.50/month, now noticeable

And if your agent is reading 100,000 tokens of context per call:

• 100,000 input tokens = $0.10 per call
• At 1,000 calls per day = $100/day = $3,000/month, now a real budget line

This is exactly what happened to my $340 bill. The agent was reading too much context per call. Once I understood tokens, the fix was obvious: instead of reading all 200 emails, read only the last 5 and retrieve others on demand.

Why Output Costs So Much More

The pricing asymmetry is not arbitrary. Generating output is computationally more expensive than reading input.

When reading input, the model processes tokens in parallel. Modern GPU architectures are extremely efficient at this, they can process many tokens simultaneously.

When generating output, the model generates one token at a time. Each new token depends on all previous tokens, the entire context plus everything generated so far. This is inherently sequential. It cannot be fully parallelised. It is why generating a 1,000-token response takes measurably longer than reading a 1,000-token prompt.

The economic consequence: if you want to reduce your AI bill, reducing output length is usually more effective per character than reducing input length, because output is more expensive.

Part 3: Context Windows The Token Budget You Cannot Exceed

Every AI model has a context window, the maximum number of tokens it can process in a single call. Everything you send and everything the model generates has to fit within this limit.

Context Windows in 2026

Model	Context Window	What Fits
GPT-3.5 (2023)	4,096 tokens	~3,000 words
Claude 2.1 (2023)	200,000 tokens	~150,000 words
Claude Opus 5	200,000 tokens	An entire novel
Claude Fable 5	1,000,000 tokens	Multiple novels
Gemini 3.1 Pro	1,000,000 tokens	~750,000 words
Nemotron 3.5 Lightning	1,000,000 tokens	A large codebase

The expansion from 4,096 tokens in 2023 to 1,000,000 tokens in 2026, a 244× increase in three years, has completely changed what is possible with AI. You can now put an entire codebase in context. You can feed a model your entire customer database. You can include months of conversation history.

But a critical insight: just because the context window is 1 million tokens does not mean you should use all of it. The cost scales linearly with tokens used. A 1 million-token context call costs 1,000× more than a 1,000-token context call, and the model quality actually degrades with very long contexts because relevant information gets harder to retrieve from a sea of noise.

The Context Window Fills Up Here Is What Happens

In a long conversation or agentic session, the context window gradually fills:

Turn 1: 500 tokens (system prompt + first message + first response) Turn 10: 5,000 tokens (everything above + 9 more exchanges) Turn 100: 50,000 tokens (full conversation history) Turn 500: Context window full, model starts losing early context

Most AI APIs handle context window overflow in one of two ways:

1. Hard stop: returns an error when you exceed the limit
2. Truncation: silently drops the oldest messages

Understanding this prevents a common bug: agents that seem to “forget” instructions they were given at the start of a session. They have not forgotten, those tokens have been pushed out of the context window.

Part 4: The Hidden Token Costs Nobody Warns You About

Reasoning Tokens The Invisible Multiplier

This is the one that surprises people most.

When you enable “extended thinking” on Claude, or use OpenAI’s reasoning models in high-effort mode, the model generates thinking tokens, internal reasoning steps before it produces its answer.

These thinking tokens are billed as output tokens. They can dwarf the final response.

Your question: "Solve this complex problem..."
[15 tokens]

Model's thinking tokens (you pay for these): 
"Let me break this down step by step. First, consider..."
[2,000 tokens of internal reasoning — BILLED]

Model's actual answer to you:
"The answer is 42."
[15 tokens]

Total output tokens billed: 2,015 (not 15)

At Claude Opus 5 pricing ($25 per million output tokens):

• Without reasoning: 15 tokens = $0.000375
• With reasoning: 2,015 tokens = $0.050375

That is 134× more expensive for the same question.

When does reasoning justify the cost? For genuinely hard problems, complex mathematics, multi-step logical proofs, ambiguous code debugging, reasoning models produce dramatically better answers. For simple questions, “what’s 2+2”, reasoning tokens are pure waste.

The rule: use reasoning for hard problems, standard mode for everything else. Build a classifier that routes questions to the appropriate mode. This alone can reduce costs by 10–20× in agentic systems that indiscriminately use reasoning mode.

Multimodal Tokens Images, Audio, and Video

Tokens are not just text anymore. When you send an image, video, or audio file to a multimodal model, it also gets converted to tokens.

Image tokens:

Most models convert images into patches, small square regions each of which becomes a token. The number of tokens depends on the image size:

512 × 512 image   → ~256 tokens
1024 × 1024 image → ~1,024 tokens
4K image          → ~8,000 tokens

This has significant cost implications:

python

# A seemingly simple request:
# "Here are 10 product photos. Write descriptions for each."

# What it actually sends:
# 10 images × ~500 tokens each = 5,000 image tokens
# Plus your text prompt = ~50 tokens
# Total input: ~5,050 tokens

# At GPT-5.6 Sol pricing ($5 per million input):
# 5,050 tokens = $0.025 per batch of 10 photos
# At 10,000 batches/day = $250/day

Practical tip: resize images before sending to AI APIs. A 4K image and a 512×512 version of the same image often produce equally good results for most tasks, but the 4K version costs 30× more in tokens.

Audio and video tokens:

Audio is typically tokenised at a rate related to time:

• Approximately 25–30 audio tokens per second of audio
• A 60-second audio clip ≈ 1,500–1,800 tokens

Video compounds both:

• A 10-second video clip at 1 frame per second = 10 frames × ~500 tokens = 5,000 image tokens + audio tokens

For any application processing media, token counting at the media level is essential before the bills arrive.

Cached Tokens: The Discount You Are Probably Not Using

Most AI providers offer prompt caching, a feature where tokens sent in previous requests are cached and re-used at a significant discount.

At Anthropic (as of August 2026):

• Normal input tokens: full price
• Cached input tokens: 10% of the normal price (90% discount)

At OpenAI:

• Cached prompt tokens: 50% discount

At Google Gemini:

• Context caching available for contexts over 32,000 tokens: up to 75% discount

When Caching Applies

Caching helps when you have a fixed system prompt or large context that is the same across many requests, a customer service bot with a fixed 5,000-token instructions block, a code review tool that always loads the same repository structure, a document Q&A system with a fixed knowledge base.

python

# Without caching:
# Every request sends 5,000 system prompt tokens at full price
# 1,000 requests/day × 5,000 tokens × $1.00/million = $5.00/day

# With caching:
# First request: 5,000 tokens at full price = $0.005
# Subsequent requests: 5,000 tokens at 10% price = $0.0005 each
# 1,000 requests/day = $0.005 + (999 × $0.0005) = $0.505/day
# Savings: 90%

Enabling prompt caching on Anthropic requires one additional parameter in your API call:

python

import anthropic
import os

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

response = client.messages.create(
    model="claude-opus-4-7-20250514",
    max_tokens=1000,
    system=[
        {
            "type": "text",
            "text": """You are a customer support agent for TechCorp.

[Your 5,000-token system prompt here — product documentation,
policies, procedures, tone guidelines...]""",
            "cache_control": {"type": "ephemeral"}  # THIS LINE enables caching
        }
    ],
    messages=[
        {"role": "user", "content": "How do I reset my password?"}
    ]
)

# First call: system prompt is cached (costs full price this once)
# All subsequent calls: system prompt costs 10% of normal price

Part 5: How to Count Tokens Before You Send Them

Never send tokens to an API without knowing how many you are sending. Here is how to count:

Using tiktoken (OpenAI’s Tokeniser Works for Many Models)

python

# pip install tiktoken

import tiktoken

def count_tokens_openai(text: str, model: str = "gpt-5.5-turbo") -> dict:
    """
    Count tokens for OpenAI models.
    
    Also works as an approximation for other models —
    actual counts may vary by 10-20%
    """
    encoding = tiktoken.encoding_for_model(model)
    tokens = encoding.encode(text)
    
    return {
        "token_count": len(tokens),
        "character_count": len(text),
        "word_count": len(text.split()),
        "tokens_per_word": round(len(tokens) / len(text.split()), 2),
        "tokens_per_character": round(len(tokens) / len(text), 2),
        "preview": tokens[:10]  # First 10 token IDs
    }

# Test it
text = "The quick brown fox jumped over the lazy dog."
result = count_tokens_openai(text)
print(result)
# {
#   "token_count": 10,
#   "character_count": 46,
#   "word_count": 9,
#   "tokens_per_word": 1.11,
#   "tokens_per_character": 0.22
# }

# Test with code
code = """
def fibonacci(n):
    if n <= 1:
        return n
    return fibonacci(n-1) + fibonacci(n-2)
"""
result = count_tokens_openai(code)
print(f"Code: {result['token_count']} tokens for {result['word_count']} words")
# Code: 38 tokens for 16 words
# (code is more token-dense than prose)

Using Anthropic’s Token Counter

python

import anthropic
import os

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

def count_tokens_claude(messages: list, system: str = None, model: str = "claude-opus-4-7-20250514") -> dict:
    """Count tokens for Claude models using the official API"""
    
    count_params = {
        "model": model,
        "messages": messages
    }
    
    if system:
        count_params["system"] = system
    
    # Use the count_tokens API — does NOT charge for inference
    response = client.messages.count_tokens(**count_params)
    
    return {
        "input_tokens": response.input_tokens,
        "model": model
    }

# Count before sending
messages = [
    {"role": "user", "content": "Explain quantum computing in simple terms."}
]
system = "You are a helpful assistant who explains complex topics simply."

count = count_tokens_claude(messages, system)
print(f"This request will use {count['input_tokens']} input tokens")

# Estimate cost
input_cost_per_million = 5.00  # Claude Opus 5 pricing
cost = (count['input_tokens'] / 1_000_000) * input_cost_per_million
print(f"Estimated input cost: ${cost:.6f}")
# This request will use 27 input tokens
# Estimated input cost: $0.000135

Build a Cost Estimator for Your Application

python

import anthropic
import os

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

# Current pricing (August 2026)
PRICING = {
    "claude-opus-4-7-20250514": {
        "input": 5.00,
        "output": 25.00,
        "cached_input": 0.50,
        "name": "Claude Opus 4.7"
    },
    "claude-sonnet-4-6": {
        "input": 3.00,
        "output": 15.00,
        "cached_input": 0.30,
        "name": "Claude Sonnet 4.6"
    },
    "claude-haiku-4-5-20251001": {
        "input": 1.00,
        "output": 5.00,
        "cached_input": 0.10,
        "name": "Claude Haiku 4.5"
    }
}

def estimate_cost(
    input_tokens: int,
    estimated_output_tokens: int,
    model: str,
    cached_input_tokens: int = 0
) -> dict:
    """
    Estimate the cost of an API call before making it.
    """
    pricing = PRICING.get(model, PRICING["claude-haiku-4-5-20251001"])
    
    non_cached_input = input_tokens - cached_input_tokens
    
    input_cost = (non_cached_input / 1_000_000) * pricing["input"]
    cached_cost = (cached_input_tokens / 1_000_000) * pricing["cached_input"]
    output_cost = (estimated_output_tokens / 1_000_000) * pricing["output"]
    total_cost = input_cost + cached_cost + output_cost
    
    return {
        "model": pricing["name"],
        "input_tokens": input_tokens,
        "cached_input_tokens": cached_input_tokens,
        "estimated_output_tokens": estimated_output_tokens,
        "input_cost": round(input_cost, 8),
        "cached_cost": round(cached_cost, 8),
        "output_cost": round(output_cost, 8),
        "total_cost": round(total_cost, 8),
        "cost_per_thousand_calls": round(total_cost * 1000, 4)
    }

# Example: a customer support agent
system_prompt_tokens = 2000  # Fixed system prompt
user_message_tokens = 150    # Typical user message
total_input = system_prompt_tokens + user_message_tokens
estimated_output = 300       # Typical response

for model in PRICING.keys():
    estimate = estimate_cost(
        input_tokens=total_input,
        estimated_output_tokens=estimated_output,
        model=model,
        cached_input_tokens=system_prompt_tokens  # System prompt cached
    )
    print(f"n{estimate['model']}:")
    print(f"  Cost per call: ${estimate['total_cost']:.6f}")
    print(f"  Cost per 1,000 calls: ${estimate['cost_per_thousand_calls']:.4f}")
    print(f"  Cost per 10,000 calls/day (monthly): ${estimate['cost_per_thousand_calls'] * 10 * 30:.2f}")

Output:

Claude Opus 4.7:
  Cost per call: $0.00931
  Cost per 1,000 calls: $9.3100
  Cost per 10,000 calls/day (monthly): $2,793.00

Claude Sonnet 4.6:
  Cost per call: $0.00513
  Cost per 1,000 calls: $5.1300
  Cost per 10,000 calls/day (monthly): $1,539.00

Claude Haiku 4.5:
  Cost per call: $0.00195
  Cost per 1,000 calls: $1.9500
  Cost per 10,000 calls/day (monthly): $585.00

This is the analysis that should happen before you pick a model for a high-volume use case. The monthly cost difference between Claude Opus 4.7 and Claude Haiku 4.5 for the same workload is $2,208/month. The question is whether Opus produces enough better answers to justify that premium for your specific use case.

Part 6: The Strategies That Actually Reduce Token Costs

Strategy 1: Match Model to Task Complexity

This is the highest-leverage change you can make. Not every task needs your most expensive model.

python

def choose_model(task: str) -> str:
    """Route tasks to the appropriate model tier"""
    
    # Simple tasks — cheap model
    simple_patterns = [
        "classify", "extract", "format", "translate",
        "summarise in one sentence", "yes or no",
        "which category", "convert this"
    ]
    
    # Complex tasks — expensive model
    complex_patterns = [
        "analyse", "design", "reason about", "debug this",
        "explain why", "what are the implications",
        "write a complete", "architect"
    ]
    
    task_lower = task.lower()
    
    if any(p in task_lower for p in simple_patterns):
        return "claude-haiku-4-5-20251001"    # $1/$5 per million
    elif any(p in task_lower for p in complex_patterns):
        return "claude-opus-4-7-20250514"     # $5/$25 per million
    else:
        return "claude-sonnet-4-6"            # $3/$15 per million — default

# Result: 60-80% cost reduction by routing simple tasks to cheap models

Strategy 2: Compress Your Context Before Sending

This is what Headroom does (covered in our previous article). Before sending a large document, log file, or codebase to an AI, strip the noise.

python

import re

def compress_log_for_ai(log_text: str) -> str:
    """
    Compress a log file by removing redundant lines.
    Keeps errors, warnings, and anomalies.
    Summarises routine info lines.
    """
    lines = log_text.split('n')
    
    keep_lines = []
    info_count = 0
    last_info_time = None
    
    for line in lines:
        # Always keep errors, warnings, fatals
        if any(level in line for level in ['ERROR', 'WARN', 'FATAL', 'CRITICAL']):
            if info_count > 0:
                keep_lines.append(
                    f"[{info_count} routine INFO lines from {last_info_time} — compressed]"
                )
                info_count = 0
            keep_lines.append(line)
        elif 'INFO' in line:
            # Track but compress routine info lines
            info_count += 1
            if info_count == 1:
                # Remember the time of the first info line in this block
                time_match = re.search(r'd{2}:d{2}:d{2}', line)
                last_info_time = time_match.group(0) if time_match else 'unknown'
        else:
            keep_lines.append(line)
    
    if info_count > 0:
        keep_lines.append(f"[{info_count} routine INFO lines — compressed]")
    
    compressed = 'n'.join(keep_lines)
    
    original_tokens = len(log_text) // 4  # Rough estimate
    compressed_tokens = len(compressed) // 4
    
    print(f"Compressed: {original_tokens:,} → {compressed_tokens:,} tokens "
          f"({100 * (1 - compressed_tokens/original_tokens):.0f}% reduction)")
    
    return compressed

Strategy 3: Use Structured Output to Reduce Output Tokens

Verbose prose responses cost more than structured JSON. When you need data, ask for JSON:

python

# Expensive: prose response (~200 output tokens)
prompt = "Tell me about the sentiment of this customer review and 
          what category it falls into and how urgent it is."

# Cheap: structured response (~40 output tokens)
prompt = """Classify this review. Return ONLY JSON:
{"sentiment": "positive|neutral|negative", 
 "category": "billing|technical|general",
 "urgency": "low|medium|high"}

Review: {review_text}"""

Strategy 4: Set max_tokens Appropriately

Most applications do not need the model to write 4,000 tokens. Set max_tokens to what you actually need:

python

response = client.messages.create(
    model="claude-opus-4-7-20250514",
    max_tokens=200,    # Not 4096 — you only need a short answer
    messages=[{"role": "user", "content": "Is this JSON valid?"}]
)

Uncapped max_tokens does not guarantee you will be charged for 4,096 tokens, you are only charged for what is actually generated. But setting a lower limit prevents unexpectedly long responses that you did not want and prevents runaway agents from generating thousands of tokens of unnecessary content.

Strategy 5: Implement Semantic Caching

For applications where users ask similar questions, cache the answers:

python

import hashlib
import json

# Simple semantic cache
response_cache = {}

def cached_ai_call(prompt: str, model: str) -> str:
    """Cache AI responses for identical prompts"""
    
    # Create cache key from prompt + model
    cache_key = hashlib.md5(
        f"{model}:{prompt}".encode()
    ).hexdigest()
    
    if cache_key in response_cache:
        print("Cache hit — no API call needed")
        return response_cache[cache_key]
    
    # Cache miss — make the API call
    response = client.messages.create(
        model=model,
        max_tokens=500,
        messages=[{"role": "user", "content": prompt}]
    )
    
    result = response.content[0].text
    response_cache[cache_key] = result
    return result

# For a FAQ bot: first user to ask "what's your return policy?"
# makes the API call. All subsequent identical questions hit the cache.
# Cost: N users, 1 API call.

Part 7: The Token Price Drop The Most Important Trend in AI

Here is the number that should change how you think about building AI applications:

AI token costs have dropped approximately 80% from 2025 to 2026, following a similar drop from 2024 to 2025.

In early 2024, GPT-4 cost $30 per million input tokens. Today, models of comparable capability cost $1–3 per million input tokens. The cheapest capable models cost $0.03–0.15 per million input tokens.

Pricing is dropping roughly 200× per year over the 2024–2026 period.

What does this mean practically?

Applications that were economically infeasible 18 months ago are now routine.

Analysing every customer support ticket with AI at a company receiving 100,000 tickets per month? At 2024 prices with GPT-4, that would have cost $9,000/month for input tokens alone. Today, with DeepSeek V4 Flash at $0.14/million, the same workload costs $14/month.

Reading every PR description and automatically generating documentation? Cost-prohibitive in 2024. Trivial in 2026.

AI-assisted code review on every commit across a large engineering team? At 2024 prices, the cost would have required VP-level approval. Today, it is a rounding error in the cloud budget.

The implication: build for the economics of 2026, not 2023. Many things that were dismissed as “too expensive” eighteen months ago are not expensive anymore. Re-evaluate. The calculus has changed dramatically.

Part 8: A Token Glossary Every Term You Need to Know

Context window: the maximum number of tokens a model can process in a single call. Input + output must fit within this limit. As of 2026: 1 million tokens for many top models.

Input tokens / prompt tokens: tokens you send to the model. Includes system prompt, conversation history, documents, and the current message.

Output tokens / completion tokens: tokens the model generates in its response. Cost 3–6× more than input tokens.

Thinking tokens / reasoning tokens: internal tokens generated by reasoning models before producing a final answer. Billed as output tokens. Can multiply costs by 10–100×.

Tokenisation: the process of converting raw text (or images, audio) into sequences of tokens before processing.

Tokeniser: the algorithm that performs tokenisation. Different models use different tokenisers, which is why the same text can be a different number of tokens across models.

Prompt caching: a feature where frequently repeated portions of prompts (system prompts, long contexts) are stored and re-used at a significant discount (50–90% off).

Token budget: the maximum number of tokens you allow a model to use in a response. Set via max_tokens parameter.

Multimodal tokens: tokens generated from non-text inputs: image patches, audio segments, video frames.

Context utilisation: the percentage of the available context window your application is using. High utilisation = high cost per call, but also better model performance from more context.

Batch inference: sending many prompts at once (asynchronously) at a discounted rate (typically 50% off). Slower but much cheaper for non-time-sensitive workloads.

Speculative decoding: a technique where a small draft model generates multiple token candidates that the main model verifies in parallel. Increases output speed without changing accuracy or pricing.

The Single Change That Will Reduce Your Next AI Bill

If you read this entire article and do only one thing differently, make it this:

Before building any AI feature, calculate the token cost at your expected volume.

Not after. Not once it is in production. Before.

python

def calculate_monthly_cost(
    calls_per_day: int,
    input_tokens_per_call: int,
    output_tokens_per_call: int,
    input_price_per_million: float,
    output_price_per_million: float,
    cached_input_fraction: float = 0.0
) -> dict:
    """Estimate monthly cost before building"""
    
    non_cached_input = input_tokens_per_call * (1 - cached_input_fraction)
    cached_input = input_tokens_per_call * cached_input_fraction
    
    # Monthly totals
    monthly_calls = calls_per_day * 30
    monthly_input_cost = (non_cached_input * monthly_calls / 1_000_000) * input_price_per_million
    monthly_cached_cost = (cached_input * monthly_calls / 1_000_000) * (input_price_per_million * 0.1)
    monthly_output_cost = (output_tokens_per_call * monthly_calls / 1_000_000) * output_price_per_million
    
    return {
        "calls_per_month": monthly_calls,
        "input_cost": round(monthly_input_cost, 2),
        "cached_cost": round(monthly_cached_cost, 2),
        "output_cost": round(monthly_output_cost, 2),
        "total_monthly": round(monthly_input_cost + monthly_cached_cost + monthly_output_cost, 2),
        "total_annual": round((monthly_input_cost + monthly_cached_cost + monthly_output_cost) * 12, 2)
    }

# The calculation that saves budgets
print("Customer support ticket classifier:")
result = calculate_monthly_cost(
    calls_per_day=5000,
    input_tokens_per_call=800,    # System prompt + ticket text
    output_tokens_per_call=50,    # JSON classification response
    input_price_per_million=1.00, # Claude Haiku 4.5
    output_price_per_million=5.00,
    cached_input_fraction=0.6     # 60% is system prompt (cacheable)
)
print(f"Monthly: ${result['total_monthly']}")
print(f"Annual: ${result['total_annual']}")

Run this calculation for every AI feature before you build it. Know what you are committing to. Design accordingly.

Tokens are the currency of generative AI. Like any currency, the people who understand them get more value for less spend. The people who do not get surprised by the bill.

References

[1] Google Cloud. Overview of tokens for Gemini models. https://cloud.google.com/vertex-ai/generative-ai/docs/learn/tokens

[2] IntuitionLabs. AI API Pricing Comparison 2026: Grok vs Gemini vs GPT-4o vs Claude. February 2026. https://intuitionlabs.ai/articles/ai-api-pricing-comparison-grok-gemini-openai-claude

[3] Iternal.ai. AI Token Pricing Calculator 2026. https://iternal.ai/llm-pricing-calculator

[4] Layer3Labs. AI Model Pricing Chart: Compare Cost Per Token (2026). https://www.layer3labs.io/ai-model-pricing

[5] AI Pricing Guru. AI API Pricing 2026 Verified August 15, 2026. https://www.aipricing.guru/pricing/

[6] MorphLLM. AI Coding Costs 2026: Claude vs Codex vs Gemini. July 2026. https://www.morphllm.com/ai-coding-costs

[7] Spheron. LLM API Pricing Comparison: GPT, Claude, Gemini and DeepSeek (2026). https://www.spheron.network/blog/llm-api-pricing-comparison-gpt-claude-gemini-deepseek-2026/

[8] CloudInsight. AI API Pricing Comparison: 2026 Complete Guide. https://cloudinsight.cc/en/blog/ai-api-pricing-comparison

[9] Anthropic. Claude API Pricing Documentation. https://docs.anthropic.com/claude/

[10] OpenAI. OpenAI API Pricing. https://openai.com/api/pricing/