Definition
Context Window
A context window is the maximum amount of text an LLM can process at one time — the 'working memory' of the model during a single conversation turn. Context windows are measured in tokens. A 128,000-token context window can process roughly 96,000 words simultaneously. In support, context windows determine how much conversation history, retrieved knowledge, and system instructions the AI can hold in a single response generation.
How it works
Context window size matters for support in two scenarios. Long conversations: a customer support thread that has run for 20+ messages may approach the context limit, causing the AI to 'forget' early messages. Large knowledge retrieval: a support question that requires synthesizing multiple long documents may exceed the retrievable context if the window is too small.
Modern frontier LLMs (GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro) have context windows of 128,000–1,000,000 tokens, making context window limitations rare in practice for typical support conversations. Older or smaller models with 4,000–8,000 token windows create visible truncation artifacts in long threads.
Context window economics also affect cost. Larger context windows cost more to fill — passing the full conversation history and retrieved documents costs more tokens than passing only the last three messages. Production support systems optimize context by summarizing older conversation turns rather than passing every message verbatim.
When you'll encounter it
Context window is mentioned in technical comparisons of LLMs, in documentation for AI support platforms, and in edge-case debugging (e.g., 'why did the AI repeat a question it already asked two turns ago?'). For most production support deployments, context window size is not a practical constraint.
How SupportSyndicate handles this
SupportSyndicate uses frontier LLMs with large context windows and manages context efficiently by summarizing older conversation turns when threads grow long, preserving the most recent exchanges and key customer facts at full resolution.
Related terms
- LLM (Large Language Model)A large language model (LLM) is an AI system trained on massive text datasets to understan…
- RAG (Retrieval-Augmented Generation)Retrieval-augmented generation (RAG) is an AI architecture that combines a large language …
- Vector EmbeddingA vector embedding is a numerical representation of text — a list of hundreds or thousands…