An AI coding glossary — the vocabulary that makes teams effective
Half the confusion in AI tooling adoption is vocabulary. The terms I find myself explaining most, from tokens and context windows to handoffs and grilling.
Something I learned rolling out AI coding tools to engineering teams: half the confusion isn’t technical, it’s vocabulary. When an engineer asks “why is my API bill so high?” or “why did the agent get dumber halfway through?”, the answer usually starts with knowing the right words. So I wrote them down.
This is the glossary I wish I’d had on day one — the shared vocabulary that turns “the AI is being weird” into a diagnosable problem. It’s organised the way the concepts stack: model → sessions → tools → failure modes → handoffs → memory → patterns of work.
1. The model
| Term | Definition |
|---|---|
| Model | The core engine — a large set of numerical parameters that produce output one token at a time |
| Inference | Running a trained model to produce output; the primary cost driver at scale |
| Token | The basic unit a model reads and writes — roughly word-sized but not word-aligned |
| Non-determinism | Identical input can produce different output — a fundamental property of all models |
| Model provider | The service that runs the model and responds to inference requests |
| Harness | The software layer that wraps a model with tools, a system prompt, and session management |
| Model provider request | A single round-trip from the harness to the provider — one context window in, one response out |
| Input tokens | The full context window sent on each request; billed at a lower rate |
| Output tokens | Tokens the model generates; billed at a higher rate than input |
| Prefix cache | Provider-side optimisation that reduces cost by reusing already-processed context |
A few of these deserve more than a one-liner.
Model. A large set of numerical parameters — fixed at inference time — that take an input and produce an output, one token at a time. It’s stateless and does nothing else. On its own, a model can’t do anything useful; it needs a harness to give it tools, instructions, and a purpose.
Token. Roughly word-sized, but doesn’t map to words — common words are one token, while technical terms and variable names split across several. All limits, costs, and performance metrics are expressed in tokens. Avoid thinking in “words”: token counts determine cost and context usage, word counts don’t.
Non-determinism. The same input can produce different output each time, and no configuration eliminates it entirely. Evaluate results across multiple runs rather than treating any single one as fixed.
Harness. The layer that provides the system prompt, manages the context window, exposes tools, enforces permissions, and handles the back-and-forth. Two tools built on the same model behave very differently because their harnesses differ. This explains most “tool X is better than tool Y” debates.
Prefix cache. Consecutive requests sharing a common prefix skip reprocessing, and cached tokens are billed at a much lower rate. Anything that modifies the prefix — a timestamp injected into the system prompt, reordered context — invalidates the cache and bills the whole prefix at full rate. If your bill looks wrong, check this first.
2. Sessions, context windows, and turns
| Term | Definition |
|---|---|
| Stateless | No memory between interactions — nothing carries over unless re-supplied |
| Context | The information relevant to the current task |
| Context window | The complete input the model receives on each request; finite and model-specific |
| Agent | A model wrapped in a harness with tools, a system prompt, and a context window |
| System prompt | Standing instructions injected at the start of every request; invisible to the user |
| Session | A single bounded interaction, from first message until the context is cleared or compacted |
| Turn | One complete exchange — a user message plus everything the agent does in response |
Context vs. context window vs. session — the distinction that unlocks everything else. Context is the abstract concept: what’s relevant to the task. The context window is the raw input the model receives. The session is the accumulated history. Deciding what to include, exclude, and when — context engineering — is one of the highest-leverage skills in this whole space.
Context window. Everything the model can perceive — system prompt, history, loaded files, tool results — must fit in it. It’s finite, and it’s the only surface through which the model accesses information. Don’t call it “memory”: it’s transient working state that doesn’t persist between sessions.
Turn. One user message can trigger many provider requests, because each tool call needs its result sent back in a new request. The hierarchy is session > turn > model provider request — worth internalising before reading any usage bill.
3. Tools and environment
| Term | Definition |
|---|---|
| Environment | Everything outside the harness the agent can perceive and act on |
| Tool | A function the harness makes available for the agent to call |
| Tool call | The structured output the model produces to invoke a tool |
| Tool result | The harness’s response — the agent’s only window onto the environment |
| MCP | A standard protocol for connecting external tool servers to a harness |
| Permission request | A prompt asking the user to approve a tool call before execution |
| Permission mode | Which tool calls auto-approve vs. require confirmation |
| Agent mode | A named preset combining a permission mode with behavioural instructions |
| Sandbox | An isolated environment limiting the blast radius of unintended actions |
The one I emphasise: the tool call itself does nothing. The harness reads it, executes the function, and returns the result. If the harness doesn’t execute — permission denied, parse error — the action doesn’t happen, regardless of how confidently the agent described it. And the tool result is the agent’s only window onto your environment: without a tool for something, the agent is blind to it.
Sandbox. An isolated environment — container or VM — where the agent runs with restricted access. Sandboxing is what makes unattended runs practical rather than reckless.
4. Failure modes
| Term | Definition |
|---|---|
| Sycophancy | Agreeing with the user rather than being accurate |
| Hallucination | Confidently incorrect output |
| Knowledge cutoff | The date beyond which the model knows nothing |
| Attention degradation | Context influence weakening as a session grows |
| Smart zone / dumb zone | The observable shift from reliable to degraded output over session length |
Sycophancy. Frame prompts neutrally. “Review this implementation” gets a more honest assessment than “Is this implementation good?” — the second invites agreement.
Hallucination comes in two flavours with different remedies: factuality (the model asserts something wrong — fix by loading accurate docs into the session) and faithfulness (output drifts from what’s already in context — fix by clearing or compacting).
Smart zone / dumb zone. Early in a session, the agent has a focused context and produces reliable output. As content accumulates and attention degrades, quality visibly drops — on current frontier models this typically starts around 100,000 tokens. The single most useful habit: don’t push through the dumb zone. Compact or clear and start fresh. It always feels slower and it never is.
5. Handoffs
| Term | Definition |
|---|---|
| Clearing | Ending a session and starting fresh with an empty context |
| Handoff | Deliberately transferring relevant context from one session to another |
| Handoff artifact | A document produced by one session to carry context into the next |
| Spec | A handoff artifact describing a body of work spanning multiple sessions |
| Ticket | A handoff artifact scoping a single session with clear completion criteria |
| Compaction | Summarising session history to seed a fresh session |
Work too large for one context window has to be planned across sessions, and the written artifact — spec or ticket — is what carries the context across the boundary. For critical sessions, manual compaction with an explicit summary prompt beats relying on the automatic version.
6. Memory and steering
| Term | Definition |
|---|---|
| Memory system | An architectural layer persisting information across sessions |
| AGENTS.md | A project file loaded into every session as the agent’s standing brief |
| Progressive disclosure | Loading only what’s needed now, with pointers to more |
| Context pointer | A reference directing the agent to pull another document in |
| Skill | A self-contained task instruction document loaded only when relevant |
| Subagent | An agent spawned by another to complete a bounded task and return one result |
The model is stateless — the memory system is what creates the appearance of continuity. And the classic mistake with AGENTS.md: stuffing it. Keep it lean and use context pointers; the agent can pull detail in when it needs it.
Skill vs. tool trips people up constantly: a tool is a function the agent calls to act on the environment; a skill is a document the agent reads to understand how to perform a task.
7. Patterns of work
| Term | Definition |
|---|---|
| Human-in-the-loop | Engineers stay engaged — reviewing, redirecting, approving |
| AFK | The engineer steps away and the agent completes work unattended |
| Automated check | Deterministic pass/fail verification — tests, linters, type checks |
| Automated review | Non-deterministic evaluation of agent output by another agent |
| Human review | An engineer reads the actual output — the diff, not the description of it |
| Vibe coding | Accepting output on observed behaviour without reading the code |
| Design concept | The shared mental model between engineer and agent |
| Grilling | Making the agent ask clarifying questions before writing anything |
AFK runs are the throughput multiplier of AI-assisted engineering — and they belong in a sandbox, always.
Human review deserves its italics: narration is not review. Agents can accurately describe what they intended while the actual output contains errors. Read the diff.
Vibe coding is legitimate for low-stakes, easily reversible work. It is not appropriate for security-sensitive code or shared infrastructure — knowing which mode you’re in is the discipline.
Grilling is my favourite technique in the list. Before committing to a spec: “I want to build X. Ask me one clarifying question at a time about the design. For each question, suggest what you think the answer should be, and I’ll confirm or correct.” Ten minutes of this saves hours of the agent confidently building the wrong thing.
Further reading
- Dictionary of AI Coding by Matt Pocock — a broader community effort in the same spirit