What an AI agent actually is

“Agent” is currently the most used and least defined word in AI. Strip away the marketing and an agent is something quite concrete: a model that can act, surrounded by four decisions that determine whether it works. One picture, then the five parts one by one.

Session context: what the agent sees right now

The session context is the agent’s short-term memory: your question, the files it just opened, the results of the tools it just ran. It lives in the model’s context window, the limited working space a model can read at once, and it’s gone when the session ends.

This part is a solved problem. It’s cheap, reliable, and every serious framework handles it. If a vendor’s big claim is “our agent remembers the conversation”, that’s table stakes, not a feature.

Procedural memory: the rules that always apply

Procedural memory is static content loaded into the agent every single time: the rules of the house. Your security requirements, your compliance constraints, the tone it may use, the things it must never do.

The important property is that this memory does not learn. That’s a feature, not a gap: security and compliance requirements shouldn’t drift because the agent had an interesting week. They change when a human changes them, deliberately and reviewably.

Long-term memory: the hard part

Long-term memory is everything the agent should keep across sessions, and it splits into kinds: episodic (what happened: meetings, decisions, tool calls), semantic (what is asserted: entities, facts, and since when), and cognitive (what we concluded from it, and with how much confidence).

This is the part that is genuinely difficult, and it’s why long-term memory is still a research topic rather than a checkbox. The core problem: errors in memory don’t just sit there, they get reasoned from and multiply. Two consequences follow. Retrieval should expose enough provenance and scoring detail to inspect why a record was used; a hash can check integrity, not truth. And memory should be scoped per use case so an error that does get in has a smaller blast radius. We wrote both up in detail: Memory an AI agent can inspect and Don’t build a company brain. The architecture and current evidence boundary are documented in MEMA research and its status page.

Runtime: where does the agent actually live?

An agent has to run somewhere, and that somewhere is a real architectural decision, not an implementation detail. The current options span a wide range: a client on the user’s own machine (a Mac in the simplest case), a managed cloud platform such as Microsoft Foundry, or agents that run inside your delivery pipeline, like GitHub’s agentic workflows.

The runtime decides where your data flows, which permissions the agent physically has, and which compliance regime applies. An agent on a laptop sees what the laptop sees. An agent in the pipeline can touch every repository it’s granted. Choose the runtime for the risk, not for the demo.

Model: chosen per use case, not per loyalty

Last, the part most discussions start with: which model. It matters, but it’s the most replaceable piece of the five. Deep reasoning over a legal document, fast triage of support tickets, code review: different use cases genuinely favour different models, whether that’s Claude, GPT, Gemini or an open-weight model you host yourself.

The practical consequence is to stay vendor-neutral: build the agent so the model is a configuration choice, and let everything around it, context, rules, memory, runtime, carry the value.

The five questions

So when someone says “we built an agent”, the definition above turns into a checklist: What does it see in the session? Which rules does it always carry? What does it remember long-term, and can you verify that memory? Where does it run? And which model, chosen for which use case? The demo usually answers the first and the last one. The value, and the risk, live in the three in the middle.