Understanding AI Agents: From Simple Assistants to Autonomous Systems
Mapping the Landscape of AI Agents
Mapping the Landscape of AI Agents
Understanding AI Agents: From Simple Assistants to Autonomous Systems
Introduction
About me
- Dominik Lukeš
- [email protected]
- AI Consultant, AI Competency Centre
Michaelmas 2026 Mapping the Landscape Series
20 October · 12:30
27 October · 12:30
3 November · 12:30
10 November · 12:30
Today: seven questions
- What is an agent?
- Where did agents come from?
- How do they work?
- How do you choose a model for one?
- Which agents will you meet?
- What can go wrong?
- Where is this going?
What is an agent?
Dean W. Ball, Among the Agents, 2026
By the end of this year, the least important thing you will be able to do with frontier AI systems will be getting chatbots to answer questions, but this is still how most people will think of “AI.” Expect cognitive dissonance as a result.
Dean W. Ball, Among the Agents, 2026
What makes an agent useful?
York expenses: from email to completed forms
Blank form
Completed form
Completed word form
Agents in normal language
- Secret agent
- Travel agent
- Real estate agent
- Customer service agent
What makes a real estate agent?
- Knowledge of the rules and laws
- Knowledge of the local housing market
- Understanding of human needs for housing
- Ability to communicate
- Ability to interpret and follow instructions
- Access to databases and reference books
- Use of tools like Excel and Word
What do LLMs have?
- Some knowledge
- Understanding
- Ability to communicate
- Ability to interpret and follow instructions
What is missing
Ability to use tools like Excel or Email or database.
Until now!
Until now!
There is no one definition of "agent"
There is no one definition of "agent". Here is the one I use.
Agent definition
Model using tools in a loop to achieve complex tasks that require multiple steps.
Every AI app is two things
- Model
- Orchestration
From assistant to autonomous system
New chatbots are "agentic"
- User types prompt
- LLM answers
- User types prompt
- LLM makes plan
- LLM calls basic tools
- Virtual machine executes command
- LLM synthesizes results
- LLM shows results to user
Agents are just a series of chats running in the background!
Agents are just a series of chats running in the background!
Where did agents come from?
Agents in AI have a long history
An agent is just something that acts. Of course, all computer programs do something, but computer agents are expected to do more:
1 / 2
Agents in AI have a long history
operate autonomously, perceive their environment, persist over a prolonged time period, adapt to change, and create and pursue goals.
Russell and Norvig, Artificial Intelligence: A Modern Approach, 3rd ed., 2010
Warning: Agents have a long history in AI (most of it not relevant)
The old way: agents as rules
Until 2023, an agent was a program following rules written by hand: expert systems, software agents, multi-agent systems.
From orchestration to self-orchestration
2022: ChatGPT was a chat
The app sent your prompt, parsed the answer and showed it to you. All the orchestration was in the app.
You don’t need LLMs to orchestrate agents!
You don’t need LLMs to orchestrate agents!
Power Automate flows are sort of agents
You can enhance existing orchestrations with the power of LLMs.
You can enhance existing orchestrations with the power of LLMs.
A model inside a workflow
- The model drafts a reply to an email
- The model classifies an email or an image
- What happens next is written by a person, ahead of time: if this, then that
Microsoft called them agents too
2019: Microsoft launches Power Virtual Agents, a builder for scripted chatbots. In 2023 it becomes part of Copilot Studio.
2023: ChatGPT plugins
Only three plugins at a time. The model chose when to use one of them. The first element of self-orchestration.
Calculator rule
If you need to multiply numbers, emit calculator(a, b) rather than guessing.
Calculator rule
Tool calling
- Models fine-tuned to generate a specific JSON shape reliably
- Originally called function calling, then tool calling
- Same idea: the model decides when to invoke a tool
- The harness defines the tools; the model picks
June 2023: Bard starts running code for sums
Google: running code in the background made Bard's answers to word and maths problems about 30% more accurate. It still often got them wrong.
The “first” agent: BabyAGI
BabyAGI · 2023.
- Three "agents" — task-creation, prioritisation, execution
- Each agent = one LLM call, one prompt template
- ~140 lines of Python around them
- The Python loop was the intelligence
Brittle
Models in early 2023 were not reliable enough to behave well across long sequences of these calls. A single bad turn stalled the loop.
Agents are simple workflows
GPTs and Copilot Agents are also example of tools
GPTs and Copilot Agents are also example of tools
Most Copilot agents are just GPTs
Late 2024: reasoning models
Before answering, the model produces a planning string. More like brainstorming than logic.
2025: tool calls inside reasoning
Mid-thought, the model can fire off a search and read the result before continuing.
First, in coding: many files at once
- Old: better autocomplete · GitHub Copilot
- New: "build me a website that does X"
- Plans the files · writes them · runs them · notices errors · revises
- Output: files in folders, not snippets in chat
2025 was going to be the year of agents
And it was, for writing code
The agents of 2025 mostly wrote code: a task where a loop can run for a long time and check its own work.
Then people noticed it works for knowledge work
The tools a coding agent uses to create and edit files work just as well on documents, data and projects.
BUT
BUT
Something happened in November 2025
Something happened in November 2025
The inflection: three models
Late November and early December 2025: Claude Opus 4.5 first, then GPT-5.2 and Gemini 3 Flash. Much more reliable in long loops.
New model capabilities
- Awareness of problems and routes to solutions
- Ability to make plans
- Trained on purposeful data
- Awareness of which problems require external tools
- Ability to formulate commands based on problem type
- Trained on real environments with Reinforcement Learning
Orchestration is moving into training
A reminder from last week: how fast models are released
Gemini and Muse releases, with the Flash models
And how much they improved
How long a task can a model finish?
BabyAGI and today, side by side
The loop was there from the beginning
What was missing was the judgement of a smart model: to design the loop for the task in front of it, and to adapt it when the situation changes.
History and future together: four eras
Oracle, tool maker, tool user
- Capabilities: answer questions, summarise, translate
- Chatbots: ChatGPT, Gemini, Claude
- Specialist apps: NotebookLM, Elicit, Consensus
- Capabilities: write code, manage code base
- Outcomes: create scripts, dashboards, simulations and workflows
- Tools: Cursor, Lovable, Google AI Studio
- Capabilities: plan, work with files, run utilities
- Outcomes: more ambitious projects, manage data and replicate analyses
- Desktop agents: Codex, Claude Code, Antigravity
How do agents work?
The agentic loop
An agent is a model using tools in a loop
- sends prompts to the model
- asks you for permission
- runs the command on your computer
- sends the result back to the model
- interprets what you asked
- makes plans and considers options
- writes a command for the harness to run
- interprets the result
One real step, round the loop
Open https://agents-data.learnweaver.app and follow its setup instructions in this folder.
open_page {"url": "https://agents-data.learnweaver.app"}Tool results
Internal Error
The website reader failed. Try curl for the plain-text setup instructions.
shell curl --fail --location --max-time 45 https://agents-data.learnweaver.app/agents.mdTool results
(the setup instructions, as plain text)
The website reader failed, but curl retrieved the setup instructions. Confirm setup in this folder: ~/Desktop/workshop
Brain, body, tools
Where are last year’s evaluation numbers?
I need to find the evaluation numbers from last year, I think they were in a report or a presentation. I think it was Geoff who now left who did it last February.
Where are last year’s evaluation numbers?
Human: searching from memory
Agent: following evidence beyond the prompt
I need to find the evaluation numbers from last year, I think they were in a report or a presentation. I think it was Geoff who now left who did it last February.
Search by subject across the year; treat the remembered name and month as clues. Extract document text, then compare authors, dates and figures.
search_files evaluation, feedback, outcomes (2025, including drafts and presentations)Tool results
Programme review draft May programme update
read_files programme review draft; May programme updateTool results
Draft: George; analysis completed in March. Presentation: delivered in May; cites the March analysis.
compare_figures responses, rating, analysis, authorTool results
Both files: 148 responses; average rating 4.3 out of 5.
Found them: 148 responses, with an average rating of 4.3 out of 5. George did the analysis in March. The figures appear in the draft programme report and the presentation delivered in May. Your memory of Geoff and February pointed us towards the wrong files.
Why the agent searched better than I did
- Treat remembered details as clues.
- Search, extract and compare.
- Return the answer with its sources.
- Judgement: George, not Geoff; March, not February
Tools Agents can use
- Search files
- Search web
- Run commands
- CLI tools
- APIs
- Other models
- Custom scripts
- Browser
- Applications 🆕
How does a model generate text?
- Facts
- Background knowledge
- Skills
- Prompt
- Chat history
- Attached files
- Found files
And nothing else
And nothing else
What's in an actual context window?
- About 38k: instructions, tools, memory, skill descriptions, before your first message
- The rest (78.7k): messages already in this chat
- One installation: your numbers will differ. Claude Code: type
/context
A desktop agent works on files in a folder
A desktop agent works on files in a folder, not on a chat. What it makes stays in the folder when the chat is gone.
Instructions in files: AGENTS.md
Skills: instructions loaded when needed
- 1It always sees each skill's name and description
- 2When your task matches, it reads the full SKILL.md
- 3It opens references and runs scripts only when it needs them
yaml---
name: pdf-extract
description: Extract text and tables from PDFs. Use when the user asks to read or summarise a PDF.
---
# Body — only loaded when the model asks for it
Detailed instructions, options, scripts, examples...Skill use illustrated
Make a PowerPoint explaining five ways students can use coding agents.
read_file skills/pptx/SKILL.mdTool results
Loaded the PowerPoint skill.
read_file skills/pptx/references/html-authoring.mdTool results
Loaded guidance for slide sizes and text placement.
run_script build-deck.jsTool results
Converted the HTML slides into coding-agents.pptx.
Your PowerPoint is ready: coding-agents.pptx.
Connectors: one plug for many apps
- Model Context Protocol (MCP): a shared way for agents to connect to apps and data, since November 2024
- Connectors to email, calendar, OneDrive, Notion, Zotero, databases
- Each connector adds tools the model can call, and data it can reach
- A shared standard: given to a neutral foundation in December 2025
Memory is notes
It tells the hand to open a notebook and write a note. The notebook is now part of memory.
Where do you keep the context?
- Notes, instructions and records as plain text
- Any agent can read them
- The place to start
- The same files, with their full history
- Every change can be seen and undone
- Shared between people, agents and machines
- For many records of the same kind
- The agent queries them instead of reading everything
- A workspace that people and agents both edit
- Reached through a connector
- Becoming very popular
Notion on why agents need a shared memory
As software engineering changes, tribal knowledge is increasingly concentrated not only in a single person's mind, but in a single agent session. … every org has its own lore.
Hesham Salman, "Building Shared Memory for AI Agents in Notion", August 2026
Keep the context where it is yours
Keep your context in a place that belongs to you, not inside one chatbot: files any agent can read, with a history.
One agent or many
How do you choose a model for an agent?
What a model needs to power an agent
- Good judgement
- The ability to choose tools
- The ability to make plans
- Coherence over long loops
- Managing its own context
Who leads on agentic work today
The agentic score is not the overall score
Different criteria
The criteria for a model that can power some kinds of agents are slightly different from the criteria for the most advanced model.
Local open models are not a full replacement
Long tasks need the smartest model
In a single answer, a small mistake is one mistake. In a long loop, every later step builds on it.
Which agents will you meet?
Kinds of agent you will hear about
- Deep research: searches, reads and writes a report with sources
- Coding agents: build and change software, and now much else
- Computer and browser use: click and type in applications and websites
- Workflow agents: run a fixed process with a model at some of the steps
- Personal assistants: always on, acting for one person
Where agents run
A sidebar: agentic browsers
2025
- Agentic browsers are all the rage: ChatGPT Atlas, Perplexity Comet, Dia
August 2026
- OpenAI retires ChatGPT Atlas
Now
- Nobody talks about them as a category: the models use a browser as one more tool
Agents inside the office tools you know
- General release June 2026
- Runs long tasks in the cloud
- Off until an administrator turns it on
- Launched July 2026
- Works across your files and connected apps
- Same agent technology as Codex
- General release April 2026
- A desktop agent for people who do not code
- On the web and phones since July
ChatGPT Work on web vs desktop
- Gives you virtual machine
- Uses connectors to get to files
- Works on your local files
- Functionally equivalent to Codex
Where did OpenClaw come from?
Spring 2025
- Claude Code launches
Autumn 2025
- People start noticing its possibilities for more than coding
- Peter Steinberger starts experimenting and releases ClawdBot
January 2026
- ClawdBot generates interest
- ClawdBot renamed to MoltBot and then to OpenClaw
- Other Claws start to emerge
February 2026
- OpenAI hire Peter Steinberger
A desktop agent and a claw
- Runs on users machine
- User types directly into Codex
- Acts based on user prompts
- Runs on a remote machine
- User communicates via WhatsApp
- Acts on a schedule
Jensen Huang
Every company in the world today needs to have an OpenClaw strategy, an agentic system strategy.
This is the new computer."
Jensen Huang, 17 Mar 2026
2026: assistants that are always on
- February: OpenClaw: runs on your own computer, talks on WhatsApp
- 19 May: Google Gemini Spark
- 11 August: SpaceXAI Grok Bot
- 8 September: Meta Muse, hosted for you
- 29 September: OpenAI dots, hosted for you
- 21 October: Anthropic Claude Duet, research preview
Prices of chatbots and agents
- ChatGPT Plus: $20 a month
- Full agent use: $100 to $500 a month
What did the expenses claim cost?
- The original task and the demonstration used 64.76 million tokens.
- At 21 September API list prices, the estimated cost was $39.94.
- On the subscription meter, it represented roughly 6% of an average Plus month or 1.2% of Pro, based on weekly changes.
On OpenRouter, three quarters of the traffic is agents
OpenRouter classed 76% of the text it processed in October 2026 as coming from agents. A year earlier it was 8%.
What can go wrong?
What can go wrong when it works on your files
| Risk | Mitigation |
|---|---|
| Unintended file edits / deletions | Add clear guidelines to your custom instructions, stay in plan mode |
| It can take or suggest actions you do not fully understand | Get ChatGPT to explain unclear actions and suggestions |
| Files could be deleted and unrecoverable | Work in a OneDrive folder or with backups |
| Changes can be made on mass and rapidly | Get ChatGPT to create a changelog for each project |
What a permission request looks like
What protects your data when an agent works
OpenAI on its own incident
without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed
OpenAI, report on the Hugging Face incident, August 2026
Local folder ≠ local model
The files can be on your computer while the model processes information online.
People are doing more things than ever
I'm now more productive than ever. But I'm exhausted.
Senior researcher
Where is this going?
The same request, five months apart
- Seven pictures as a slideshow
- ffmpeg to join them
- Music downloaded, licence checked
- An animated story with dialogue
- Two voices designed and generated on this computer, then checked by speech recognition
- Music generated on this computer
- The agent measured its own output (loudness, words, voices) and fixed it
Better judgement: it tests instead of guessing
Ask a model in 2025 how much to compress a picture: it guessed "80%". Ask one today: it tries several levels, shows them side by side and gives you a table. Evidence, not a guess.
Agents in science, 2026
| System | What the agents did | What people still did |
|---|---|---|
| The AI Scientist (Nature, March) | Proposed ideas, ran experiments, wrote the papers | Filtered the output at every stage |
| Co-Scientist (Nature, May) | Generated and ranked hypotheses and drug candidates | Ran and judged every laboratory test |
| Robin (Nature, May) | Proposed a test and a drug candidate for an eye disease | Chose the candidates and ran every experiment |
How researchers describe agentic systems
Agents and society: a new ethics
- "We need a new ethics for a world of AI agents" (Nature, August 2025)
- Agents act for people, and with each other, at scale
- Open problems: whose interests an agent serves, trust, and the power of the companies that run them
- First step proposed: better ways to evaluate what agents do in the world
Open questions
- Identity: whose agent is this, and who allowed it?
- Money: how does an agent pay for something?
- Agent to agent: how do agents from different companies work together?
- Responsibility: who answers for what an agent did?
- Measurement: the tests can no longer measure the longest tasks
A health warning
Anything I say today about these tools will be out of date within six months.
What to do now
- Collect more recordings, documents, videos, images, kept in folders
- Write down what you know institutional knowledge, SOPs, histories of everything, to build instructions for agents
- Build as few barriers to access to data
Coming up
Hands-on workshops on agents
20 November · 10:00–16:00
30 November · 10:00–16:00
Join MondAI for Latest News every second Monday
