AI and Expertise: Judgement, Learning, and Work in the Age of Large Language Models

Dominik Lukeš (dominiklukes.net)

handouts.fyi/xmtg

About me

My work

Dominik Lukeš dominiklukes.net

AI Competency Centre University of Oxford

AI Trends Tracking

ainewsroundup.pages.dev

What I used to do

Reading and Writing Innovation Lab

What I really do

Applied epistemology

What I teach people

How to become better at learning

DeliberatePractice.DominikLukes.net

What happens to knowledge and expertise with Large Language Models in the world?

ChatGPT Timeline

30 Nov 2022

ChatGPT is released

7 Dec 2022

1 million people have used ChatGPT

2023-2024

100s of millions people use ChatGPT to code, write, translate, learn and ... cheat

Sept 2025

1 billion people use ChatGPT every week

2026AI agents happen

What happens to knowledge and expertise with Large Language Models in the world?

What is ChatGPT?

Large Language Model

outputs tokens
provides rich knowledge and judgment
limited in precision and consistency

Orchestration

parses tokens to run them
provides absolute logic

What happens to knowledge and expertise with Large Language Models in the world?

Large Language Model is a Universal translator

language-to-language
style-to-style
unstructured-to-structured / structured-to-unstructured
text-to-code / code-to-text
image-to-text / text-to-image
question-to-answer / answer-to-question
problem-to-plan

What happens to knowledge and expertise with Large Language Models in the world?

Sometimes LLMs are a literal translator

What happens to knowledge and expertise with Large Language Models in the world?

Orchestration

parses formatting
interprets commands
passes commands to computer
sends outputs from computer to model

What happens to knowledge and expertise with Large Language Models in the world?

Two parts of intelligence

Pure cognition

recognise objects
recall ideas
mental manipulation
thinking about problems

Cognition plus tools

keeping mental lists
look up facts
creating budgets
asking questions

What happens to knowledge and expertise with Large Language Models in the world?

Strengths of cognition and tools

Cognition

Judgement
Knowledge
Comparison
Estimation

Tools

Precision
Exact retrieval
Complex conditional
Precise calculation

What happens to knowledge and expertise with Large Language Models in the world?

Intelligence as Grep and Grok

grepandgrok.fyi

What happens to knowledge and expertise with Large Language Models in the world?

What do humans share with Large Language Models

Cognitive baseline

Tools

Intelligence

What happens to knowledge and expertise with Large Language Models in the world?

What tools do we use

What happens to knowledge and expertise with Large Language Models in the world?

What tools do we use

Humans

Mental checklists
Focus
Body parts
Physical objects
Computers
Relationships

Large Language Model

Shell commands
CLIs
APIs
Scripts
Search

What happens to knowledge and expertise with Large Language Models in the world?

What can artificial "intelligence" do with tools today

Vivian Balakrishnan, Foreign Minister of Singapore

Al agents have crossed a threshold I didn't expect so soon. Not just impressive demos - but practical tools for daily use.

What happens to knowledge and expertise with Large Language Models in the world?

What does it mean for humans?

Work is amplified
More people get access to that work
No job for someone to do the work

What happens to knowledge and expertise with Large Language Models in the world?

Is this true?

Access to an LLM does not replace:
- Domain expertise - Architectural experience - Understanding of trade-offs - Awareness of what is not written in the documents
AI democratizes capability. It does not democratize judgment.

Alex LeBlanc (AI Amplifies Expertise, Not Replaces It \| LinkedIn)

What happens to knowledge and expertise with Large Language Models in the world?

Why are you studying at university?

Change your brain
Change the world
Prove to the authorities you can pass an exam

What happens to knowledge and expertise with Large Language Models in the world?

There's a hidden conflict

Changing your brain to change the world is in conflict with changing your brain to pass exams.

What happens to knowledge and expertise with Large Language Models in the world?

Can LLMs help us resolve this conflict?

What happens to knowledge and expertise with Large Language Models in the world?

True fact about learning

The amount of time and effort required to change your brain so that you can do something is constant.

What happens to knowledge and expertise with Large Language Models in the world?

Nobody can change your brain for you

What happens to knowledge and expertise with Large Language Models in the world?

Effort spent on building pyramids

Pyramid

Food for workers

Housing for workers

Ramp to pyramid

What happens to knowledge and expertise with Large Language Models in the world?

Effort spent on learning

Learning subject

Learning skills

Finding resources

Organising your life

What happens to knowledge and expertise with Large Language Models in the world?

Role of LLMs in learning

Giving feedback

Develop scaffolding

Building learning tools

Scheduling time

Use AI as a ramp to higher learning.

What happens to knowledge and expertise with Large Language Models in the world?

What LLMs can't do for you

Learning

What happens to knowledge and expertise with Large Language Models in the world?

What LLMs cannot do at all

Learn from experience

What a new expertise can look like

What was the last time you used ChatGPT?

What a new expertise can look like

I haven't ChatGPT regularly in months and not at all last week.

What a new expertise can look likeI have been using Codex every day

What a new expertise can look like

I have been using Codex every day

What a new expertise can look like

What this means

I am still using AI
I am using the same model power in a different context
{icon=trending-up} I can get much more done

What a new expertise can look like

With Codex and Claude Code I have

Scheduled my calendar to come here
Built my own slide presentation tools
Organised old file archives
Rescued old websites and built new ones
Build my own apps for
- Work out tracking
- Photo management
- Searching my highlights

What did I need to know to do this?

What did I need to know to do this?The knowledge needed today

What did I need to know to do this?

The knowledge needed today

Knowledge you don't need

Complex algorithms
Syntax of computer languages
Variables, functions, etc.
How to use developer tools

Knowledge you do need

How your computer works
Principles of software architecture
Developer tools landscape
How AI agents work
What is possible with software

What did I need to know to do this?

Some useful things to know

Computer

File system details
Plain text / Markdown vs Rich Text
CLI tools
Pipes and deamons

Development

Libraries
Git and GitHub
What an API can do for you
Hosting (Cloudflare)

Software architecture

When you need a database
Refactoring
Content vs display
Security concerns

Agents

Model v harness
Model tool use
Model capabilities

What did I need to know to do this?

Big lesson from the agent revolution

It is now worth learning more about computers at every level because you can now do something about it.

What did I need to know to do this?

Yes, you do need judgment

What did I need to know to do this?

What kind of judgment do you need?

Fast

What something is
Something may be a problem

Slow

How to backtrack through a problem
How to evaluate a plan step by step

What did I need to know to do this?

Problem with "critical thinking"

Good slow judgment needs good fast judgment.

What did I need to know to do this?

What is fast judgment like?

Fluency in language
Participation in a game
Playing a sport

What did I need to know to do this?

You cannot act in a world and look up everything. You must know things in a special way.

What did I need to know to do this?

You cannot just have a list of facts and rules for

speaking a language
participating in games
being active in sports
playing music

What did I need to know to do this?

You cannot

look up words and speak
look up rules and play
follow manual and play

What did I need to know to do this?

You must have knowledge that can only be developed through

Repetition

Reflection

Repetition

Reflection

What did I need to know to do this?

You cannot evaluate claims and look up every concept or fact. You must know these in a special way.

What did I need to know to do this?

How do you develop good fast judgment?

Time

Effort

Learning

Future

Why are you at university

Knowledge and skill
Relationships
Opening doors

Future

What is the impact of AI on the system?

Knowledge is less valuable
What is behind the door is different
Some doors disappear

Future

What is the impact of AI on you?

You can learn more
You can do more once you're through the door
You can make new doors

Future

Will AI make learning easier?

No. You now must learn more, work harder, think harder. There will be less pseudo learning. There are no excuses for not learning what you need to know.

Future

Will AI replace your need to learn?

Future

Lessons from chess

Machines are better than best humans
More people play chess than ever
Best people practice with chess machines

Future

Why are people getting better at

Musical instruments
Sport
Coding

Future

New world

YouTube

More variety of learning sources
Easier access to instruction

Podcasts

Easier access to expert thinking

Future

Finally

You live in the golden age of learning.
It is now worth learning more not less.
Learn because you can
Learn with and about AI
Don't give up

Future

You must

Use all the tools to change world.
You have some of the most powerful tools available.
Do not waste them!

Thank you

Dominik Lukeš (dominiklukes.net)

handouts.fyi/xmtg