Start here · Guide 1 of 10
Get real work done faster, and keep learning while you do it.
Professionals doing writing tasks in their own field took 40% less time with ChatGPT, and quality rose by 18%.
Try this
Here are my notes: [paste notes]. Draft a 200-word update for [reader]. Keep my facts as they are and list anything you had to assume.
A well-designed AI tutor beat an active learning class in one trial. Tutors that make you do the thinking protect what you learn.
Try this
Be my tutor on [topic]. I am at [level]. Ask me one question at a time and do not give the answer until I have tried.
Novice support agents resolved 34% more issues per hour with an AI assistant. Experienced agents saw minimal change.
Try this
I am new to [task]. Show me what a good one looks like, then give me the checklist an experienced person would use.
On tasks within GPT-4's abilities, consultants improved by nearly 40%. Use it to widen your thinking, and keep the decision.
Try this
Here is my plan: [paste plan]. Give me the three strongest reasons it could fail, and what you would ask me before approving it.
Say who it is for, what it is for and what good looks like. Paste the material it needs.
Treat the first answer as a starting point, then tell it what to change.
In one small trial, people who asked conceptual and follow-up questions scored higher on a later quiz. That pattern is correlational.
Open the sources, and confirm names, numbers and quotes before you use them.
If you need the skill, try first and test yourself with the tool closed.
16 experienced developers took 19% longer with AI allowed, yet believed they were about 20% faster. Time your work before deciding it helps.
On a task beyond GPT-4's abilities, consultants using it did worse by an average of 19 points. Be most careful where you cannot judge the answer.
Students with plain GPT-4 did 17% worse once it was taken away. In another trial, the AI group scored 50% on a quiz against 67% without it.
Notes, evidence and sources
7 sources, each with its catch.
Noy and Zhang, Science · July 2023
453 professionals doing writing tasks in their own field took 40% less time with ChatGPT, and quality rose by 18%.
Set tasks in an online experiment, with an early model. Only the abstract was read.
In a preregistered online experiment published in Science (Noy and Zhang, July 2023), 453 college-educated professionals did occupation-specific writing tasks, and half were randomly given ChatGPT. "The average time taken decreased by 40% and output quality rose by 18%." Inequality between workers decreased.
Only the abstract was read. These were set tasks in an experiment, with an early version of ChatGPT, so real jobs may differ.
Strength: Strong for what it measured
Kestin et al. · June 2025
In a randomised trial, college students learned significantly more in less time with a custom AI tutor than in an in-class active learning lesson.
A purpose-built tutor, not a general chatbot. Only the abstract was read.
In a randomised trial at college level (Kestin et al., 2025; full article), students "learn significantly more in less time" with an AI tutor than in an in-class active learning lesson, and felt more engaged and motivated. The tutor was custom-built on the same teaching practices as the class.
Only the abstract was read, so effect sizes and sample size are not given here. This was a purpose-built tutor, not a general chatbot.
Strength: Moderate
Brynjolfsson, Li and Raymond · 2025
An AI assistant raised issues resolved per hour by 14% across 5,179 support agents, and by 34% for novices, with minimal change for experts.
One kind of work. Only the abstract page was read.
A study of 5,179 customer support agents (Brynjolfsson, Li and Raymond; published in the Quarterly Journal of Economics, 2025) found an AI assistant raised issues resolved per hour by 14% on average, "including a 34% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers."
Only the abstract page was read. It covers one kind of work, and the tool was introduced in stages rather than in a lab trial.
Strength: Strong for this setting
Dell'Acqua et al., via MIT Sloan · October 2023
Across more than 700 consultants, GPT-4 improved performance by nearly 40% on tasks it could do, and cut it by an average of 19 points on a task it could not.
From MIT Sloan's summary, because the paper could not be opened.
A study with more than 700 Boston Consulting Group consultants (Dell'Acqua et al.), as summarised by MIT Sloan, found GPT-4 could improve performance by nearly 40% on tasks within its abilities. On a task outside them, performance dropped by an average of 19 percentage points.
This rests on MIT Sloan's summary because the paper's page refused access. The summary's 19-point average is not reconciled with its separate 13 and 24 point figures.
Strength: Moderate until the paper is read
Bastani et al., PNAS
Nearly 1,000 maths students with plain GPT-4 scored 48% higher in practice, then 17% lower on later exams without it. A tutor version built to protect learning largely avoided the drop.
One study, and only the abstract was read.
A field experiment with nearly 1,000 high-school maths students (Bastani et al., PNAS abstract; working paper) found GPT-4 access raised practice grades by 48% with a standard chat interface and 127% with a tutor version built to safeguard learning. When access was removed, the standard group did 17% worse than students who never had it. The authors say the safeguards largely mitigated this.
The abstract was read, not the full paper. Peer-reviewed. That it was one school in Turkey comes from search summaries.
Strength: Moderate
Anthropic · 29 January 2026
52 mostly junior engineers finished in a statistically similar time, but the AI group scored 50% on a later quiz against 67%.
A small vendor trial, tested shortly afterwards, using a chat assistant.
In an Anthropic randomized trial (29 January 2026), 52 mostly junior engineers learned the Trio library. Completion time was not significantly different (about 2 minutes faster with AI, p = 0.391). On a later quiz the AI group scored 50% and the no-AI group 67% (p = 0.01), with the largest gap on debugging. Lower-scoring usage patterns delegated code or used AI only to debug; patterns averaging 65% or higher asked conceptual or follow-up questions.
Anthropic is an AI vendor studying its own product category, although the result is unflattering. The authors say the pattern analysis is correlational, the sample small, comprehension was measured shortly afterwards, and the tool was a chat assistant, not an agentic one.
Strength: Moderate (small RCT)
METR · 10 July 2025
16 experienced developers took 19% longer with AI allowed, yet still believed they were about 20% faster.
Small, one setting, early-2025 tools. METR now marks the results as out of date, and does not claim AI slows most developers.
In a randomized trial by METR (published 10 July 2025), 16 experienced open-source developers worked on 246 real issues, with AI tools randomly allowed or not. When AI was allowed, tasks took 19% longer. The developers had expected a 24% speedup and, afterwards, still believed it was about 20%.
The authors list limits: a small sample, one setting, early-2025 tools and possible sampling bias. They say they do not claim AI slows most developers. METR's page now carries a notice that the results are out of date.
Strength: Strong for what it measured, weak for generalising
1. Noy and Zhang, Science · July 2023
2. Kestin et al. · June 2025
3. Maths students in Turkey (Bastani et al.)
4. Brynjolfsson, Li and Raymond · 2025
5. Dell'Acqua et al., via MIT Sloan · October 2023
6. Anthropic's trial on learning a library · 29 January 2026
7. METR's 2025 randomized trial · 10 July 2025