The pitch for AI-only training is seductive: hand each learner a tireless tutor that watches every move, scores it in real time and never needs a room booked. For knowledge that lives in a manual, that works. For skill that lives in the hands and the nerve, a recent controlled trial suggests something more awkward. The human in the loop is doing the heavy lifting, and taking the human out makes learners measurably worse.
A 2026-reported McGill randomised trial in JAMA Surgery had learners practise on a surgical simulator under three conditions: an AI tutor alone, or a human educator coaching them, with one arm using AI performance data to tailor that coaching. The human-plus-AI group transferred their skill to a harder task significantly better and made fewer risky errors than the AI tutor alone. Practice still needs a person.
Why this matters now
The fastest-moving corner of corporate learning right now is AI role-play and AI tutoring: bots that run a sales objection or a compliance scenario, score you, and let you go again. The convenience is real, which is why it is worth pausing on a trial that asked a precise question: does the human educator add anything an AI tutor cannot? In February 2026 the American Journal of Managed Care surfaced exactly that study, and the answer is not the one the AI-only pitch wants.
What the research says
The study is a single-blinded randomised clinical trial from the McGill Neurosurgical Simulation and Artificial Intelligence Learning Centre, published in JAMA Surgery on 6 August 2025 (Giglio and colleagues, including Rolando Del Maestro). It analysed 87 medical students with no prior experience on the NeuroVR simulator, split into three arms. Group 1 received feedback from an AI tutor alone, which scores surgical skill from -1.00 (novice) to 1.00 (expert) every fifth of a second. Group 2 got a human educator using the same words as the AI. Group 3 got a human educator who used the AI's error data to tailor personalised feedback.
The point of training is not the score on the exercise you just practised. It is whether the skill holds up on something harder you have not seen. On that transfer task, the human-plus-AI group scored significantly higher than the AI tutor alone: a mean difference of 0.20 on that -1.00 to 1.00 scale (95% CI 0.06 to 0.34, P = 0.02), and 0.26 on a late practice trial (P = 0.01). It also came out ahead on the metrics that matter most in an operating theatre, bleeding and injury risk. The authors' conclusion is blunt: personalised expert instruction produced better performance and skill transfer than the AI tutor alone, highlighting the importance of human input and participation in AI-based surgical training.
Surgery is not a sales call, but the shape of the finding travels: a human coach, armed with good data, beats an automated tutor at turning practice into a skill that survives a harder, novel situation. That rhymes with our own work. Sidestream's academic behaviour-change research, building on work from UCL, Cambridge and Bocconi, found immersive role-play around 20% more effective than passive modalities such as slide-shows and video e-learning at teaching communication skills. The mechanism is the same: a person, practising under realistic pressure, with a human reading the moment and feeding it back.
What most organisations do (and why it fails)
The default reaction to a squeezed training budget is to automate the practice and keep the content: roll out an AI role-play tool, let people drill against the bot, and read the completion dashboard as if it were learning. An AI tutor is good at the part that is easy to measure, the score on the thing you just did, and silent on the part that decides whether training was worth it: whether the skill shows up later when the situation is unfamiliar and the room is real.
The deeper trap is that learners cannot see the gap themselves. People reliably over-rate their own skill, the Dunning-Kruger effect in plain sight, and a bot that hands out a friendly score after every attempt feeds that false confidence rather than puncturing it. You end up with people who have practised a lot, scored well, and learned to perform for the machine. The human educator in the McGill trial was not a cost to be removed. They were the part doing the teaching.
What works
Some things cannot be taught, they have to be felt, and they have to be coached. Use the technology for what it is genuinely good at: data, repetition, a safe place to fail at 2am. Then put a person in the loop for the part that changes behaviour: reading what a learner actually did, naming it without flinching, and turning a number into a moment they will not forget. The evidence is clear, that combination beats the automated tutor working alone.
That is how we build practice. In our simulation training, learners make live decisions under realistic pressure while trained actors push back and a facilitator reads the room and feeds it back: the human coaching the McGill trial found indispensable. It is the same logic that makes experiential learning stick where a slide deck slides off, and the same reason immersive training outperforms e-learning on skills that only show up under load. Practice scales. Coaching is what makes it count.
Where Sidestream fits
We are a behaviour change consultancy that combines organisational psychology with immersive theatre, and the human-versus-AI question sits at the centre of how we design practice, from workshop-scale training through larger immersive simulations to tailored behaviour change programmes. If your people are drilling against a bot and you are not sure the skill is landing, book a free 30-minute diagnostic call and we will talk it through.
Human-Led Practice vs AI-Only: The Takeaways
A 2026-reported McGill randomised trial in JAMA Surgery tested an AI tutor against a human educator coaching learners through simulated practice. The group coached by a human using AI data transferred their skill to a harder task significantly better, and made fewer risky errors, than the AI tutor working alone. The lesson for corporate learning is not anti-technology. It is that practice scales, but a human in the loop is what turns practice into a behaviour that holds.
- In the McGill JAMA Surgery trial (Giglio et al., 6 August 2025, 87 students), the human-plus-AI group beat the AI tutor alone on skill transfer by a mean of 0.20 on a -1.00 to 1.00 expertise scale (P = 0.02), and on a late practice trial by 0.26 (P = 0.01).
- The human-coached group also performed better on safety metrics: bleeding and injury risk. The authors stress the importance of human input in AI-based training.
- AI tutors score the part that is easy to measure and miss whether the skill shows up later. Learners over-rate themselves (Dunning-Kruger), and a constant friendly score feeds that false confidence.
- Sidestream's UCL, Cambridge and Bocconi research found immersive role-play around 20% more effective than passive e-learning. Roediger and Karpicke (2006) found active retrieval raised retention by around 50% over re-reading.