A research collaboration involving Moorfields Eye Hospital NHS Foundation Trust, the UCL Institute of Ophthalmology, and Lufthansa, has warned of the “automation paradox” involving the deskilling of clinicians with the introduction of AI and automation, making a series of recommendations to prevent what it refers to as the erosion of human skills and awareness in the healthcare space.
Explaining the automation paradox outlined in the study, researchers share that this refers to a loss of manual flying skills resulting from a dependence on autopilot’s capabilities. They suggest that rather than seeing AI as an autopilot, it should be embraced within the health and care sector as a “digital copilot”. Cooperation between clinicians and flight safety experts thus sought to explore meaningful comparisons and how automation has reshaped pilot expertise, as a way of gaining insight into what this could mean for clinicians using AI.
Five recommendations are put forward: monitoring real-world clinician performance without AI assistance and implementing minimum unaided practice requirements; prioritising independent reasoning skills prior to the introduction of automation; ensuring clinicians understand AI limitations; introducing mandatory simulation training on AI failures; and cultivating “operational understanding” of how AI tools make decisions and when to override them.
Lead author Ariel Ong commented: “Medicine risks repeating aviation’s early automation mistake of placing too much faith in the machine while losing critical skills. Aviation learned that the goal was never to replace the pilot, but to enable rigorous simulation training. We argue for the need to embrace that same philosophy to ensure clinician judgement is not eroded as AI becomes increasingly embedded in healthcare.”
The study’s authors conclude that regulation on AI should look to address competence and accountability with humans and AI functioning as “co-intelligent” partners.
Citation: Ong, A.Y., Merle, D.A., Pollreisz, A. et al. Flight rules for clinical AI: lessons from aviation for human-AI collaboration in medicine. npj Digit. Med. 9, 201 (2026). https://doi.org/10.1038/s41746-026-02410-1
Wider trend: AI from Moorfields Eye Hospital
Teams from Moorfields Eye Hospital NHS Foundation Trust and the University College London Institute of Ophthalmology have developed an AI tool capable of predicting patients at risk of developing retinopathy, following use of a common autoimmune medication, said to be widely prescribed to treat rheumatoid arthritis, lupus, and other autoimmune conditions. Trained using more than 8,000 eye scans from 409 patients in the US and UK, the HCQuery algorithm works by analysing retinal images captured using optical coherence tomography, a standard part of screening for hydroxychloroquine patients. It correctly identified 100 percent of patients with retinopathy up to 2.74 years earlier than doctors, according to Moorfields, also achieving 91 percent accuracy in ruling out patients without the condition.
Moorfields spin-out Cascader has announced its partnership with Specsavers focused on harnessing the potential of AI innovation to improve patient care in optometry. Cascader, a spin-out from Moorfields, UCL and Topcon Health, is focused on building clinical-grade AI for ophthalmology. A mission statement from its website outlines its work to use AI “to enable safe, evidence-based decisions in high-volume, high-risk eye conditions” and to use oculomics for early detection of systemic disease.
HTN also recently caught up with Mahi Muqit, senior vitreoretinal consultant at Moorfields Eye Hospital and the Institute of Ophthalmology at UCL, to learn more about the European clinical trial of a new bionic eye implant involving 38 patients at 17 sites across five countries. The study tested the PRIMA device in patients with dry age-related macular degeneration (AMD), who had lost complete sight. Following activation, participants, some of whom reportedly could not see the vision chart at all prior to surgery, were able to read five lines on average.
