We spoke with Max Gattlin, commercial director at X-on Health, about the use of AI technology in healthcare. We discussed the rapid emergence of AI, its value and whether patients are likely to be receptive to its implementation. We also explored current guidelines and the potential for future protocols.
The rapid emergence of AI
Max began by describing AI and technology as “an extension of human capability.” He elaborated: “If you think back to the days before telephones, it was impossible to easily contact people in other countries. The telephone allowed us to project our voice across the world. In the same way, Artificial Intelligence is an extension of our cognitive abilities and behaviour.” It’s for this reason that Max fully supports the rapid emergence of AI technology.
Beyond his support, Max emphasised the “need for it” within healthcare but recognised how the sector falls behind others regarding AI adoption because of the “sensitivity of healthcare data”. To address this, Max explained, “AI is a broad spectrum of technologies. We need to truly understand the use cases and ask ourselves, ‘How will it benefit everyone?’ and ‘Which element of AI can we apply to each use case to ensure it’s safe, scalable, effective and impactful?’”. By starting from a point of understanding the challenges before choosing from the variety of solutions and approaches available, Max believes AI can be key in solving industry-wide problems, such as smart patient navigation first to manage demand better, and progressing access to healthcare like never before.
Using X-on Health as an example, Max explained how they integrated an AI solution with an existing model about 18 months ago and since then, “there have been around 14 major model releases (and hundreds of different versions), but we’ve stayed consistent with the way we apply each one”. Although there are some rapidly emerging, smart models that may be faster or more accurate, Max highlighted that “many of these don’t allow you to control where your data is going or how it’s being processed”. At X-on Health, data visibility and control are prioritised and that’s why they’ve stuck with their original model.
AI technology and voice
Looking at the role of AI in voice technology, Max noted the prevalent discussions around dialects and how effective AI has been in that area, but that “voice AI includes many different technologies” such as speech-to-text, dynamic response and generative AI. Although these can do a lot of different things, Max emphasised that “in order to answer a question properly, the system needs to understand the intent and respond using accurate data”.
The surge in interest around voice AI at conferences and public forums, caused Max to voice his concern of “going from emergence to rapid, unchecked adoption”. With the exciting potential of voice AI tools like ambient scribes bringing clarity and consistency to care, Max emphasised that “not everyone needs an AI product but instead, a solution to their specific problem”. With some voice agents that are poorly configured for example, Max added that “they don’t actually solve a problem; they just shift work into another channel”. To mitigate this, Max firmly believes in the importance of “always having a human-in-the-loop” whether through training the model or monitoring outputs.
When considering AI solutions, he advised asking two questions: “What’s the problem, and what’s the best technology to solve it? Whether that includes AI depends on the use case.” For Max, the value of AI lies in applying it selectively and appropriately. He noted that a one-size-fits-all approach has often hindered the effective use of technology in primary care.
Patient acceptance of AI in healthcare
Turning to patients and how to work on AI acceptance, Max shared insights from X-on Health’s work with clinical safety and innovation consultancy Curistica over the past two years. “We wanted to ensure we’re building and deploying AI effectively and safely. One key recommendation was to make sure patients are comfortable using the technology.” Stressing the need for transparency, Max added that “patients need to know when a product uses AI, why it’s being used, how it’s being used and how their data impacts its performance”. To support this, X-on Health developed a user guide within their digital assistant that “notifies patients when a section will be using AI and asks whether they want to proceed. They can click yes, no, or learn more. Choosing ‘learn more’ provides a full explanation of what will happen.”
Max also highlighted a recent development to X-on Health’s centralised dashboard ‘Surgery Insights’ that’s enhancing this approach. “We’ve added a new Patient Intent module within Surgery Insights for Surgery Assist customers, which is already helping us better understand and train the AI on queries it hasn’t yet seen. It’s allowing us to work more closely with partner practices to analyse gaps and improve local models, so we’re not just responding to patients more accurately but also shaping future services around their needs.”
Exploring more general public concerns around data capture, Max pointed to the reaction to ChatGPT as an example: “When it first came out, there was a lot of talk about it taking over the world and misusing data. That simply isn’t true.”. Instead, he emphasised that “it’s all about how the AI is being used and applied” and the key is “ensuring patients understand the purpose of AI and how it can help them”.
Reflecting on previous deployments in primary care, Max noted how patients initially resisted changes like moving from face-to-face to phone appointments and later to online consultations. AI is no different: “We’re always facing resistance when deploying technology that could improve the system and the patient journey. Often, we find the resistance comes from what we see in the media but at X-on Health, we’re happy to go on this journey with patients. We’ve done it before and we’ll do it again, taking the time needed to help patients understand what it means for them.”
Concerns around AI and clinical safety
Diving deeper into clinical safety, Max identified several key areas to consider: “data governance, compliance and clinical safety”. Focusing on data, he emphasised the importance of UK-based servers for storage and shared how X-on Health has introduced protocols like auto-redaction: “Our AI runs within a secure UK-based Microsoft Azure environment. We don’t use patient data for model training. Safety, privacy and performance improvement are tightly controlled.”
More generally, Max acknowledged and appreciated the high standards AI suppliers to the NHS must meet: “Clinical safety and governance requirements are high, which is a good thing. It holds everyone to account and certainly makes me feel more comfortable!”. While more protocols are needed, he believes the sector is moving in the right direction.
Parameters and protocols that help protect data
When asked about future parameters to safeguard patient data, Max outlined that “It depends on the AI model so whether the supplier developed their own model or if the data is held locally on servers or on-site” which is something the NHS guidance and compliance guidelines answer.
Most importantly, he emphasised the need for robust foundations: “Data protection in healthcare AI must be built from the ground up. This means adopting privacy-by-design, avoiding the unnecessary collection of identifiable data and ensuring secure processing environments. When performance data is used, it should be anonymised, carefully governed and never used for training without strict safeguards.”
For X-on Health, protecting patient data is critical to building trust in healthcare AI. Max pointed to Surgery Assist, which follows privacy-by-design principles and does not collect identifiable information: “Any feedback is handled with strict safeguards,” he said. “We also have a privacy policy and an AI statement front and centre. Within our Surgery Assist product, we do not currently allow AI to make clinical decisions.”
He concluded by calling for industry-wide collaboration: “We need to work with online consultation platforms, triage systems, telecoms providers, website developers and, most importantly, the healthcare providers themselves. The goal is to make the patient journey as smooth as possible.”. With the anticipated UK AI Act expected to follow the EU AI Act, Max recognises X-on Health “will always ensure, as a supplier, to meet those requirements.”
We’d like to thank Max for taking the time to share his expert insight on this rapidly evolving topic.
