For a recent session focusing on AI in healthcare, HTN was joined by an expert panel including Simon Brown, head of digital at Royal Papworth Hospital NHS Foundation Trust; Wahida Jabarzai, clinical AI and automation delivery lead at University Hospitals of Northamptonshire (UHN) and University Hospitals of Leicester (UHL); and Julian Wiggins, healthcare solution director at Rackspace Technology.
Our panel considered the wider challenge of AI adoption, looking at what makes a successful deployment, introducing AI safely and sustainably at scale, and some of the use cases currently delivering value across their organisations.
Wahida talked to us about her role at UHL and UHN, managing clinical AI programmes including ambient voice technology, and working with colleagues on the use of AI for corporate functions. “We’ve got our own digital division, and we’re particularly interested in using digital solutions to tackle key issues we’re facing,” she said. “We’ve split that up into two verticals – one corporate and one clinical – and ran through an exercise asking our staff what their biggest pain points are, to help prioritise what would be most beneficial to our workforce, to patients, and to end users.” As well as AVT, the team is working on using AI to automate calling and booking, on clinical coding using machine learning and AI, and medical workforce using AI agents.
Offering an introduction to his role and to Rackspace Technology, Julian said: “We’re a hybrid cloud service provider that also has a large branch of AI capability we’ve built over the last five years; my role is focusing specifically on healthcare to make sure we understand what the needs are, and how the technology services we provide are able to deliver on those.” One of the key points is prioritisation, he continued, “as there’s a risk that we anticipate that AI is capable of doing things that extend beyond current capability, either in terms of data quality, infrastructure quality, or organisational readiness”.
“I’m trying to keep up with the pace of clinicians!” Simon told us. “They’re looking for the newest and best information, and at how they can get to the best possible outcomes for patients. It’s how you can manage that and how that’s embedded in what we do every day.”
AI deployments
Reflecting on lessons learned from AI deployments, Wahida emphasised the importance of safety. “With AI being relatively new and constantly evolving, we don’t always know much about the AI or its use cases, so one way we’ve tackled that is through pilots,” she noted. Running a small-scale pilot can help gather information on benefits or learnings, that can then be used to put together a business case or to go out for procurement, she continued, “and that’s been very successful with AVT”.
Some AI programmes like clinical coding are being pursued in-house, according to Wahida, “but when we’re looking at working with suppliers, it’s how we work with them to improve the product to suit the end-user”. With legacy NHS data “not always being the best”, having a larger sample dataset is key to improve data quality and sift out data that may not be useful for a particular model, she shared. “The last thing is that we tend to think about just deploying AI and then leaving it at that. That isn’t what AI in practice looks like, and it’s essential we have the resource and team on hand to monitor any changes, particularly since your dataset is also evolving at the same time.”
Patient safety is always first and foremost, with clinical safety officers involved in everything that happens throughout an AI deployment, Simon told us. Having stakeholders engaged and the right subject matter experts involved is also key. “We do run pilot projects, and that’s one of the great things about being at Royal Papworth with our research and development team,” he explained. “There was some work done a couple of years ago with regards to cardiovascular and using a stethoscope and an app – that was with 1,200 patients, with the aim of identifying changes before they become irreversible in terms of heart failure or heart conditions. The challenge is that things like that just keep evolving, and you can’t get ahead of it, so it’s always an ongoing learning.”
More needs to be done from an NHS perspective on working with commercial partners or those outside of the NHS world, Simon considered, to benefit from wider learnings. “We’ve got greater governance internally now with things like our reporting, and as an organisation I think that’s helping us become more transparent, promoting that visibility of what is going on so it can be discussed at the right levels. You can have all the frameworks you like, but at the end of the day it’s human behaviour and the practice of what people do that is most important.”
“The curse of going last is that everybody’s already taken all the good bits!” Julian said. “But I would echo what’s been said about governance, safety, and security. The use of pilots is fantastic, because you can start quickly and get moving. One lesson we learned was that when you’re building a pilot, you should be mindful of the need to scale, because if fixes need to be made to a very expensive component or model, the impact on the trial is very limited because of its small scale – you need to think about how that will work and the resources it will consume once running at scale.”
Traditionally in technical deployments, you design, build, and test something, before it goes on to run happily by itself, acknowledged Julian. With AI models, however, the data they consume, the infrastructure they have, and the availability of new ways of doing things mean an effectively moving target. “Building a target operating model for your AI infrastructure is important early in your AI journey, so that as solutions proliferate and new tech becomes available, it becomes easier to manage the challenge,” he suggested. “Establishing robust frameworks, governance, and architecture will give you a very challenging project, but a viable one versus the one you’ll have if things grow too quickly. And with your clinicians, if you can’t provide something quite quickly, you run the risk of shadow AI emerging along with the risks that poses for data protection.”
Measuring success
Simon shared some of the metrics and measurements in place at Royal Papworth to observe how something is performing. “For our deployment of clinical voice dictation to speed up the process of getting letters and information out to patients, we measured around 1,169 appointment notifications being sent out faster, looking at performance against the previous month,” he said. “We broke down our understanding of that further by examining what device people were using, and then by specialty, and also by clinician. We shared this information on a Power BI stack for clinical teams to discuss, and a learning we had here was that things like that can be better received when coming down from other clinical staff, rather than tech and digital people.”
For the UHL and UHN AVT programme, Wahida talked about sending clinicians both a baseline survey and a post-pilot survey to understand how much time was saved in clinic, and whether it made life easier. “Some of the learnings that came out of that included that one template will not fit every specialty in terms of the content required,” she reflected. “Separately, we’re also looking into product specifics like AI hallucinations, which we wanted to address early on, and we measured edit rates, because if a lot of time is being spent on editing, that would tend to indicate that that sort of thing might be an issue.” Another measure was user drop-off during the pilots, to understand why that was happening, and using forums like Teams and clinician feedback sessions to collect information, she added.
“Another thing we do is to measure numbers we’re expecting to send versus how many have actually saved in our systems, just to make sure from a governance and clinical safety perspective that’s aligned,” Wahida continued. “We can then use that in our benefits case, to say what kind of savings we’ve been able to realise through the use of AVT. That’s just a quick overview, there are obviously many other detailed metrics in the Power BI dashboard that we are being provided with by the supplier.”
Keeping track of metrics and measuring benefits is key, because ultimately you need to be able to demonstrate those benefits to stakeholders to get them on board, Julian shared. “Having a good feedback mechanism in place is important, because quite often you may think you achieved one thing, but actually there can be adjacent benefits to these things, too,” he considered. “And if the benefits you were expecting when you set out haven’t actually been delivered, it’s also valuable to know that before you scale it.”
NHS AI adoption
Looking at what is needed to make AI adoption in the NHS successful, Simon highlighted the challenges that can arise from different AI policies between trusts or collaborating organisations. “Ideally, you would have one policy and approach across the whole of the NHS,” he suggested, “but each organisation has its own, and there are different ways people work – we lock down really tight for that reason, and we don’t allow people to do things like download software onto a device, because you don’t know how secure that device is.”
Wahida mentioned leadership, noting being “lucky” to have good leadership at UHL and UHN, “where there is a bit of risk-taking and really just going for it”. Change management and changing people’s behaviour is one of the biggest pain points, and clinicians are so busy that building excitement around something is important, she explained, getting clinical champions in place early on. “Another thing we have noticed is compliance, your cyber security standards, DPIAs, clinical safety, and so on – getting that done early helps prevent it from becoming a blocker later on. One thing I will say is that with AI, that’s not going to be something that can be done once and forgotten about; it’s a continuous process that’s going to need to be updated as things evolve.”
It can be intimidating to start something with AI when you haven’t seen anyone else doing it, Wahida told us, making sharing learnings across the NHS extremely important. “Having frontrunners and those who have already started their journey share their documentation, or talk about some of the challenges and how they overcame them, is key to ensuring the wider NHS landscape can benefit from that as well, not only your own organisation.” UHL and UHN has been “very open” with sharing its compliance documentation so other trusts don’t have to go through the same process, she went on, “and it’s pivotal that we work together as a collaborative, rather than in silo”.
“My overriding sentiment is to keep your eyes on the prize here,” shared Julian. “That prize is not a successful pilot; it’s a successful production system. It’s building your solutions knowing the compliance they’re going to be subject to, the costs they will incur, and making sure they are as flexible as possible – think open standards and commonality, which will help keep complexity of digital systems as manageable as possible.” He also recommended having “a very high level” of documentation in place, talking with suppliers about their compliance and what they’re doing with data, and doing some “tyre kicking” to check a solution or product fits with the needs identified.
Maintaining safety and sustainability in AI at scale
Having governance is important, said Simon, but it ultimately comes back to the people. “We need to keep opening up those discussions, not only at executive level, but also at operational and clinical level, talking about benefits and learnings, and also where something hasn’t quite worked. If we’re fearful about sharing what hasn’t worked, we’re never going to make it sustainable or better.” While implementing AI means asking people to follow a strict process or guidelines, there will also be instances whereby they have to use their own judgment and experience to make an informed decision or to decide what information to follow, he considered. “You have to put your trust in those people. That’s the best safety, just having those professional standards and that human care that people have to do the informed and the right thing.”
Managing safety can be a challenge due to the complexity of the domain, Julian noted, and driving the cultural aspect of transformation is a crucial step. “The NHS is full of highly motivated people with a fantastic purpose,” he said. “Give them the training to understand what they’re working with, and create a culture where people aren’t afraid to indicate when they have done something wrong, and where they understand the dangers – this is a new frontier for a lot of us. The sustainability piece is challenging, because from time-to-time people do have concerns about technology taking over their jobs, but it’s important they see the benefits these tools can bring them.” The opportunity for the NHS is in collaboration, he continued. “That’s an enormous opportunity for us to come together and share our learning, taking advantage of the collective intellectual power of all these organisations across the country. If we can find a way to bring that together and build communities of interest, there’s some value in that for all of us.”
Wahida again stressed the importance of pilots in allowing things to move forward safely, providing the protection to explore solutions in a controlled environment before scaling up to the size of a whole service or trust, which might span thousands of clinicians. “You can’t just go for a big bang approach across a clinical setting when you haven’t really evaluated all the risks involved – do it on a small scale first, get your learnings out, and then gently look to introduce that to different areas.” UHL and UHN have an AI safety board which is helpful as a front door process when presenting new projects or programmes, she went on, and that includes clinical safety officers as well as AI experts, and oversight from key SMEs who offer different insight when considering risk. A specific board for AI programmes also offers oversight and clinical input, along with operational perspectives from those managing things on the ground.
Key challenges and key takeaways
A key challenge around AI more generally is the enormous amount of computing resource it consumes, Julian discussed. “If I could fix one thing within this domain, it would be to make AI more efficient, because the value it can help us to bring is becoming clearer, and to make that sustainable and cost effective would be a real win-win.” The key goal for Rackspace this year is to help customers safely scale with AI, he explained. “The potential is there; now it’s how we build safely, how we afford the infrastructure, and so on. I think the way to achieve that is to have more conversations like this, and my goal is to spend more time discussing requirements with our customers to see how we can collaborate to get the most benefits possible.”
“One of the key areas that our shortlisting exercise highlighted was around using AI for translation in consultations,” Wahida told us. “Another thing was AI calling and booking, which removes the admin burden associated with that process. And we’re also working on clinical coding – we’ve got so much data already available, making it a good use case for AI.” UHL and UHN will be working on their shortlist of AI programmes over the coming 12 months, she continued, looking to get out as much as possible in terms of learnings, expanding collaborations, and looking at things like monitoring, reporting, and the BAU process.
“For me, it’s data and information sharing,” said Simon. “You’ve got all of this fragmented AI and siloed work happening – I’d like the national team to find a way for us to bring that data together to inform our adoption.” Focuses for the next year at Royal Papworth include tighter governance, he acknowledged, and other projects like a new EPR and upgrades to infrastructure and architecture will likely mean greater AI use. “The future is going to be governance frameworks, and getting our foundations right.”
We’d like to thank our panel for taking the time to share these insights with us.
