Last week NHSX published a report on Artificial Intelligence in healthcare. The report highlights where the organisation sees practical examples for AI and what the organisation is doing to build ethics and transparency into the use of AI.
To view the report please click here.
We asked the HTN community for their thoughts on this report and in this article we have curated a series of their comments. If you would like to comment, please join the discussion and share your thoughts on LinkedIn here.
Dilshan Arawwawala, Chief Clinical Information Officer, Mid and South Essex Group
There are lots positive statements which are welcome. The need to support AI as part of the longer-term plan to deliver safe, effective and efficient care. The correct identification of AI as a support to clinical excellence and the clear need for ethical and operational “end to end” impact assessments. There is also recognition of the challenges of keeping regulatory pace with the “fast AI train”.
The document does raise many questions and challenges which I hope will be addressed. The AI landscape is vast, prioritising clinical and public health domains is a sensible approach. How were these domains chosen and where will the responsibility lie for regulating “out of scope domains” lie? Products and clinical environments evolve over time which will have an impact on accuracy and reliability. Relying on vendors for longitudinal analysis and self-reporting introduces potential conflicts of interest. Getting the balance between driving innovation and ensuring clinical safety will be challenging.
AI requires accurate and complete datasets. What is in place to ensure enough high-quality data is available for solutions to be rigorously tested prior to release into heterogenous patient and clinical environments?
The NHS desperately needs more data analysts. Many private sector posts in this field remain unfilled due to a pan-European skills shortage. How are we going to compete to recruit and retain this much needed staff group?
Locally, as part of our regional digital and strategic roadmaps we are looking at how “AI” can help deliver better pathways for our staff and patients. Areas include radiology and capacity planning. We are currently reviewing and adapting our governance framework to ensure robust processes are in place for our ever-evolving health information infrastructure.
Overall, this document is a big positive step forward in striking a balance between supporting innovation and keeping our staff and citizens safe. The devil will lie in the detail around roles, responsibilities for initial evaluation, procurement and longer-term regulation and management of these tools.
Graham Walsh, Consultant Knee Surgeon and Chief Clinical Information Officer, Calderdale and Huddersfield NHS Foundation Trust
As a CCIO, but also an orthopaedic knee surgeon I can see real future benefits in how I manage my own patients in the future using AI. It can help improve our patients outcomes and satisfaction, we will be able to better predict which patients will do well from which surgery and will be able to give individual patients the bespoke treatments based on outcome models. With AI will come a new wave of learning robotic technology which is just around the corner.
NHSx commitment to AI adds a real excitement to the NHS’s digital journey and points to a serious commitment into our digital futures. The NHS is fast becoming the world leader in healthcare digital disruption and our trusts need to embrace this to help us deliver the best care to our patients and cement the NHS as the best healthcare provider in the world.
Adrian Smith, Head of Digital Transformation, NHS Arden & GEM CSU
This report is exceptionally timely, because hype across the NHS around AI has made decision-making difficult for many people. There is no doubt AI has huge potential, but it is also easy to over inflate expectations and to become confused about what is actually possible now, and what could be achievable in the future. This report helps people understand where AI is already helping, how and why, and decide where else is could be applied.
At NHS Arden & GEM CSU, we are already using AI in powerful ways in an area that many people, including care professionals and patients, have come to accept – the analysis of large databases of diagnostic imagery. It is now widely accepted that computers can analyse hundreds of thousands of images faster and more accurately than a human operator and that a machine can ‘learn’ from doing so. The more images an AI system processes, the better it becomes at recognising differences between healthy and diseased. The domain in which we are currently applying this is in bowel cancer where we believe AI has the potential to help eradicate this curable cancer.
We are also interested in how we can apply AI across care pathways, helping to select patient cohorts and manage treatments to personalise healthcare. This is an emerging and exciting use of AI and one we are keen to work with partners to explore.
Michael Abtar, CEO and Andrew Capey, Senior Consultant, IG Smart Ltd
In order for AI to be widely used in healthcare, in a lawful manner, organisations will need to place great emphasis on transparency. Ultimately, providing patients with enough information to understand what the actual, or potential risk implications of having their data processed, and/or care delivered using AI technologies may be. Explaining how complex technologies and data processing activities work, is something that the NHS has failed to do successfully in the past – a prime example being with Care.Data. So, we feel the UK healthcare sector is quite a way away from being able to successfully (and lawfully) deploying AI technologies in a meaningful way, en masse.
One major risk with AI, is that even if the algorithm is sound, if the data that goes in is of poor quality, or is in anyway biased, then no matter how good the logic behind the AI is, the results will always be skewed. For healthcare, this risk could easily lead to individuals or groups of individuals being discriminated against, or in the worst-case scenario, patients losing their lives.
Whatever one does with AI (or any technology for that matter) in healthcare, to be safe, there always needs to be an element of human oversight and opportunity for intervention. The healthcare industry can learn lessons from the recent Boeing 737 MAX tragedy, where a faulty altitude sensor/control kept pushing the nose of the aeroplane down, and would not allow the pilot to override it.
We have witnessed numerous incidents in the past where patient’s lives could have been put a risk, because of faults and glitches in technologies, which have thankfully been prevented because of people whose intelligence is not artificial!
Nicola Hall, Chief Operating Officer, Ingenica Solutions
We see the use of AI in healthcare as incredibly exciting in aiding the management of resources across the healthcare back office, we can see huge potential in helping the staff across the board manage better. Whether that is scheduling operations, ensuring medical consumables and medicines are available, managing demand patterns, bed management, equipment management and whole areas that have not been thought of as yet. However it is an area of development that needs careful governance, and as this technology blooms so should the controls around the software development.
Aiding clinical decision making means that software development steps into the arena of medical devices and then the companies developing the solutions need to be managed by current regulations for medical devices manufacture. We wonder whether those standards are developed enough themselves to manage an explosion in this use of technology.
Sara Nelson, Programme Director, DigitalHealth.London Accelerator
We welcome this report in consolidating the current thinking on AI. It is a significant step in supporting NHS staff with the information needed when assessing AI opportunities, and for the innovative companies to refer to regarding governance, data access and prevention.
It is great to see live examples of machine learning in the report, including one of our Accelerator innovators Lifelight; an exciting innovation that supports observations in acute and community sectors.
AI has a huge potential to benefit the NHS and its staff and improve patients’ experiences and health outcomes. I hope the publication of this report and the on-going debate around AI will enable AI to deliver its full potential safely.
Martin Bell, Independent Consultant, The Martin Bell Partnership
The paper on AI from NHSX is exactly the start that is needed around the discussion of AI, the NHS and it’s place in healthcare delivery and research.
It is to be welcomed, and AI should be welcomed into healthcare. The use of AI to support frontline clinicians, support research, process insights across massive amounts of data and to support finding new cures and to augment and transform how some healthcare services are delivered, are potentially limitless over the coming decades.
The commercial opportunities for companies in this sphere are immense, and whilst this can be a great economic driver for good, we must ensure the population as a whole benefits (no equivalent of 4G “not spots”, as with mobile phone coverage or poor bandwidth) – “AI for All” perhaps!
Equally we must recognise we are at the early stages of this technology. It is not a panacea that will cure all ills, either with health or with technology. We must not forgo the basics of digital health, for the “shiny new thing” – it must be additional investment, not instead of.
We must ensure we don’t build bias into AI when dealing with whole populations (as has been evidenced from some AI projects around the globe), we must have proper regulation and we must ensure transparency – no secrets. We must also ensure that the patient remains in control.
Two phrases jumped out at me:
“Using Data Driven Tools to compliment the expert judgement of frontline staff” – compliment, not replace. With a 15 million global shortfall in doctors and nurses alone projected by WHO for 2030, we must adopt technology to keep delivering healthcare services.
“the science of making machines do things that would require intelligence if done by people” – all industries that have modernised have used technology to do so. But we equally must avoid the “techno-hell” of “just wanting to speak to someone” from the patient’s perspective, especially in an area of society that is all about people.
Both of these will require education to shift, with technology and information, including AI, getting embedded into clinical training for all clinicians, and a wider debate with the public to ensure they grow comfortable with new ways of delivery. We all have experiences of online commercial services in our day to day lives that make things easier, but also sometimes make things harder, as they have used it just to cut costs.
Nothing wrong with saving money, but not at the cost of the quality of care.
Mat Oram, CEO and Co-Founder, AdviseInc
I think NHSX showing such a strong intent in the field of AI is hugely positive. Tech has the ability to transform healthcare, but it also has the ability to waste a lot of time / money. It’s very immature at the moment, as a market, and you need to encourage an eco-system to make best use. The risk from NHSX’s point of view will be the trade off between strong governance / ethics and the need to move at speed. The fear being the NHS inadvertently slow down the development of solutions and the market leaves them behind.
The impact on people is also important. The alogirthims are ahead of the humans in that systems can do many things, but a workforce that’s not there digitally can’t make the most of it, so the up skilling of staff to make best use is critical.
Tom Blake, Software Developer / AI Lead, DrDoctor
The NHSX report highlights multiple areas for us that we consider important when developing these technologies. We must consider:
- Protecting the patient – is their data secure at all times? Can the data being used identify a patient in any way?
- Using the right data to avoid bias in our technology – are all patients being treated equally?
- Contributing to the wider AI in healthcare community – using open source software and standards that are transparent and can be adopted and improved on by others
Kevin Ross, research director for Orion Health
Orion Health has made a significant investment in AI with its Precision Driven Health initiative. This is a pioneering partnership between New Zealand health organisations, academia and industry that is applying machine learning to large data sets in order to improve health outcomes by delivering a more personalised experience for patients.
The partnership has created a website with 23 tools and calculators that are built on New Zealand data: one of the best-known is nzRISK, which is a simple calculator for patients undergoing non-cardiac surgery. As a result of this experience, Orion Health has been able to create a suite of data-handling and machine learning tools for international customers: we had a great reception for these at our UK and Ireland Customer Conference in Manchester last week.
We have also learned some valuable lessons about data science in healthcare. The NHSX report talks about the value of data, and one thing we have learned is that it is not collecting data that is difficult, but collecting good data and understanding its context. nzRISK is a good example.
New Zealand hospitals were using a risk calculator that was based on a British study of a small population, and it was not attuned to the New Zealand population, particularly the Maori population. Part of our job was to dig down into the data to understand the impact that had on our population.
Another thing we have learned is that it is very important to think carefully about how much information you need to give people about your model, and how to explain its results. The NHSX report discusses this point in detail and focuses on what the future governance of AI should look like to encourage its adoption and spread.
I think that focus is completely right. The potential of AI in healthcare is almost limitless, but we need to maintain the confidence of clinicians and the public and to take people with us.
Dr Anjum Ahmed, Global Director, Imaging Information Technology Solutions, Agfa HealthCare
It is encouraging to see NHSX championing the opportunities that AI can provide for the NHS whilst also recognising the complexities around uptake in an NHS infrastructure that has multiple systems and silos that don’t always talk to each other.
We have focused on addressing the challenges of clinical community around seamless integration of multiple algorithms on our modern Enterprise Imaging platform. Aggregation of patients’ medical records data and automated correlation with medical imaging intelligence, automation of tasks, workflow optimisation, reports automation, AI enabled peer review and learning will help diagnosticians improve evidence-based collaboration and better care delivery.
Antoine Lever, Babblevoice
We read the AI in Healthcare report with great interest and were encouraged by the broad scope and forward-thinking tone. It is clear (and right) that the most attractive advantages of AI in healthcare are in support of clinical decision making (diagnosis, treatment plans, medications, etc) however it is also clear that this is where the greatest risks and concerns are to be found.
I fear that we are far from the point of AI making clinical decisions but there are other areas where the technology can be used. As a telephony provider we have a lot of call detail records and AI, at its core, is just statistics. It is likely that a number of insights, cost-savings and efficiencies could be gleaned by applying various AI techniques to the vast number of records stored by telephony companies like babblevoice.
David Markwick, Head of Information, Southern Health NHS Foundation Trust
We are always looking for new and innovative ways to help us continue to deliver the best care to our patients and recognise that artificial intelligence will play an important role in helping us to do this. The Trust is working with AIMES, a research and innovation company, to develop a series of artificial intelligence algorithms that can use the rich source of information within a mental health service users electronic patient record, to highlight when an individual’s health may be deteriorating; possibly to the extent of that person nearing an episode of crisis.
AIMES works in partnership with the University of Liverpool and Kings College London to develop the machine learning and natural language process technologies that underpin this exciting project. If successful, the tool developed is intended to support clinicians in the intelligent scheduling of appointments, prioritising those individuals that are highlighted as being at greater risk. The project is expected to be completed within the next 12 months.
Dr Mark Harmon, Strategy and Brand Director, eConsult
According to Deloitte’s 2018 Human Capital Trends study, 42% of companies believe that automation and machine learning will be widely deployed at their organisation within three to five years. Yet the recent state of AI in healthcare report, published by NHSX, reveals that the burgeoning sector of AI, once the reserve of finance or technology sectors, has the potential to transform the NHS too.
When we think of AI, we might first think of smartphone voices, Google ‘assistant’ tools or driverless cars. Though we may be familiar with the convenience of using Alexa and Siri, identifying how this AI tech can be safely introduced in healthcare without posing risks to patients or causing severe disruption might seem like a leap.
One of the key concerns with the emergence of AI in healthcare is that many believe it removes agency from clinicians, receptionists and in turn, patients. Yet a crucial point to consider is that no matter what, AI still requires human interaction to operate.
In the long run, AI can help to simplify processes for professionals, by reducing the need to manually input information, or by building better predictive models to estimate future needs without the risk of human error.
Elliot Engers, Cho and Founder, Infinity Health
The report by NHSx provides a welcome snapshot of the complex challenges and significant opportunities associated with the introduction of AI systems in healthcare. In particular, we agree with the emphasis given to evidence generation. It is clear that AI has the potential to dramatically improve the quality and efficiency of care provided by the NHS. However, quantifying the true and ongoing impact of AI is going to be essential in building the necessary trust and transparency that is required for widespread adoption in the NHS.
Kathy Scott, Director of Operations at Yorkshire & Humber Academic Health Science Network
We welcome the publication of this report. AI has huge potential to change the way people are diagnosed and treated but this report doesn’t gloss over some of the issues that need to be resolved. At Yorkshire & Humber AHSN we’ll continue to work closely with the AHSN Network on this important area to benefit patients locally and nationally.
Richard Strong, VP and MD of EMEA, Allscripts
Allscripts welcomes the global lead NHSX is taking in the field of Artificial Intelligence. AI type systems have been on the periphery of clinical healthcare delivery for several years, however they are now entering the sphere of mainstream clinical care. As these systems become mainstream, it will become increasingly important that they way they are developed, implemented and utilised is managed in a structured fashion.
Allscripts applauds the investment NHSX is making in setting up an AI lab and is particularly pleased at the decision to put focus on defining a Governance Framework, on clarifying data governance elements of AI and on developing global best practice guidance. The UK has been at the forefront of scientific and health research, and this paper will help to cement the UK’s place at the forefront of AI in healthcare.
Peter Gribnbergs, CMO and Co-Founder, EQL
AI undoubtedly has the potential to revolutionise healthcare. The ability to provide meaningful insight on vast data sets whilst simultaneously providing user experiences that are highly personable is incredibly exciting.
This report is a welcome addition to the growing amount of exceptional information being produced by NHSX. It goes along way to answering many of the questions faced by AI providers. The addition of case studies is particularly useful as you can see real world examples of how other innovators have interpreted the policies and practices that can at times be overwhelming.
To have an understanding of the challenges facing the NHS when it comes to implementing AIS is invaluable as too often you can become obsessed with addressing your own unique problem and forget that there are other drivers that can impact on decision. Understanding the motivations behind these decisions helps us to create meaningful strategies that we can bring to the table.
Gary Hutson, Draper & Dash
This report signifies a move in the right direction for NHS and predictive analytics.
The opportunities presented in the report show that the wider role of AI is being considered more seriously, especially in terms of image recognition and algorithms to automatically process scans via methods such as convolutional neural networks. However, there are many more use cases of where ML can be applied to hospital datasets.
It’s encouraging that the report and AI group want to separate out who is using AI vs who is over hyping the potential of their solutions. This will allow a true understanding of how far AI is spread across the healthcare sphere.
The governance and a code of ethics creation for algorithm creation is a good start, as getting the right structures in place will allow for a more sensible and governed approach to the embedding of AI in the NHS.
The report tackles what ‘good’ innovation looks like and how AI (mainly ML) can be used to tackle four key domains: precision medicine, genomic (deep learning models to be used to detect genomic sequences, for example), image recognition and operational efficiency (with the focus being around how free text information captured in systems can be processed in a more advanced method using Natural Language Processing and other methods).
For me, what needs to be considered alongside the rise of AI in healthcare, is also the technology to support these advances. Meaning more funding required to shape the future of AI and not be limited by.
