Next up in our special report series, we’re looking at all things AI.
Artificial intelligence is currently at the forefront of health tech news. Since the new year, we’ve seen a wealth of stories hit the headlines, from Google Research and DeepMind researchers publishing an evaluation of an AI application used to understand and generate language in a clinical context, to the AI Centre for Value Based Healthcare consortium of clinical, research and industry partners expanding its reach beyond the South East.
At the start of this month, we covered the news of nine projects securing funding through the Artificial Intelligence in Health and Care Awards, run by NHS AI Lab, the National Institute for Health and Care Research and the Accelerated Access Collaborative. The projects cover a variety of areas in which AI plays a role: monitoring images of patient cells to pick up indications of bowel cancer; using AI to improve diagnosis of lung artery blockages and high blood pressure; capturing disease features from electronic health records across a patient population; identifying biomarkers in screening to help deliver personalised care for patients with colorectal cancer; assisting with treatment of monitoring of neurological disorders; predicting the outcome of a kidney transplant; identifying pregnant women most at risk of premature births or complications; processing colonoscopy images; and analysing images of tissue extracts to help pathologists determine the presence of cancer. The funding, totalling nearly £16 million, is to support projects with early stage concepts and prototypes, supporting development evaluation, along with testing and further adoption across health and care.
Last week, Health Education England published their AI and digital capability framework, aiming to recognise and support the learning needs of healthcare teams and to allow individual learners to understand their own needs in this space. The framework covers six key areas: digital implementation; digital health for patients and the public; ethical, legal and regulatory considerations; human factors; health data management; and artificial intelligence. The framework shares capabilities based on each of the key areas and splits them into four levels, setting out a capability statement for each level and signposting towards relevant learning resources.
Yesterday we covered the news of NHS England launching an artificial intelligence and digital regulations service for health and social care, through which the National Institute for Health and Care Excellence, Medicines and Healthcare products Regulatory Agency, Health Research Authority, and Care Quality Commission will support adopters and developers with guidance. The service is designed to help innovators in navigating the regulatory landscape and brings together a central source of regulatory and best practice guidance related to AI in health and social care.
Looking further back, last summer we interviewed Dr Hatim Abdulhussein, GP and national clinical lead for AI and digital media workforce at the Directorate of Innovation, Digital and Transformation at Health Education England. Hatim discussed his interest in AI, HEE’s AI roadmap, adoption of AI in healthcare and more. Read the interview here.
Exploring AI research
Next, we’ll take a look at some recent research in the area.
How are PROMs being used to assess AI health technologies?
Earlier this month, a study was published in The Lancet: Digital Health entitled ‘The role of patient-reported outcome measures in trials of artificial intelligence health technologies: a systematic evaluation of ClinicalTrials.gov records (1997-2022)’.
“The extent to which patient-reported outcome measures (PROMs) are used in clinical trials for artificial intelligence (AI) technologies is unknown. In this systematic evaluation, we aim to establish how PROMs are being used to assess AI health technologies,” the study states.
The researchers highlight how inclusion of PROMs in AI clinical trials “offers the incorporation of patients’ perspectives as an important metric through which these technologies can be assessed.”
The website ClinicalTrials.gov was searched for interventional trials from its inception through to September 2022 and data was extracted regarding form, function and intended use of the AI technology. In addition, researchers explored where PROMS were used in these trials, and whether they were incorporated as an input or output in the AI model.
627 trials were included in the analysis, with technologies including AI-enabled smart devices, clinical decision support systems, and chatbots. Examining the data further, the researchers found that the most common clinical areas that AI health technologies were designed for were digestive system health for non-PROM trials and musculoskeletal health for PROM trials, followed by mental and behavioural health.
24 percent of the trials included one or more PROM, visual analogue scale, patient-reported experience measure or usability measure as a trial endpoint, and PROMs were most commonly used for AI technologies involved in mental health and long-term conditions, for which assessment of quality of life and symptom burden is particularly significant.
In terms of international scope, 34 percent of the trials in the study were conducted in the USA, 14 percent in China, six percent in France, five percent in Spain, five percent in Canada, and four percent in the UK. “The distribution by country of clinical trials of AI health technologies that used PROMs differs from the distribution of all clinical trials of AI health technologies,” the study shares.
“Notably, AI health technologies are sometimes met by a lack of acceptance or understanding from patients,” the researchers conclude. “In this systematic evaluation, we identified very few trials that used a patient-completed questionnaire that was specifically designed to assess the usability or acceptability of the AI health technology being trialled. These types of questionnaires could be used to identify and address patient concerns on the accessibility of AI health technologies intended for patient use.”
Evaluating accuracy of clinical decision support system
Elsewhere, a study entitled ‘Accuracy of a tool to prioritise patients awaiting elective surgery: an implementation report’ was published in BMJ Journals, highlighting the research team’s objective to evaluate the accuracy of a new elective surgery clinical decision support system.
For context, the researchers highlight how elective waiting lists for surgery in England are stratified based on a prioritisation system which relies on healthcare professionals to manually assign a priority code (P-code) and an associated timeframe to each patient on the list. However, with the COVID-19 pandemic widely disrupting services and the elective backlog growing, they note that the current method of prioritisation is procedure-specific, simplified to allow for rapid prioritisation. “It is not designed to manage the priority within a group of P-coded patients,” they state. “Yet, there will be those who deteriorate faster than others due to their pattern of comorbidities. There is a need to improve the accuracy of assessing patients listed for elective surgery and prioritise based on greater objectivity. As such, digital tools to improve this process have been proposed, such as the use of predictive algorithms and artificial intelligence.”
The researchers note that the Physiological and Operative Severity Score for the enUmeration of Mortality and morbidity (POSSUM) scoring system is “one of the most established and widely accepted constructs” in the field of predicting the risk of mortality and complications in surgery. The tool evaluated in this study, Patient Tacking List (PTL), combines POSSUM, the planned surgical procedure details and time on the waiting list, and applies these to a referential dataset in order to produce a matrix score. This score represents the difference in mortality rate and complication rate between the procedure being completed electively, in comparison to waiting for the patient to decompensate and present as an emergency; the higher the score, the greater the risk of a poor outcome without an elective procedure.
Through the study, researchers aimed to evaluate the accuracy of the PLT tool through receiver operating characteristic (ROC) analysis for mortality and complications in a retrospective cohort, using data across a 12 month period from three NHS trusts. The team constructed ROC curves for mortality and complications based on PLT risk predictions and compared them with actual patient outcomes to assess the tool’s accuracy. Analysing data from 11,837 patients, the researchers found that the tool was accurate with matrix scores correlating well to potential adverse outcomes.
“Given the increasing complexity in healthcare and larger number of patients on waiting lists, the tool has the potential to lighten the administrative burden and reduce costs related to service planning and delivery,” the team state. “To achieve maximum impact from such implementations, the importance of utility must be shared with all stakeholders including healthcare professionals, administrative staff and patients. This will result in the wider cultural change associated with digital transformation.”
AI-assisted diagnosis of ocular surface diseases
This study, entitled ‘Artificial intelligence-assisted diagnosis of ocular surface diseases‘, was published last month in Frontiers.
Whilst AI research related to ocular surface diseases (damage to the surface layers of the eye, namely the cornea and conjuctiva) has increased, the researchers say: “The main issue on ocular surface diseases is that the images involved are complex, with many modalities.” Therefore, the review aimed to summarise current AI research and technologies used to diagnose ocular surface diseases to identify mature AI models suitable for research in this area, and potential algorithms that may be used in the future.
A systematic literature search was performed in PubMed and Web of Science, using key words relevant to ocular surface diseases along with words relevant to AI.
The researchers found that AI studies have been conducted and algorithms developed for ocular surface diseases such as pterygium, keratoconus, infection keratitis, and dry eye. Suggested uses include AI being used as a diagnostic tool; to develop a grading system; to suggest treatment methods; for the allocation of medical resources; to predict recurrence; to analyse severity; to differentiate between different types of the same condition; and more.
“The timely emergence of AI has given rise to optimism in the field of ophthalmology, particularly in areas involving big data and image-based analysis,” the researchers note. “Using DL (deep learning) to process and analyse images of ocular surface diseases can significantly improve accuracy and efficiency, reduce manual analysis costs, and overcome errors between different experienced annotators. Currently, different AI models are used for AI applications for different ocular surface diseases. Among them, CNN (convolutional neural network) model accounts for the majority of the AI applications for pterygium, keratitis and dry eye, while RF (random forest algorithm) model has good accuracy in predicting healthy eyes and keratoconus in all stages in the AI application for keratoconus.”
The researchers comment on limitations, such as insufficient training and validation sets, with more image data training needed in order to improve accuracy, sensitivity and specificity. They add that image acquisition and diagnostic accuracies are affected by the varying inspection equipment used in different countries, regions and institutions. However, they conclude, “AI has great potential to improve the diagnostic efficiency of ocular surface diseases… the results reveal that although AI still faces certain challenges in model building, it can assist doctors with objective clinical decisions and lay the foundation for the accurate treatment of patients.”
