Next up in the HTN Interview Series, we explore bias with artificial intelligence and speak to David Newey, the Chief Information Officer at The Royal Marsden NHS Foundation Trust to hear about what bias is, the different types, and what the future of AI could look like…
We start by asking, what are the types of bias?
Within AI, there are five types of bias – implicit bias; sampling bias; temporal bias; over-fitting the training data; and edge cases and outliers. David explained what each are and what happens…
The first one is implicit bias, the unconscious prejudice that has been formed by the teams that are developing the artificial intelligence algorithms. So that is typically due to the fact that you may have a very homogeneous team developing an algorithm that has got a predominant demographic, and therefore they don’t necessarily have sight of other issues for large groups outside their own. In grouping, they may not take into account factors that impact on the design of the algorithm.
In regards to sampling bias, this happens when data is taken from a particular demographic group. That data is profiled on a particular demographic and used to be generalised for other demographics. For example, sex, race, ethnicity, sexuality, all those components. It’s about making sure the data used is correctly fitted.
The next one is temporal bias, this is my particular favourite! For example, the oldest bit of code that is still being used in production was a piece of software that was developed in the 1950s, and is still being used to this day!
Temporal bias is all about the fact that when these algorithms were developed, they were being developed with scientific knowledge of the time, of the demographic make-up of the time, and using the data of the time. The danger that poses is that things move on and new science is developed, or the training data is broadened, enhanced, and more complete – or society moves and the demographics change. If we are still continuing to use an algorithm that was developed with a snapshot in time, it may no longer be appropriate for current circumstances.
The next type of bias is over-fitting the training data. So again, this goes back to using the training data inappropriately and therefore not being able to predict new data accurately, and not being representative of the general population. It is all about not trying to use training data to generalise, if it’s not appropriate to do so.
That falls into the last type of bias, which is edge cases and outliers. That is where there will be a statistical probability in terms of the prediction models, but there will be some outliers to that. It’s about picking those particular outliers up and making sure that they are not at a disadvantage by the conclusions of artificial intelligence produces.
What are some of the areas the Royal Marsden is looking to utilise AI?
The types of artificial intelligence that are looked at can vary. The Royal Marsden do a lot of work around developing AI to look at diagnostics and analyse radiology images looking for abnormalities that may be quite small and hard to pick up, and then will therefore be able to triage the data.
Other applications include things like using and listening to chatbots. So, if a clinician is chatting with a patient, they actually pick up on what the patient is saying in the chat and start assisting clinical decision support.
Another area is interpreting data from wearables and coming to conclusions about the patient based on wearable data. Also population health – where to allocate funding, where is the area of most need, where should one direct their resources in order to get the biggest bang for their buck.
So, there is a whole series of applications for it, and that is just where we stand today. Aside from that, of course, there are also other applications – for example, going through unstructured data and trying to generate coded data for further analysis and insight. We’ve used that ourselves at the Marsden when stratifying lung cancer nodules (Hunter at al, 2021, Development of a Structured Query Language and Natural Language Processing Algorithm to Identify Lung Nodules in a Cancer Centre. Front. Med.).
What do think the future of AI could be?
When looking at the future of artificial intelligence David had a range of ideas from early technology adopters to governance structure to AI systems having a ‘license period’…
The danger with any new technology is that you get those earlier adopters who innovate and iterate very quickly, but before you know it, it actually ends up in production and can potentially become uncontrolled.
The important thing going forward is all around governance structure. One of the ways of addressing these biases, for example, is introducing a governance structure that looks at the design of the algorithm; the training data that has been used and its intended purpose to make sure that it is correct, and corresponds to best practice.
For example, the MHRA and the USFDA developed a good set of guidelines for best practice to develop artificial intelligence. So the governance structure, a bit like a research board, can look at the design and the development of an AI algorithm before what I like to call it, ‘licensing it for use’.
I personally believe that every AI algorithm should be given a shelf life or be given a license period, after which it should be taken out of use. Each AI algorithm, in my opinion, should be catalogued. It should have all its associated documentation which supported its license in the first place and then it should be subject to review before it can be continued to be used in an environment – just to make sure that temporal bias hasn’t set in and it’s no longer fit for purpose.
Is there anything you are working on at the moment that you would like to share?
In terms of personal work, David had lots to share about what he was personally working on in areas such as radiology, genomics, and digital pathology…
At the moment, the key one that we are looking at is imaging, and Dr Christina Messiou is working with colleagues at Imperial College in terms of developing those radiology algorithms. We are constantly looking at various algorithms particularly around genomics and digital pathology. We are expecting to use AI to look at the pathology slides and the cellular structures and highlight any abnormalities to improve diagnostic outcomes whilst assisting clinicians with managing the workloads to improve patient outcomes.
Thank you to David for chatting with us!
