Our latest interview sees us sit down for a chat with Dr Paul Deffley (chief medical officer) and Tom Scott (chief commercial officer for UK operations) from Alcidion on the topic of benefits realisation.
Paul has been an NHS doctor since 2003 and now focuses on his work with Alcidion, where key parts of his role include clinical safety, making sure that product strategy represents real-life challenges, and benefits realisation.
Tom, meanwhile, focuses on the growth of UK operations and sits on the senior leadership team at Alcidion alongside Paul. He brings 12 years of experience in health tech roles, centring around the delivery of solutions into the secondary care environment.
Why do benefits matter?
“In digital healthcare, there are lots of different ways of judging solutions,” Paul said. “You can look at specifications, technical capabilities, functionality. The reason that we put any of these digital changes in place is to make a tangible difference. It’s not just making a change for the sake of it; there must be a benefit to the community that we serve.”
Unless a solution truly moves to that tangible end point whereby the patients or staff can experience the benefit, Paul emphasised, then the technical solution has not really delivered on its purpose.
“There are different levels of focus and priority, and I think people often talk about benefits – but I don’t believe that realisation of those benefits always receives the focus that it should.”
Tom said that in his view the industry tends to over-focus on technology and purchasing solutions. “Change management and focusing on people outside of technology – that’s where the marginal gains are made, and that’s where the benefits come from. A benefits realisation framework allows us to expand that focus away from buying a technology solution, which is an enabler to change.”
Do providers measure the impact of technology deployments?
“I don’t think we consistently see measurement of the impact that we deliver,” Tom said, adding that there can be a tendency to focus on benefits at the beginning of a process and not to come back to them.
Whilst procurement often puts some thought towards benefits in the first instance, benefit realisation “should be in partnership with suppliers to understand what technology enables, what good looks like, and what experience your supplier can bring.” In particular, he noted, suppliers should be able to support customers with baselining change and with measuring it.
Emphasising the importance of baselining, Tom continued: “Coming back to benefits plans and realisations that are baselined with a supplier in partnership can be really powerful. It allows you to understand whether you were foreseeing the right benefits at the start, or whether your frame of reference has changed slightly. Your supplier can share expertise and lessons from other organisations.
“Crucially, underlying all of that, you need to have a baseline measure of something, whether it’s productivity, output, engagement – whatever it is you choose to focus on. You have got to have something to measure against. Once you have that baseline, you can iteratively look at how you are performing against it, whether you measured the right things, and where else you might be able to derive benefits. These are not one-time implementations. Being able to deliver benefits and then adapt the benefits case as we move forward means that a new baseline is constantly being created to drive improvement.”
On whether we always see measurement of the impact delivered, Paul said that in his view the answer is “an overwhelming no”.
He said: “I think we measure what we can measure, not what we’d like to measure. Often benefit realisation will be focused on activity-type metrics, it will be very transactional.” He highlighted an example of this from regional care records, where one of the metrics often reviewed is the number of times somebody has logged into the record. “That is viewed as an activity measure to show success. When it was investigated in more detail, every time you logged into the EPR it automatically logged you into the regional care record and that was counting as a measure. The more useful and reflective measure would be where a decision has been impacted by the information seen in the record, but that’s much harder to measure.”
The challenges of capturing benefits
“It takes time to measure benefits and to see change,” Paul acknowledged. “Often we see initial honeymoon benefits, however it’s not really until you get through to the maturing phase of the deployment that you start to unlock the long-term benefits that make a difference. One of the challenges we see with procurement cycles is a desire to release in-year benefits. In healthcare, the complexity of change management means that it is often multiple years before those benefits are realised.”
Paul agreed that another challenge lies in following through on benefits realisation beyond the business case approval. He also highlighted several other challenges, including difficulties with finding the right expertise and skills to measure and articulate the delivered benefits; that “the system is not sympathetic to things that take time”; and the complexity of measuring benefits through optimisation cycle.
“Critically, it comes back to needing to measure what matters,” he said. “We need to look at patient impact, staff workload and not just focus on what we can measure because it’s easy.”
Tom reminded us that some benefits can be intangible and can surface in survey responses or through patient engagement.
Benefits realisation in action
Considering examples of where benefits realisation has been done well, Paul highlighted a deployment of Alcidion’s noting solution at The James Cook University Hospital in Middlesbrough. The team understood the importance of baselining, he explained, and “really invested a significant amount of time to engage with the workforce, who were using a pen and paper solution. This was a deliberate effort to really understand their experiences and the barriers they were facing in day-to-day work.” That effort gave the team a solid understanding of the problems that they were trying to solve and the elements that really mattered to the clinical workforce. “When we deployed the solution, we could see really significant improvements in things like time reduction associated with administrative burden.”
It is not all about the metrics, Paul added, describing how staff shared qualitative statements on day one of go-live saying that it was the first time that they had been able to go home on time. “Sometimes it’s about the person, the story, the experience and how it makes staff feel. Ultimately, looking closely at the intangible benefits helped us all to understand and communicate the difference and impact that the digital solution is having.”
Paul also raised an example around deployment of Alcidion’s flow solution at Alfred Health in Melbourne. The team there commissioned an independent evaluation by an academic institution in the region, aiming to deliver an independent view of before, during and after the solution deployment. “Sometimes when organisations do internal evaluations about benefits, it’s a bit like marking your own homework,” Paul pointed out. “It’s very difficult for a team, having just spent the last months deploying something, to step back and credibly evaluate their work.”
The research undertaken at Alfred Health found more than 12 percent reduction in length-of-stay during the period of the solution’s deployment as well as “some really significant improvements around patient flow and the release of capacity to avoid access block”.
Tom added that this project also led to a reduction of 18 percent in outliers. Outliers create compound inefficiencies, impacting staff who have to cover greater distances to see the patient, and consume time coordinating the logistical efforts to transfer the patient back when a bed does become available. Patients who are outliers typically have twice the length of stay. “Statistics like this aren’t just paying lip service to benefits – it’s the reality, showing a solution works and that there is a return on investment. Additionally. that insight is really powerful for other organisations looking to deploy and needing to build their own investment cases as well.”
Paul and Tom agreed that a key element of getting benefits realisation ‘right’ is culture, particularly the culture of wanting to measure and improve and iterate, and emphasised the importance staff feeling they can speak openly. “There can be a culture where people don’t feel able to talk about a digital programme that’s gone on for two years without a tangible benefit,” Paul noted. “The culture needs to be open to these conversations, to be data-driven and looking to continually improve.”
The other side of the story
To offer a considered perspective, Paul went on to share his views on the other side of the matter: what might be missed if organisations focused purely on short term benefits realisation?
“The other side of the benefits works is that it has to be able to fulfil a certain brief and solve a problem,” he said. “It can’t all be about benefits here and now.”
He highlighted an example around futureproofing, with regards to native FHIR capabilities, which facilitates integration across various parts of data in the health system. “There’s no tangible benefit you can measure from having those capabilities in the near term,” he said, “but if you went into a 10-year contract purely focusing on benefits and you didn’t partner with an organisation that has FHIR capabilities, you’d miss out on what it can offer as technology evolves in the longer term. You might not be able to future-proof your deployment and innovate as time progresses.”
Whilst benefits are “under-estimated and don’t receive enough focus”, Paul acknowledged, there does need to be a solid understanding of the situation-at-large beyond benefits. “Essentially, I am saying that benefits need more attention; but they should not be the sole factor to judge a solution on.”
How procurement can support benefit realisation
“Being really open about the baseline and the current state is really important,” Tom said. “Even if the current state isn’t very well developed, it’s useful to know.”
“It takes time,” stated Paul. “I’ve previously worked as a commissioner and I’ve been part of procurement teams on really large-scale digital solutions, and it is challenging for commissioning infrastructure to allow the time it takes to demonstrate the benefits.” From a procurement perspective, Paul said he would urge organisations to explore the academic evidence base that highlights how long it takes to meaningfully adopt, mature and realise benefits.
He also suggested that there should be a “hierarchy of evidence” when it comes to looking at benefit evaluation work supplied by vendors and commercial organisations. “Quality can vary, and an independent evaluation by an academic institution, for example, should have a greater weighting and credibility than an internal assessment from the organisation. The organisation’s assessment in turn should have a greater weighting than an evaluation that has been commissioned and delivered by the supplier.”
Advice for capturing benefits
Asking Paul to give one piece of advice to those responsible for capturing benefits associated with digital deployments, he said it would be to emphasise the importance of having a learning organisation and a culture associated with that. “Fundamentally there are always going to be different experiences with different digital solutions. You need to have a culture that asks the right questions, captures the right information, and is willing to look at a piece of work that has not delivered benefit yet. How can it be optimised to make sure that the project delivers the impact that you are wanting to see?”
For Tom, it comes back to baselining and iterating, as well as being open to learning – from other organisations, from the centre, from suppliers. “Post-procurement should be a living, breathing process by which benefits can continually be measured and iterated. That’s how we can get the most from our technology.”
Many thanks to Paul and Tom for sharing their insights.
