How to measure what can't easily be measured

Reported by Makoto Takahashi, ESRC-funded CSaP Policy Intern (January - April 2016)

How can research findings influence patient care more quickly? It takes an average of 17 years for a promising development in a laboratory to translate into better treatments or services for patients. In 2014, the National Institute of Health Research (NIHR) launched its Push the Pace programme, which aims to shave 20 months off this process.

In support of this initiative, CSaP held a Policy Workshop to brainstorm ideas on how the NIHR can measure the impact of its interventions. Chaired by Dr Julian Huppert, the workshop brought together academic experts with senior policy makers for an exploratory discussion.

"For me, there's nothing more practical than generating great ideas," said Mark Samuels, Managing Director of the NIHR Office for Clinical Research Infrastructure (NOCRI). "This is a live project...so any good ideas...will be kicked into action. I think that's a good reflection on CSaP's practical delivery and impact."

Participants agreed that it is very difficult to model the impact of Push the Pace accurately, owing to the complex nature of the NIHR's work. As there is no 'typical' journey, it is hard to establish exactly how long a research idea takes to flow through the system.

But the difficulty of measuring the impact of the NIHR's reforms does not diminish their importance, and participants were keen to suggest mechanisms through which the NIHR could encourage efficiency.

It was stressed, however, that any new system would have to be carefully calibrated, in order to avoid unintended patterns of behaviour.

The NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) had been wrestling with the question of measurement for some time. Principal Information Officer, Richard Dudley, said of the workshop: "Getting into a room with people who say 'you can't do that, but what you could do is this,' or 'actually that's not what you want to do' is extremely valuable, and we will be taking away some fantastic stuff to think about."