Skip navigation

Patient Reported Outcome Measures (PROMs)

Chaired by Professor W Angus Wallace, Division of Orthopaedic and Accident Surgery University of Nottingham/Nottingham University Hospitals and produced in association with Mr Bibhas Roy, Consultant Orthopaedic Surgeon in Manchester, today's PROMs Summit opened with a national update from NHS England.

Dr Paramala Santosh, Child and Adolescent Psychiatrist at South London and Maudsley NHS Foundation Trust and Kings College London opened the conference discussing how we can use PROMs to predict suicide risk at a population and individual level. The presentation is available for download here.

Dan Wellings, Head of Insight at NHS England updated delegates on national developments in PROMs. Dan emphasised the importance of looking at the impact on patients and users. He said that "some of the presentations he has seen this morning were fantastic in demonstrating "what does that mean?" for patients.  Dan also stated that the National Information Board is absolutely clear on the move to digital". Dan discussed the use of PROMs in keeping track of patients and how they are feeling, when do you screen and offering risk based intervention - Dan said we are not talking about this enough, ensuring the data is being used to influence patient care.

Dan went onto discuss how PROMs fits into the future NHS and forward view stating that beyond compliance with the national programme is very important. With the national programme the data coming through is 12-18 months old  which is a significant problem for the CQC for example when they are quoting the data in inspections. Dan stated that "we need to move to real time data for PROMs" and said that some of the examples delivered in the supplier showcase this morning are good demonstrations of how this could happen.

Dan concluded by stating that we need to ensure we are collecting the right data and we are also using it to the potential. We are starting to learn more and more about the impact on patients. We need to turn the data into practical solutions -dont just give people data and expect them to change. 

Dan's slides are available here

The conference then split into a masterclass focused on PROMs Explained and case study sessions focused on The Long Term Conditions Questionnaire - presentation here, using PROMs in patients with deteriorating conditions - presentation here, and using PROMs results to inform patients and the health professsionals about how they will feel during their patient journey - they aim of the presentation from Prof Simon Rogers, Consultant Maxillofacial Surgeon at Aintree  was to demonstrate how the patient concerns inventory -  has provided holistic patient focus to consultations with the resulting patient benefit and relatively little impact on the cost of care in the outpatient setting. 

The afternoon sessions split into streams focusing on involving patients in PROMs developments and changing clinical practice on the basis of PROMs results. 

PM Session 1: Using PROMS to change commissioning practice: Sue Jackson & Justin Lim
Abstract: UHNM is a large hospital serving a population of 700,000 in the North Midlands. In 2012 we moved all our T&O wards into the new PFI hospital and at the same time became a Major Trauma Centre. Since PROMS started we have been monitoring our participation rate and outcome data. We were surprised to find we were an outlier for our TKR patients especially with the Oxford Knee Score (OKS) results.

Several work streams were set up to look at pre-ams input, patient information and education, ward management and post op discharge therapy provision.

From 2010 our local commissioning teams set out a document relating to Procedures Of Low Clinical Value (POLCV). This included primary THR and TKR and they set requirements including an OKS score threshold to ensure they would commission the procedure.

Without specific consultation or evidence this was set as 16/48 or below for the OKS.

When we were shown to be an outlier of OKS health gain again in 2013/14 data we reviewed our data and other peer trust data. We found we had the lowest average pre-op OKS score, but a good health gain. However when this was adjusted we fell outside the national average.

We felt that part of the issue was due to the fact that we were operating of patients that were comparatively disabled by their knee symptoms resulting in less than optimum outcome. We reviewed the available evidence on the use of scoring to decide surgery threshold and found nothing to support this practice. Following presentation of this evidence and our concerns to the commissioners they agreed to remove the scoring threshold from the 2015 POLCV document.Reviewing and understanding our PROMS data for TKR patients helped us change our commissioner’s minds.

PM Session 2: Understanding PROMs results through audit and analysis of outliers: Catherine Armstrong

A Retrospective Audit of Patient Reported Outcome Measures (PROMS) was undertaken at Royal Liverpool and Broadgreen University Hospitals Trust (RLBUHT) to look at:

  • Why PROMS results for primary THA showed poor outcomes for RLBUHT?
  • What changes could be made to improve these outcomes?


DEVELOPMENTS: The Health and Social Care Information Centre (HSCIC).

6 month PROMS for THA showed RLBUHT as an outlier compared to other trusts in England


The HSCIC reported response rate of 51.8%. Of the responders, 39 patients were identified as being worse off at 6 months. All of these patients were called back and reviewed by an independent arthroplasty care practitioner. 

Results revealed many discrepancies in coding of the procedures and between patient perception of success and how they scored on PROMS. The need for education regarding expectation post-surgery, education on PROMS and the socioeconomic status which can affect PROMS outcomes were highlighted.

A Prospective Audit of PROMS for Total Hip Arthroplasty (THA) was then implemented

PROMS assessed at 6 months Post-operatively to facilitate

  • Comparison of PROMS by post and in Clinic setting

    OBJECTIVE: Is there a difference between PROMS collected via post and via face to face contact with an independent clinician in the out-patient department for patients following Primary THA.

METHOD: As part of the clinical review at six months by an independent arthroplasty care practitioner in the arthroplasty follow up clinic, PROMS were also recorded. This was then used for direct comparison with the postal 6 month PROMS recorded by the HSCIC.

In addition to the scores, patients were asked if they felt their condition had changed since their pre-operative state and whether they were satisfied with the outcome.

RESULTS: Response rate improved from 51.8% to 76.5% compared to the retrospective audit. Average oxford score was 36.86 compared to 15.87 pre-op (mean improvements of 51.82%). 48.5% of patients had an Oxford Hip Score of less than 40 despite 80.2% describing their outcome as excellent or very good and 85.1% reporting they were much better post-operatively.

HSCIC received 34 returned questionnaires at 6 month post op 24.8% of those sent out. Of these patients only 23 corresponded to our patients.

Statistical analysis was carried out to compare these 23 patients.

Analysis of results showed no significant difference between RLBUHT and HSCIC with respect to EQ5D index, EQ5D VAS and Oxford Hip Score (Wilcoxon signed ranks test p≥0.1).

A retrospective audit of poor outcomes of PROMS for THA revealed discrepancies in coding of procedures and in patient reporting of their outcome compared to outcome measures (in 39 patients). A prospective audit of a 6 month time period compared PROMS recorded in the clinic compared to postal PROMS recorded by HSCIC. Outcome measures of EQ5D, EQ5DVAS and Oxford Hip Score revealed no significant difference between clinic completed and postal PROMS. However, small numbers of patients were matched between the two data sets. Further discrepancies were noted in coding of the procedures and matching of patients with RLBUHT data.

Analysis of a longer time period to correspond with the time frames used by HSCIC may result in larger numbers for analysis. Continued efforts need to be made to ensure patients are coded correctly and HSCIC receive information regarding primary or revision surgeries. Additionally further patient education needs to be undertaken to improve response rates to postal PROMS.

PM Presentation 3: The development of a new National PROMs for Major Injury: Antoinete Edwards, TARN
Abstract: The Trauma Audit and Research Network (TARN) is the national clinical audit for trauma care across England, Wales and the Republic of Ireland and has been supporting trauma receiving trusts for 25 years by providing each hospital with case mix adjusted outcome analysis, performance of key process measures and comparisons of trauma care. TARN hold the largest trauma registry in Europe with over ½ million cases. 

PM Presentation 4: PROMs data to support commissioning of varicose vein procedures: Elizabeth Lingard 
Presention available here 

Involving patients in PROMS developments - Sue Jones, Jonathan Hope MBE, Anju Keetharuth
Sue and Jonathan spoke about working with patients to develop PROMS and involving patients in PROM developments. Sue is a person living with ME,

Susan believes that PROMs could have a major part to play in bringing order to chaos, in bringing together the UK psychiatric lobby (who seem to have influence over the UK media and the NICE guidelines) from one side, and the patients, medical researchers and other physicians from the other.  This meeting of ways must start by establishing, through BIG DATA collected via PROMs, clear-cut diagnostic criteria for ME based on objective rather than subjective data, and should not rely solely on exclusion of other conditions.  This would enable the two sides to work with, rather than against, each other, in the knowledge that they are talking about the same set of patients. Two published papers (*1) put the current misdiagnosis rate at approximately 50%. Once this well-defined patient base is established, behavioural and ‘bio medical’ approaches can be directly compared using the same objective outcome measures alongside appropriate subjective ones. ME patients have been collecting this objective data through a lottery funded website with some very encouraging results. They would be delighted to get some support and input from those charged with their care.  At the summit, Susan will illustrate the measures relating to autonomic nervous system abnormalities and cardiovascular abnormalities giving results from controls as well as from ME patients.

Co Production of PROMS: Anju Keetharuth

The overall aim of this paper is to report on the development of a new generic PROM for routine use in the area of mental health.  This work has been commissioned by the Department of Health as there are no existing PROMs that capture what really matter to service users of mental health services.

Background and method

The construction of the measure is divided into four distinct stages. In stage 1, the domains of the items are identified through a systematic review and qualitative interviews.  In Stage 2, potential negative and positive items are selected from existing measures and interview manuscripts. In Stage 3, the items are tested with service users to ensure the face validity of the items. In the final Stages, data is collected on the shortlisted items from service users and psychometric analyses are performed to help identify items for the final ReQoL measures.


Results and conclusions

The seven themes identified for the ReQoL measures are: Well-being; Self-perception; Activities; Hope; Belonging and relationships; Autonomy, control and choice and; Physical health.  From 2000 potential items selected, the number of items has been reduced to about 100 through adoption of a set of criteria. Service users in qualitative interviews contributed to reducing the number of items to 61.  In the first psychometric stage over 2000 service users have filled in the ReQoL questionnaire. Classical psychometrics techniques are used to confirm the dimensionality of the instrument and Rasch analysis is used to reduce the number of items to produce a pool of 40 items. In the second psychometric stage, over 2000 service users have filled in the latter item pool and these results will be used to construct the short and the long ReQoL with about 10 and 30 items respectively. It is expected that the shorter version will be used routinely in mental health services and the longer version can be used in trials and in clinical decision making. 

The development of the ReQoL measures highlights the steps in developing an outcome measure from service users’ perspectives with inputs from clinicians and other stakeholders at every step.

The main strengths of this study arise from the following:

  • Involvement of service users at all stages of development of the PROM
  • Establishing the dimensions through qualitative research to ensure that the PROM covers what really matter to service users 
  • Constant validation of the outputs at each stage of development
  • Inputs from clinicians and other stakeholders. 


Final Joint Plenary

Future events of interest:

A Practical Guide to Improving Outpatient Services
Friday 22 January 
Hallam Conference Centre, London

Measuring, Understanding and Acting on Patient Experience Insight
Friday 29 January 
Hallam Conference Centre, London

Measuring & Monitoring Clinical Quality
Wednesday 3 February 
ICO Conference Centre, London

Hip Fracture: Meeting the NICE Quality Standard and Learning from the NHFD Findings
Tuesday 1 March 
Hallam Conference Centre, London


1 December 2015


    Partner Organisations

    The Tavistock and Portman NHS Foundation TrustInPracticeClinical Audit Support CentrePlayoutJust For Nurses
    GGI (Good Governance Institute) accredited conferences CPD Member BADS (British Association of Day Surgery) accredited conferences