A pragmatic effectiveness-implementation study comparing trial evidence with routinely collected outcome data for patients receiving the REACH-HF home-based cardiac rehabilitation programme
BMC Cardiovascular Disorders volume 22, Article number: 270 (2022)
Cardiac rehabilitation for heart failure continues to be greatly underused worldwide despite being a Class I recommendation in international clinical guidelines and uptake is low in women and patients with mental health comorbidities.
Rehabilitation EnAblement in CHronic Heart Failure (REACH-HF) programme was implemented in four UK National Health Service early adopter sites (‘Beacon Sites’) between June 2019 and June 2020. Implementation and patient-reported outcome data were collected across sites as part of the National Audit of Cardiac Rehabilitation. The change in key outcomes before and after the supervised period of REACH-HF intervention across the Beacon Sites was assessed and compared to those of the intervention arm of the REACH-HF multicentre trial.
Compared to the REACH-HF multicentre trial, patients treated at the Beacon Site were more likely to be female (33.8% vs 22.9%), older (75.6 vs 70.1), had a more severe classification of heart failure (26.5% vs 17.7%), had poorer baseline health-related quality of life (MLHFQ score 36.1 vs 31.4), were more depressed (HADS score 6.4 vs 4.1) and anxious (HADS score 7.2 vs 4.7), and had lower exercise capacity (ISWT distance 190 m vs 274.7 m). There appeared to be a substantial heterogeneity in the implementation process across the four Beacon Sites as evidenced by the variation in levels of patient recruitment, operationalisation of the REACH-HF intervention and patient outcomes. Overall lower improvements in patient-reported outcomes at the Beacon Sites compared to the trial may reflect differences in the population studied (having higher morbidity at baseline) as well as the marked challenges in intervention delivery during the COVID-19 pandemic.
The results of this study illustrate the challenges in consistently implementing an intervention (shown to be clinically effective and cost-effective in a multicentre trial) into real-world practice, especially in the midst of a global pandemic. Further research is needed to establish the real-world effectiveness of the REACH-HF intervention in different populations.
Contributions to the literature
Understanding of real-world implementation and impact on patient-reported outcomes of an evidence-based intervention for heart failure.
Rehabilitation EnAblement in CHronic Heart Failure may be a viable cardiac rehabilitation alternative for older patients and patients with mental health comorbidities and lead to an increase in the uptake of cardiac rehabilitation in these currently underserved populations.
More effort is needed to offer the programme to a wider range of patients. This can be achieved by improving referral pathways and/or making the programme more accessible/appealing to different patients.
The study is unique in utilising routine data to evaluate the real-world implementation of trial evidenced intervention; the findings can inform future studies employing similar methodologies.
This study is part of a larger implementation project , the main output of this project is an implementation manual, which is available to healthcare teams interested in including REACH-HF in their service provision in the National Institute for Health and Care Excellence guidance .
Cardiac rehabilitation for patients with heart failure
Cardiac rehabilitation for heart failure continues to be greatly underused worldwide despite being a Class I recommendation in international clinical guidelines. Less than 20% of eligible patients in Europe receive this intervention  and fewer still in the United States . The landscape of cardiac rehabilitation provision before the COVID-19 pandemic was dominated by group-based programmes delivered in hospitals and community cardiac rehabilitation centres . Providing different modes of delivery (such as telemedicine or home-based programmes) are proposed alternatives that could help to increase the uptake of cardiac rehabilitation [4, 6,7,8,9].
Intervention – the REACH-HF programme
The Rehabilitation EnAblement in CHronic Heart Failure (REACH-HF) programme is a clinically effective and cost-effective, home-based, healthcare professional-facilitated cardiac rehabilitation programme for patients with heart failure . The 12-week programme is delivered using a mixture of home visits and telephone calls. The average contact time per patient in the REACH-HF effectiveness trial was 5.3 h (made up of 4.5 h of face-to-face contact and 0.8 h of telephone facilitation) over an average of 6.5 sessions. The REACH-HF programme and the multicentre trial are described in more detail elsewhere [10,11,12,13].
Moving research findings into routine clinical practice is a well-documented challenge [14, 15]. Once implemented, innovations often fail to replicate the effectiveness reported in clinical trials . As many as 23 contextual factors can impact the implementation process, leading to variations in the delivery of the same programme between centres/teams . For example, patients treated in real-world clinical settings might be different to those recruited into the trial, staff implementing the innovation might lack the knowledge and skills required to deliver the intervention as it was intended, or a lack of resources might hinder effective implementation [18, 19].
This study aimed to answer the following research questions: (1) What are the variations in the implementation process across the four Beacon Sites and how do such variations impact on patients’ outcomes?, (2) Are patients receiving the REACH-HF intervention at the Beacon Sites comparable to those recruited into the trial and more representative of the general population of heart failure?’ and (3) Are changes in outcomes of interest derived from routine data collected at the Beacon Sites for patients receiving the REACH-HF intervention comparable with the changes observed in the REACH-HF trial?.
Methods and analysis
The current effectiveness-implementation study used a multi-centre prospective cohort design to evaluate the implementation process and compare trial evidence with routinely collected pre-treatment and post-treatment healthcare data during the real-world implementation of the intervention at four Beacon Sites.
The Beacon Sites selection process is described in detail in the published protocol  and summarised in Additional file 1. Four Beacon Sites were set up to deliver REACH-HF to 200 patients between June 2019 and June 2020; characteristics and cardiac rehabilitation activity before becoming Beacon Sites are summarised in Table 1. Each site was required to administer two key outcome measures as were used in the clinical trial–the Minnesota Living with Heart Failure Questionnaire (MLHFQ) and the Incremental Shuttle Walk Test (ISWT).
Three cohorts of patients with heart failure were compared in the study: patients who received REACH-HF at the Beacon Sites, patients who received REACH-HF within the multicentre randomised controlled trial and the general population with heart failure – all heart failure patients (except those receiving REACH-HF at the Beacon Sites) in the National Audit of Cardiac Rehabilitation (NACR) database who received cardiac rehabilitation during the Beacon Site period (between June 2019 and June 2020).
Audit data (see Results Tables) from the NACR database were used to understand the implementation process by considering how the intervention was delivered and the socio-demographic characteristics and medical data of patients treated in the Beacon Sites. The audit data were also used to compare patient characteristic and real-world changes in outcomes of patients receiving the REACH-HF programme to the prior clinical trial findings . A detailed description of all the outcome measures can be found in Additional file 2.
Data collection and analysis
A download of the NACR data was conducted by the NACR data scientist (AH) in February 2021. We focused on the four-month trial follow-up timepoint, as it was the best match for the time period between pre-treatment and post-treatment assessments at the Beacon Sites (mean 125.2 days). Due to data governance requirements, analyses using individual patient-level data were conducted by NACR staff.
Descriptive statistics were used to report the characteristics of the study sites and the patient sample. To explore whether the MLHFQ within-group change in the NACR data was different from the within-group change in the trial, a three-group (Beacon Sites vs trial treatment arm vs trial control arm) comparison (ANCOVA) was conducted with adjustment for MLHFQ baseline score, age and sex. To understand the within-group differences at the Beacon Sites and the trial, we used a standardised mean difference approach and calculated effect sizes for primary and secondary outcome measures adjusted for the sample size (Hedges’ g) [20, 21].
Implementation process and sample characteristics
Implementation activity at the Beacon Sites is summarised in Table 2 (data for the individual sites can be found in Table 6 in Additional file 3). Based on the data recorded by the cardiac rehabilitation providers, of the 132 patients enrolled on the REACH-HF programme between June 2019 and June 2020, 96 (72.7%) completed the programme. The completion rate was calculated based on the patient having two valid assessments (baseline and end-of-treatment) or a ‘completed’ box being ticked for patients who were unable to attend their assessment appointments.
On average, patients received 7.5 (SD 3.9) clinical sessions (a combination of centre-based assessments, home visits and telephone support) over 125.3 days (SD 47.4). There was a large variation in the number of sessions attended with a minimum of one and a maximum of 24 sessions. A similar variation was observed with treatment duration, 70 days was the minimum and 299 days the maximum.
The comparison of key participant characteristics across the three cohorts can be found in Table 3. Of the 132 patients enrolled on the programme, 86 (66.2%) were male and 44 (33.8%) were female. The mean age of patients treated at the four Beacon Sites was 75.6 (SD 11.1) years old; Beacon Site 1 patients were a mean of 10.5 years older than the remaining sites. Table 7 in Additional file 4 includes socio-demographic characteristics of patients enrolled on the REACH-HF programme at individual Beacon Sites and missing data, as well as more detailed comparison data.
Table 5 compares primary and secondary outcomes achieved at the Beacon Sites with those from the trial. The ANCOVA analysis confirmed that the trial population did significantly better (Mean Difference − 7.2: 95%CI − 14.1 to − 0.3) compared with the Beacon Site population in terms of improvement in health-related quality of life measured by the MLHFQ. Furthermore, the ANCOVA comparison (NACR data vs trial control group) confirmed that there was no significant difference in the improvement of the MLHFQ scores at the Beacon Sites (Mean Difference 1.9: 95%CI -5 to 8.9).
The pre-treatment and post-treatment effect sizes calculated at the Beacon Sites (Table 5) did not match the effect sizes calculated from the trial data for the health-related quality of life – MLHFQ (Mean Difference − 0.09: 95% CI − 0.49 to 0.30 vs − 0.42: 95% CI − 0.70 to − 0.13), mental health measures (Hospital Anxiety and Depression Scale (HADS) depression – Mean Difference − 0.15: 95% CI − 0.73 to 0.43 vs − 0.20: 95% CI − 0.48 to 0.09) and exercise capacity measure – ISWT (Mean Difference 0.17: 95% CI − 0.60 to 0.94 vs 0.27: 95% CI − 0.03 to 0.58). The pre-treatment and post-treatment effect size for HADS anxiety score recorded at the Beacon Sites exceeded the effect size calculated from the trial data (Mean Difference − 0.14: 95% CI − 0.72 to 0.44 vs − 0.07: 95% CI − 0.36 to 0.21).
We observed the following pattern of variation in the magnitude of effect sizes at the individual Beacon Sites – a deterioration in all outcome measures at Site 1, no change in the MLHFQ and a small positive change in the mental health measures at Site 2, a small positive change in all available outcome measures at Site 3 and a small, a medium and a large effect size at Site 4 for the MLHFQ, and the HADS depression and anxiety domains, respectively. Table 9 in Additional file 6 lists effect sizes for primary and secondary outcome measures at individual Beacon Sites.
The primary and secondary analysis confirmed that the results found in the REACH-HF trial intervention group were not replicated in the Beacon Site population. However, it was difficult to conclude definitively that the real-world implementation resulted in lower effectiveness, as there were substantial differences between the trial and the Beacon Sites in terms of the patient population and the level of implementation (treatment dose/duration). We also encountered a small sample size and missing data, which introduced selection bias into the study, further complicated by a lack of a non-treatment control group. The discrepancies in the implementation process and population characteristics observed in the study are consistent with other studies looking at the implementation of interventions that have been found to be effective in trials. At the Beacon Sites, the usual implementation challenges were exacerbated by severe service disruptions due to the COVID-19 pandemic.
Possible explanations of the data
Firstly, the COVID-19 pandemic led to substantial variations in the delivery of the programme across the Beacon Sites. Our findings suggest that when delivered as part of routine practice, the programme was spread out over a longer time period (an additional 41 days); there was also a large variation in the number of treatment sessions received. This might suggest reduced treatment fidelity, leading to lower effectiveness. The inability to deliver the treatment as it was intended (i.e., face-to-face and in the patient’s home) might have resulted in a diminished quality of care compared to the trial. For example, having to substitute face-to-face home visits for phone sessions might have impacted the nurse’s ability to deliver the person-centred interactions that are at the core of the intervention .
Secondly, compared to the trial population, patients treated at the Beacon Sites were older, more depressed and anxious, had lower exercise capacity, and experienced more debilitating symptoms of heart failure. The 4.7-point difference in mean MLHFQ between cohorts is close to the minimal clinically important difference of five points , which suggests that the health-related quality of life of patients treated at the Beacon Sites was lower than that of those recruited into the trial. In more clinically morbid populations with heart failure, the normal prognosis is for a worsening of the condition and an associated decline in quality of life over time . Hence, in this population, a pattern of ‘no decline’ in symptoms over time may be a positive outcome. As we do not have data from a control group with the same baseline characteristics, we cannot be sure that this is the case here. However, the pattern of data between sites indicates that this is a plausible explanation. At Site 1, which treated the oldest/the most frail patients in our cohort, there was a consistent deterioration in all outcome measures (i.e., MLHFQ, Dartmouth Cooperative Functional Assessment Charts, HADS and ISWT), whereas, across the other sites the trend was more positive (see Table 9 in Additional file 6).
Historically, the uptake of cardiac rehabilitation in older patients with heart failure and in patients with mental health issues has been particularly low [25, 26]. As patients who received REACH-HF at the Beacon Sites compared to those recorded in the wider NACR dataset for the same period were older (on average 6.8 years) and had more mental health morbidity (HADS depression score 6.4 vs 5.5 and HADS anxiety score 7.2 vs 5.9), it may be suggested that the REACH-HF home-based cardiac rehabilitation programme might be a more acceptable form of rehabilitation to older/more frail patients and patients with mental health comorbidities compared with centre-based programmes (or at least that cardiac rehabilitation staff feel more comfortable recommending this rehabilitation option to these patients). In the real-word, only a limited range of patients accessed the programme, hence, the clinical implication of the study, irrespective of the cause of this pattern (patient preference or referral procedures), is a need to make the intervention accessible to a wider range of participants.
Strengths and limitations
Using routinely collected audit data was a low-cost and a low research-burden option that allowed us to conduct a rapid review of the implementation process and to compare REACH-HF trial outcomes with real-world implementation outcomes. The study is of high clinical relevance, as the initial results suggest that the REACH-HF programme may be a viable cardiac rehabilitation alternative for older patients and patients with mental health comorbidities, both often underrepresented in traditional centre-based programmes (or that healthcare professionals are more open to offering REACH-HF to such patients). However, steps need to be taken to make the programme more accessible to a wide range of patients. The study also provides good insight into conducting implementation research in challenging contexts and during unprecedented (for healthcare services, healthcare professionals and the general population) times.
The main limitations of the study were the substantial variations in implementation of the treatment, high levels of missing data, a limited sample size that may have introduced selection bias (all impacted by the COVID-19 pandemic), and a lack of a comparable patient group (the trial population we had selected for comparison was substantially different to the implementation sample).
More research, as is currently being conducted in the SCOT REACH-HF project, which is examining the real-world roll out of REACH-HF across six health boards in Scotland , is needed to understand the impact of REACH-HF (and other home-based rehabilitation treatments) on different populations and in relation to delivery characteristics (quantity and quality) and outside the context of global pandemic and lockdowns.
Additionally, as the amount of NACR routinely collected data from cardiac rehabilitation teams offering the REACH-HF intervention UK-wide increases, more rigorous analysis of implementation effectiveness can be applied (i.e., patient-level analysis, further consideration of different implementation contexts, correlation between the intervention (e.g., number of clinical sessions attended) and outcomes). However, steps should be taken to reduce the amount of missing follow-up data, particularly for key outcome measures such as health-related quality of life.
Our data suggest that changes in patient-related outcomes seen in the REACH-HF randomised trial were not replicated in the real-world setting of this study. They also suggest that offering home-based cardiac rehabilitation may facilitate uptake amongst older patients and those with mental health comorbidities. However, our results need to be interpreted carefully given substantial differences in the populations treated and the context of the COVID-19 pandemic, which may have affected the intensity of treatment delivery.
Availability of data and materials
The datasets analysed during the current study are available from the corresponding author on reasonable request.
Patrick J Doherty
Jet JCS Veldhuijzen van Zanten
Hasnain M Dalal
Rod S Taylor
Samantha B van Beurden
Sinéad T J McDonagh
Rehabilitation EnAblement in CHronic Heart Failure
National Audit of Cardiac Rehabilitation
Minnesota Living with Heart Failure Questionnaire
Hospital Anxiety and Depression Scale
Incremental Shuttle Walk Test
Analysis of covariance
Daw P, van Beurden SB, Greaves C, Veldhuijzen van Zanten J, Harrison A, Dalal H. Getting evidence into clinical practice: protocol for evaluation of the implementation of a home-based cardiac rehabilitation programme for patients with heart failure. BMJ Open. 2020;10:e036137.
National Institute for Health and Care Excellence. COVID-19 ready rehabilitation for heart failure: REACH-HF can deliver 2021 [Available from: https://www.nice.org.uk/sharedlearning/covid-19-ready-rehabilitation-for-heart-failure-reach-hf-can-deliver.
Bjarnason-Wehrens B, McGee H, Zwisler A-D, Piepoli MF, Benzer W, Schmid J-P, et al. Cardiac rehabilitation in Europe: results from the European Cardiac Rehabilitation Inventory Survey. Eur J Cardiovasc Prevent Rehabil. 2010;17(4):410–8.
Golwala H, Pandey A, Ju C, Butler J, Yancy C, Bhatt DL. Temporal trends and factors associated with cardiac rehabilitation referral among patients hospitalized with heart failure: findings from Get With The Guidelines-Heart Failure Registry. J Am College Cardiol. 2015;66:917–26.
Anderson L, Sharp GA, Norton RJ, Dalal H, Dean SG, Jolly K, et al. Home‐based versus centre‐based cardiac rehabilitation. Cochrane Database Syst Rev. 2017;6(6).
Dalal HM, Wingham J, Palmer J, Taylor R, Petre C, Lewin R, et al. Why do so few patients with heart failure participate in cardiac rehabilitation? A cross-sectional survey from England, Wales and Northern Ireland. BMJ Open. 2012;2: e000787.
Palmer K, Bowles K-A, Lane R, Morphet J. Barriers to engagement in chronic heart failure rehabilitation: an Australian survey. Heart Lung Circ. 2020;29(8):e177–84.
Dalal HM, Doherty P, McDonagh STJ, Paul K, Taylor RS. Virtual and in-person cardiac rehabilitation. BMJ. 2021;373: n1270.
Thomas RJ, Beatty AL, Beckie TM, Brewer LC, Brown TM, Forman DE, et al. Home-based cardiac rehabilitation: a scientific statement from the American association of cardiovascular and pulmonary rehabilitation, the American heart association, and the American college of cardiology. Circulation. 2019;140(1):e69–89.
Dalal HM, Taylor RS, Jolly K, Davis RC, Doherty P, Miles J, et al. The effects and costs of home-based rehabilitation for heart failure with reduced ejection fraction: The REACH-HF multicentre randomized controlled trial. Eur J Prev Cardiol. 2019;26(3):262–72.
Dalal HM, Taylor RS, Wingham J, Greaves CJ, Jolly K, Lang CC, et al. A facilitated home-based cardiac rehabilitation intervention for people with heart failure and their caregivers: a research programme including the REACH-HF RCT. Program Grants Appl Res. 2021;9:1.
Frost J, Wingham J, Britten N, Greaves C, Abraham C, Warren FC. Home-based rehabilitation for heart failure with reduced ejection fraction: mixed methods process evaluation of the REACH-HF multicentre randomised controlled trial. BMJ Open. 2019;9(8).
Greaves CJ, Wingham J, Deighan C, Doherty P, Elliott J, Armitage W, et al. Optimising self-care support for people with heart failure and their caregivers: development of the Rehabilitation Enablement in Chronic Heart Failure (REACH-HF) intervention using intervention mapping. Pilot Feasibil Stud. 2016;2(37).
Leavitt M. Medscape’s response to the Institute of Medicine Report: Crossing the quality chasm: a new health system for the 21st century. MedGenMed. 2001;3(2):2.
Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93(8):1261–7.
Fixsen D, Naoom S, Blase K, Friedman R, Wallace F. Implementation Research: A Synthesis of the Literature. Tamps, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network. 2005.
Durlak JA, Dupre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50.
Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R, et al. A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci. 2018;13(1):125.
Breitenstein SM, Gross D, Garvey CA, Hill C, Fogg L, Resnick B. Implementation fidelity in community-based interventions. Res Nurs Health. 2010;33(2):164–73.
Hedges LV. Distribution theory for glass’s estimator of effect size and related estimators. J Educ Stat. 1981;6(2):107–28.
Goulet-Pelletier J-C, Cousineau D. A review of effect sizes and their confidence intervals, Part I: The Cohen’s d family. Quant Methods Psychol. 2018;14(4):242–65.
Sheffield KM, Dreyer NA, Murray JF, Faries DE, Klopchin MN. Replication of randomized clinical trial results using real-world data: paving the way for effectiveness decisions. J Comp Effect Res. 2020;9(15):1043–50.
American Thoracic Society. Minnesota living with heart failure questionnaire. New York: American Thoracic Society; 2004.
Jones NR, Hobbs FR, Taylor CJ. Prognosis following a diagnosis of heart failure and the role of primary care: a review of the literature. BJGP Open. 2017;1(3):bjgpopen17X101013-bjgpopen17X.
British Heart Foundation. The national audit of cardiac rehabilitation: quality and outcomes report 2019. London: BHF; 2019.
Sever S, Doherty P, Harrison AS, Golder S. To what extent is multi-morbidity associated with new onset depression in patients attending cardiac rehabilitation? BMC Cardiovasc Disord. 2019;19(1):256.
Purcell C, Daw P, Kerr C, Cleland J, Cowie A, Dalal HM. Protocol for an implementation study of an evidence-based home cardiac rehabilitation programme for people with heart failure and their caregivers in Scotland (SCOT:REACH-HF). BMJ Open. 2020;10.
We thank the REACH-HF team for setting up and managing the Beacon Sites, all NHS staff working at the Beacon Sites for their co-operation and providing the patient data.
PD’s time is funded by a PhD studentship from the University of Birmingham.
Ethics approval and consent to participate
The study (IRAS 261723) received ethics approval from the South Central (Hampshire B) Research Ethics Committee (19/SC/0304). Where required, written informed consent was obtained from all healthcare professionals and patients participating in the study. The study was conducted in accordance with the Declaration of Helsinki, the Data Protection Act 2018, the General Data Protection Regulation and in accordance with the Research Governance Framework for Health and Social Care (2005).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Setting up the Beacon Sites.
Detailed description of primary and secondary outcome measures.
REACH-HF activity at individual Beacon Sites between June 2019 and June 2020.
Socio-demographic characteristics of patients enrolled onto the REACH-HF programme at individual Beacon Sites between June 2019 and June 2020 (including missing data) AND Table 7b. Comparison of patients receiving the REACH-HF intervention at the Beacon Sites between June 2019 and June 2020, those recruited into the REACH-HF trial and the general heart failure population recorded in the NACR database between June 2019 and June 2020 (excluding Beacon Sites patients).
Referral sources for patients enrolled on the REACH-HF programme at individual Beacon Sites between June 2019 and June 2020.
Effect sizes for primary and secondary outcome measures at individual Beacon Sites for patients receiving REACH-HF programme between June 2019 and June 2020.
About this article
Cite this article
Daw, P., Harrison, A., Doherty, P.J. et al. A pragmatic effectiveness-implementation study comparing trial evidence with routinely collected outcome data for patients receiving the REACH-HF home-based cardiac rehabilitation programme. BMC Cardiovasc Disord 22, 270 (2022). https://doi.org/10.1186/s12872-022-02707-5
- Heart failure
- Cardiac rehabilitation
- Treatment outcome
- Quality of life
- Routinely collected health data
- Quantitative evaluation