The impact of the HEART risk score in the early assessment of patients with acute chest pain: design of a stepped wedge, cluster randomised trial
© Poldervaart et al.; licensee BioMed Central Ltd. 2013
Received: 24 July 2013
Accepted: 17 September 2013
Published: 26 September 2013
Chest pain remains a diagnostic challenge: physicians do not want to miss an acute coronary syndrome (ACS), but, they also wish to avoid unnecessary additional diagnostic procedures. In approximately 75% of the patients presenting with chest pain at the emergency department (ED) there is no underlying cardiac cause. Therefore, diagnostic strategies focus on identifying patients in whom an ACS can be safely ruled out based on findings from history, physical examination and early cardiac marker measurement. The HEART score, a clinical prediction rule, was developed to provide the clinician with a simple, early and reliable predictor of cardiac risk. We set out to quantify the impact of the use of the HEART score in daily practice on patient outcomes and costs.
We designed a prospective, multi-centre, stepped wedge, cluster randomised trial. Our aim is to include a total of 6600 unselected chest pain patients presenting at the ED in 10 Dutch hospitals during an 11-month period. All clusters (i.e. hospitals) start with a period of ‘usual care’ and are randomised in their timing when to switch to ‘intervention care’. The latter involves the calculation of the HEART score in each patient to guide clinical decision; notably reassurance and discharge of patients with low scores and intensive monitoring and early intervention in patients with high HEART scores. Primary outcome is occurrence of major adverse cardiac events (MACE), including acute myocardial infarction, revascularisation or death within 6 weeks after presentation. Secondary outcomes include occurrence of MACE in low-risk patients, quality of life, use of health care resources and costs.
Stepped wedge designs are increasingly used to evaluate the real-life effectiveness of non-pharmacological interventions because of the following potential advantages: (a) each hospital has both a usual care and an intervention period, therefore, outcomes can be compared within and across hospitals; (b) each hospital will have an intervention period which enhances participation in case of a promising intervention; (c) all hospitals generate data about potential implementation problems. This large impact trial will generate evidence whether the anticipated benefits (in terms of safety and cost-effectiveness) of using the HEART score will indeed be achieved in real-life clinical practice.
Patients presenting with chest pain at the emergency department (ED) pose a diagnostic challenge. Chest pain can be a symptom of an acute coronary syndrome (ACS), i.e. acute myocardial infarction (AMI) or unstable angina, which is the case in approximately 20% of the patients and requires prompt treatment. In the remaining 80%, chest pain is caused by many other, usually not life-threatening, conditions . Unfortunately, decision-making in chest pain patients is hampered by limited predictive power of patient characteristics, including signs, symptoms and additional tests [1–3]. Therefore, physicians face the challenge of not wanting to miss an ACS on the one hand, while avoiding too many unnecessary diagnostic procedures that can be time-consuming and patient burdening on the other hand. Currently, the fear of missing a relevant cardiac condition makes physicians cautious and, to be on the safe side, a large proportion of patients are kept in the hospital from several hours to days for monitoring or additional testing.
Diagnostic strategies in patients with chest pain therefore focus on identifying patients in whom ACS can be safely ruled out based on readily available clinical findings from history, physical examination and early marker measurement of cardiac damage. Recent guidelines suggest the use of well-developed and validated risk scores to stratify patients in the emergency room [4–6]. Several prognostic risk scores have been developed for patients diagnosed with ACS, such as the TIMI (Thrombolysis in Myocardial Infarction) risk score and the GRACE (Global Registry of Acute Cardiac Events) risk score [7–9]. However, scores that identify ACS in patients suspected of ACS in the emergency setting and predict short-term mortality or coronary intervention are not available. The HEART score has been specifically developed for risk stratification in all patients with chest pain presenting at the ED.
Elements to calculate HEART score for chest pain patients at the emergency department
Slightly or non-suspicious
Nonspecific repolarization disturbance
≥ 65 years
>45 – <65 years
≤ 45 years
≥ 3 risk factors*, or history of atherosclerotic disease^
1 or 2 risk factors
No risk factors known
≥ 3x normal limit
>1 - <3x normal limit
≤ normal limit
Summary of results of previous validation studies of the HEART score
Number of patients
N = 880 
N = 2388 
N = 2906 
N = 6174
9 countries in the Asia-Pacific region
Jan ‘06 – Mar ‘06
Oct ‘08 –Nov ‘09
Nov ‘07 – Dec ‘10
Type of patients
Patients presenting with chest pain at the ED
Patients presenting with chest pain at the ED
Patients presenting with chest pain at the ED
AMI, revascularisation, all cause death
AMI, revascularisation, stenosis managed conservatively, all cause death
AMI, revascularisation, death unless clearly non-cardiac
Duration of follow-up
Cumulative incidence of MACE stratified by HEART scores
1.6% (95%-CI 1.05-2.15)
12.5% (95%-CI 11.34-13.66)
49.4% (95%-CI 46.37- 52.43)
Therefore, we designed the HEART Impact study to investigate whether the use of the HEART score in the management of chest pain patients indeed leads to these positive health effects, while not causing an increase in the occurrence of MACE.
Study design: stepped wedge randomised trial
‘Usual care’ and ‘HEART’ period
Usual care is defined as ‘daily practice of the cardiologist or attending emergency doctor to diagnose a patient with chest pain’. In this period, attending doctors assess the risk based on their clinical skills, previous experiences, gut feeling and various other criteria (for example, described in European Society of Cardiology Guidelines ), without using the HEART score. No attempt was made to explicitly standardise usual care across all hospitals. The assessment will typically include: gender, age, medical (cardiac) history, symptoms, risk factors, and current drug use, physical examination with special attention for the heart and lungs, blood pressure, heart rate, blood tests, ECG, and any other diagnostic procedures the physician considers necessary. The standard blood tests include measurements of troponin, glucose levels, creatinin levels (with a calculated estimated glomerular filtration rate (eGFR) according to MDRD (modification of diet in renal disease)), haemoglobin, and any other blood test required. A standard 12-lead ECG is recorded by a trained employee of the ED and classified by a single cardiologist according to the Minnesota coding criteria. All investigations take place in the ED.
During the HEART period, the HEART score will be formally determined in all patients. Decision-making about whether to admit a patient, any further testing or treatment decisions will be carried out similarly to usual care, with the exception of the availability of the HEART score in each individual patient and the recommendations linked to that score. This is also known as “directive use” of a prediction rule, as opposed to “assistive use” where only the predicted risk is given to the physician . The recommendation for patients with a HEART score of 3 or lower will be reassurance and discharge. In those low-risk patients who are discharged, a second troponin will be performed at home to identify any missed ACS. A similar approach of home visits performed by ambulatory lab services was successfully applied in our earlier study in suspected ACS in primary care . Obviously, in accordance with daily practice, the attending physician may decide to overrule the recommendation corresponding to a low HEART score and admit a patient. In such a case, information about the reasoning for this escape will be collected. Patients with a HEART score in the intermediate range (4–6), will generally be admitted to the hospital for further observation and investigation. The high-risk group (7–10) will typically receive prompt (invasive) treatments.
Study population and recruitment
All patients above 18 years presenting with chest pain to the (cardiac) ED of 10 participating hospitals are eligible. Only patients presenting with evident ST-segment elevation myocardial infarction (STEMI) will be excluded, since there is no diagnostic dilemma in these patients. Typically, such patients are directly taken to the intervention room. In both study periods, information about the study procedure will be provided by the treating physician and written consent for the use of data and follow-up is obtained at the first appropriate moment after presentation at the ED. In the HEART period, no consent from the patient is needed for the use of the HEART score, for several reasons. First, the number of additional procedures for patients is minimal since the HEART score consists of elements that are collected routinely. Furthermore, the HEART score is proven to be safe, and is a decision support tool rather than a real intervention, with the possibility for physicians to override the recommendations provided by the rule. The study was approved by the Institutional Review Board, and subsequently by the Boards of the participating hospitals.
The HEART impact trial aims to measure both the intended positive changes as well as any unintended negative effects associated with the use of the HEART score. Patient outcomes, use of health care resources and costs will be determined in both periods.
Primary outcome: occurrence of MACE
The primary outcome is the 6-week occurrence of major adverse cardiac events (MACE), consisting of the following events: acute myocardial infarction (AMI), Percutaneous Coronary Intervention (PCI), Coronary Artery Bypass Grafting (CABG) surgery, or death due to any cause. To identify MACE after discharge, a phone-call will be made to all patients at home after 3 months. Any information that could indicate to possible endpoints will be further investigated through hospital charts, hospital discharge letters and information obtained from the patient’s general practitioner (GP). In addition, the Central Bureau for Statistics (CBS) will be consulted for information on vital status as the cause of death of participants. All cases with possible endpoints are reviewed by two independent adjudicators for endpoint classification. This adjudication committee will evaluate all relevant information to decide, using ESC guidelines, whether MACE occurred. In case of disagreement between two adjudicators, the case is discussed in a plenary adjudication committee meeting until consensus is reached.
Secondary outcomes include the following:
The occurrence of MACE in the specific subgroup of patients with a low HEART score.
Use of health care resources. The number of hospital admissions/discharges, duration of hospital stay, duration of stay on the ED, number of readmissions and GP visits after discharge will be collected.
Health-related quality of life. This will be determined in a subset of approximately 1000 patients, in both time periods in five of the participating hospitals. Data on health-related quality of life are collected at baseline (at ED) using the EuroQol Five-Dimensional (EQ-5D) questionnaire, and a 2-week and at 3-month follow-up using the short form-36 (SF-36) and the EQ-5D questionnaires. SF-36 is a short-form health survey with only 36 questions. For this study, we will only use the 11 questions addressing Health. EQ-5D comprises the following 5 dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. Furthermore, the EQ VAS (visual analogue scale) records the respondent’s self-rated health on a vertical, visual analogue scale where the endpoints are labelled ‘Best imaginable health state’ and ‘Worst imaginable health state’. Higher scores are associated with a better health-related quality of life.
Direct and indirect costs. These will be determined in a subset of approximately 1000 patients, in both time periods in five of the participating hospitals. Actual medical costs using a health-care provider’s perspective are obtained in both the time periods. Medical resource use is extracted from the electronic hospital patient files. Unit cost prices will be determined in two participating hospitals, one academic and one peripheral hospital, using micro-costing if possible and top down costing otherwise. The iPCQ (Productivity Cost Questionnaire) will be used to collect quantitative data on the relation between illness, treatment and work performance. The iPCQ is divided into 3 modules: (1) reduced productivity at paid work due to work absenteeism, (2) reduced productivity at paid work without absence from work and (3) unpaid labour production.
An additional sample of blood will be collected from patients of two participating hospitals during the emergency visit for future biomarkers investigation (Biobanking).
The 6-week cumulative incidence of MACE in both the intervention and usual care period will be analysed at the patient level using a generalised linear mixed model (GLMM) . Risk differences with corresponding 95% CIs will be estimated from this model. No baseline differences in prognostic factors between patients included in both periods are expected, but in case these do occur, covariates will be added to the GLMM model to adjust for these baseline differences. Differences in health-related quality of life at baseline, 2 weeks and at 3 months will also be assessed, separately for the different questionnaires. Costs per patient will be calculated according to Dutch guidelines for pharmaco-economic analyses , and costs of drugs prescribed will be based on Dutch formulary cost-prices.
The aim of the HEART study is to evaluate whether the use of the HEART score streamlines the further management of chest pain patients, in particular whether it can identify low-risk patients who can be discharged sooner than usual. However, these benefits become only relevant if the use of the HEART score does not lead to an increase in adverse cardiac events. Our sample size calculation is therefore based on demonstrating that proportion of patients with MACE is not inferior to the proportion observed with usual care. The proportion MACE expected during usual care is 17%. The non-inferiority margin is based on clinical judgement and available literature as 3%, thus accepting an upper limit of the 95% confidence interval (CI) during the intervention period of 20%. With 10 hospitals, inclusion of 60 patients per hospital per month, a between-hospital variation in incidence of 16 to 18%, a one-sided alpha of 5% and a power of 80%, 6600 patients with chest pain should be included in total. Taken into account our inclusion rates of previous validation studies and with special attention and encouragement for inclusion, we expect a realistic inclusion rate of 60 patients per hospital per month.
A cost-effectiveness analysis will be performed from a societal perspective, for a 3-month and a life-time time horizon. The 3-month time horizon corresponds to the actual follow-up period and will consider the observed differences in costs and quality of life. A GLMM will be used to assess cost-effectiveness, accounting for the randomisation of clusters instead of patients. Uncertainty will be addressed through the GLMM model which will be extended with cluster and patient-level covariates if baseline characteristics are imbalanced. The life-time horizon will be applied to account for long-term costs and effects of the observed MACE. Here, the observed risks of MACE, as well as the direct treatment cost and productivity losses estimated using the friction cost approach, will serve as input for a Markov decision-analytic model . If necessary, additional evidence on long-term costs and effects of adverse events will be obtained from the literature. Monte Carlo simulation will be applied to simulate the course of hypothetical patients through the model, and to estimate the number of quality-adjusted life years and costs of both strategies. Costs will be discounted with 4% per annum, and effects with 1.5% per annum, according to Dutch guidelines. The incremental cost-effectiveness ratio and the net monetary benefit (for various willingness to pay thresholds) will be estimated for the HEART score compared with usual care. Uncertainty will be assessed with probabilistic sensitivity analysis , and results will be presented in incremental cost-effectiveness planes and cost-effectiveness acceptability curves.
This study will be conducted according to the principles of the current version of the declaration of Helsinki and in accordance with the Dutch law on Medical Research Involving Human Subjects Act (WMO).
Ethics committee approval
The study was approved by the Institutional Review Board (medical ethical committee of the University Medical Center Utrecht, the Netherlands), and subsequently by the Boards of the participating hospitals.
Overview of key characteristics of the stepped wedge design
Stepped wedge design has features of cluster randomisation, i.e. during a specific time period only type of intervention (usual care or HEART score) is administered
This reduces the risk of contamination
The effect of clustering needs to be taken into account in the statistical analysis
Stepped wedge design has features of a one direction cross-over trial, i.e. each hospital contributes data from both usual care and HEART score in a fixed order.
Allows for comparison of results within hospitals which may be less confounded by differences in case mix than between hospitals
The fixed order from usual care to HEART score further reduces the risk of contamination as the HEART score is relatively simple to calculate.
Due to the cross-over, each hospital will provide data about the (problems in) implementation of the HEART score
Switch from usual care to HEART score in hospitals is evenly and randomly distributed over calendar time
This reduces the impact of potential changes over time in other factors than the intervention
It facilitates the close monitoring and logistic of all activities surrounding the switch
Gradual implementation of new strategy is carried out, thereby providing data about the process itself.
It is of importance to generate valid evidence that the use of the HEART score compared to usual care is safe and leads to fewer admissions and diagnostic procedures in real-life clinical practice. Using the stepped wedge design, we can also monitor the process of implementation of a clinical support tool at the ED across hospitals that vary in size and population. Patient inclusion has started July 1st of 2013.
We thank all participating hospitals, especially all (cardiac) residents, nurses, secretaries, R&D departments and local laboratories. Research grant was obtained from the “Netherlands Organization for Health Research and Development” (ZonMw grant Project number 171202015), as part of the Effectiveness Program (Doelmatigheidsprogramma ZonMw).
- Lee TH, Goldman L: Evaluation of the patient with acute chest pain. N Engl J Med. 2000, 342: 1187-1195. 10.1056/NEJM200004203421607.View ArticlePubMedGoogle Scholar
- Swap CJ, Nagurney JT: Value and limitations of chest pain history in the evaluation of patients with suspected acute coronary syndromes. JAMA. 2005, 294: 2623-2629. 10.1001/jama.294.20.2623.View ArticlePubMedGoogle Scholar
- Miller CD, Lindsell CJ, Khandelwal S, Chandra A, Pollack CV, Tiffany BR, Hollander JE, Gibler WB, Hoekstra JW: Is the initial diagnostic impression of “noncardiac chest pain” adequate to exclude cardiac disease?. Ann Emerg Med. 2004, 44: 565-574. 10.1016/j.annemergmed.2004.03.021.View ArticlePubMedGoogle Scholar
- Moons KG, Altman DG, Vergouwe Y, Royston P: Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009, 338: b606-10.1136/bmj.b606.View ArticlePubMedGoogle Scholar
- Anderson JL, Adams CD, Antman EM, Bridges CR, Califf RM, Casey DE, Chavey WE, Fesmire FM, Hochman JS, Levin TN, Lincoff AM, Peterson ED, Theroux P, Wenger NK, Wright RS, Smith SC, Jacobs AK, Adams CD, Anderson JL, Antman EM, Halperin JL, Hunt SA, Krumholz HM, Kushner FG, Lytle BW, Nishimura R, Ornato JP, Page RL, Riegel B: ACC/AHA 2007 guidelines for the management of patients with unstable angina/non-ST-Elevation myocardial infarction. J Am Coll Cardiol. 2007, 50: e1-157.View ArticlePubMedGoogle Scholar
- Hamm CW, Bassand JP, Agewall S, Bax J, Boersma E, Bueno H, Caso P, Dudek D, Gielen S, Huber K, Ohman M, Petrie MC, Sonntag F, Uva MS, Storey RF, Wijns W, Zahger D: ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J. 2011, 32 (23): 2999-3054.View ArticlePubMedGoogle Scholar
- Fox KA, Dabbous OH, Goldberg RJ, Pieper KS, Eagle KA, Van de Werf F, Avezum A, Goodman SG, Flather MD, Anderson FA, Granger CB: Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: prospective multinational observational study (GRACE). BMJ. 2006, 333: 1091-10.1136/bmj.38985.646481.55.View ArticlePubMedPubMed CentralGoogle Scholar
- Antman EM, Cohen M, Bernink PJ, McCabe CH, Horacek T, Papuchis G, Mautner B, Corbalan R, Radley D, Braunwald E: The TIMI risk score for unstable angina/ non-ST elevation MI. JAMA. 2000, 284: 835-842. 10.1001/jama.284.7.835.View ArticlePubMedGoogle Scholar
- de Araújo GP, Ferreira J, Aguiar C, Seabra-Gomes R: TIMI, PURSUIT, and GRACE risk scores: sustained prognostic value and interaction with revascularization in NSTE-ACS. Eur Heart J. 2005, 26: 865-872. 10.1093/eurheartj/ehi187.View ArticleGoogle Scholar
- Apgar V: A proposal for a new method of evaluation of the newborn infant. Curr Res Anesth Analg. 1953, 32 (4): 260-7.PubMedGoogle Scholar
- Six AJ, Backus BE, Kelder JC: Chest pain in the emergency room: value of the HEART score. Neth Heart J. 2008, 16: 191-6. 10.1007/BF03086144.View ArticlePubMedPubMed CentralGoogle Scholar
- Backus BE, Six AJ, Kelder JC, Mast TP, van den Akker F, Mast EG, Monnink SH, van Tooren RM, Doevendans PA: Chest pain in the emergency room; a multicenter validation of the HEART score. Crit Path Cardiology. 2010, 9: 164-169. 10.1097/HPC.0b013e3181ec36d8.View ArticleGoogle Scholar
- Backus BE, Six AJ, Kelder JC, Bosschaert MAR, Mast EG, Mosterd A, Veldkamp RF, Wardeh AJ, Tio R, Braam R, Monnink SHJ, van Tooren R, Mast TP, van den Akker F, Cramer MJM, Poldervaart JM, Hoes AW, Doevendans PA: A prospective validation of the HEART score for chest pain patients at the emergency department. Int J Cardiol. 2013, S0167-5273. doi: 10.1016/j.ijcard.2013.01.255Google Scholar
- Six AJ, Cullen L, Backus BE, Greenslade J, Parsonage W, Aldous S, Doevendans PA, Than M: The HEART score for the assessment of patients with chest pain in the emergency department: a multinational validation study. Crit Path Car. 2013, 12: 121-126.View ArticleGoogle Scholar
- Pope JH, Aufderheide TP, Ruthazer R, Woolard RH, Feldman JA, Beshansky JR, Griffith JL, Selker HP: Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med. 2000, 342: 1163-1170. 10.1056/NEJM200004203421603.View ArticlePubMedGoogle Scholar
- Schull MJ, Vermeulen MJ, Stukel TA: The risk of missed diagnosis of acute myocardial infarction associated with emergency department volume. Ann Emerg Med. 2006, 48: 647-55. 10.1016/j.annemergmed.2006.03.025.View ArticlePubMedGoogle Scholar
- Brown CA, Lilford RJ: The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006, 6: 54-10.1186/1471-2288-6-54.View ArticlePubMedPubMed CentralGoogle Scholar
- Brown C, Hofer T, Johal A, Thomson R, Nicholl J, Franklin BD, Lilford RJ: An epistemology of patient safety research: a framework for study design and interpretation: part 2. study design. Qual Saf Health Care. 2008, 17: 163-10.1136/qshc.2007.023648.View ArticlePubMedGoogle Scholar
- Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, Woodward M: Risk prediction models: II: external validation, model updating, and impact assessment. Heart. 2012, 98 (9): 691-8. 10.1136/heartjnl-2011-301247.View ArticlePubMedGoogle Scholar
- Bruins Slot MH, van der Heijden GJ, Rutten FH, van der Spoel OP, Mast EG, Bredero AC, Doevendans PA, Glatz JF, Hoes AW: Heart-type fatty acid-binding protein in acute myocardial infarction evaluation (FAME): background and design of a diagnostic study in primary care. BMC Cardiovasc Disord. 2008, 8: 8-10.1186/1471-2261-8-8.View ArticlePubMedPubMed CentralGoogle Scholar
- Molenberghs G, Verbeke G: Models for discrete longitudinal data. 2005, Springer; New YorkGoogle Scholar
- Richtlijnen voor farmaco-economisch onderzoek – geactualiseerde versie 2006 [Guidelines for pharmaco-economic research]. 2006, Diemen: College voor Zorgverzekeringen (CvZ) [Health Insurance Board]Google Scholar
- Oostenbrink JB, Bouwmans CAM, Koopmanschap MA, Rutten FFH: Handleiding voor kostenonderzoek, methoden en standaard kostprijzen voor economische evaluaties in de gezondheidszorg - geactualiseerde versie 2004. 2004, Diemen: College voor Zorgverzekeringen (CvZ) [Health Insurance Board]Google Scholar
- Briggs AH: Handling uncertainty in cost-effectiveness models. Pharmacoeconomics. 2000, 17: 479-500. 10.2165/00019053-200017050-00006.View ArticlePubMedGoogle Scholar
- Reilly BM, Evans AT: Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006, 144: 201e9-View ArticleGoogle Scholar
- Harrell FE, Lee KL, Mark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996, 15: 361e87-View ArticleGoogle Scholar
- Justice AC, Covinsky KE, Berlin JA: Assessing the generalizability of prognostic information. Ann Intern Med. 1999, 130: 515e24-View ArticleGoogle Scholar
- Mdege ND, Man MS, Taylor Nee Brown CA, Torgerson DJ: Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011, 64 (9): 936-48. 10.1016/j.jclinepi.2010.12.003.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2261/13/77/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.