A new scoring system to stratify risk in unstable angina
© Piombo et al; licensee BioMed Central Ltd. 2003
Received: 23 December 2002
Accepted: 20 August 2003
Published: 20 August 2003
We performed this study to develop a new scoring system to stratify different levels of risk in patients admitted to hospital with a diagnosis of unstable angina (UA), which is a complex syndrome that encompasses different outcomes. Many prognostic variables have been described but few efforts have been made to group them in order to enhance their individual predictive power.
In a first phase, 473 patients were prospectively analyzed to determine which factors were significantly associated with the in-hospital occurrence of refractory ischemia, acute myocardial infarction (AMI) or death. A risk score ranging from 0 to 10 points was developed using a multivariate analysis. In a second phase, such score was validated in a new sample of 242 patients and it was finally applied to the entire population (n = 715).
ST-segment deviation on the electrocardiogram, age ≥ 70 years, previous bypass surgery and troponin T ≥ 0.1 ng/mL were found as independent prognostic variables. A clear distinction was shown among categories of low, intermediate and high risk, defined according to the risk score. The incidence of the triple end-point was 6 %, 19.2 % and 44.7 % respectively, and the figures for AMI or death were 2 %, 11.4 % and 27.6 % respectively (p < 0.001).
This new scoring system is simple and easy to achieve. It allows a very good stratification of risk in patients having a clinical diagnosis of UA. They may be divided in three categories, which could be of help in the decision-making process.
Unstable angina (UA) is a complex syndrome with many different clinical presentations which share a common pathophysiologic background [1, 2]. Plaque rupture or erosion, platelet activation, coronary spasm, thrombosis and oxygen supply/demand imbalance are well known mechanisms responsible for the diverse manifestations of the disease . Prognosis of patients admitted to coronary care units with the clinical diagnosis of UA has strikingly improved in the last decades, but the spectrum of outcomes among different patients continues to be broad. There is general agreement that risk stratification is mandatory in this population and many markers of increased risk of serious events have been described over time [4–10].
Refractory angina seems to be the strongest predictor of acute myocardial infarction or death, but this marker is not available at admission, preventing an early assessment of risk . Although several clinical, electrocardiographic and biochemical factors have been clearly shown to increase risk in UA, few attempts have been made to combine them in order to improve their individual prognostic accuracy [12, 13].
We decided to test the prognostic value of a combination of such markers resulting in a prospectively designed score that could be capable of making a clear distinction of different clinical outcomes applied to patients coming to hospital with an UA admission diagnosis. With that purpose we chose the most widely available prognostic variables that, in our model, provided the best independent information for the occurrence of major in-hospital events. The new score was applied in another cohort of patients consecutively admitted to several coronary care units who were not enrolled in trials of therapeutic interventions.
Between January 2000 and June 2001, patients admitted to coronary care units with a clinical diagnosis of UA were included in the study if they fulfilled the following criteria: a) class III-IV angina beginning in the last 2 months (new onset angina) or previous stable angina increasing in frequency, duration of pain or occurring at lower threshold (progressive angina); b) last episode of pain at rest or at minimal exertion occurring in the previous 48 hours and lasting more than 10 minutes.
Exclusion criteria were: a) Braunwald class A (secondary angina) or class C (postinfarction angina); b) acute myocardial infarction (AMI) defined as the elevation of creatine kinase at least twice the upper limit of normal values and a creatine kinase-MB fraction higher than 5 % of the total creatine kinase value within the first 8 hours from the onset of the last episode of ischemic pain; c) left bundle branch block.
Electrocardiographic (ECG) changes were evaluated using the admission ECG recordings. ST segment deviation was defined as 1 mm or more elevation or depression of the ST segment measured at 0.08 sec from the J point in at least 2 contiguous leads.
Ten coronary care units participated in the study. Seven of them had catheterization facilities on site. The protocol was approved by the local ethics committees at each participating center.
Cardiac-specific troponin T was measured using a rapid bedside assay where blood reacts with monoclonal antibodies, with a minimal detection level of 0.1 ng/mL .
Determination of C-reactive protein: blood samples were stored in evacuated tubes (BD Biosciences) at -80°C to be processed by immunoturbidimetric assay (Tina-quant CRP; Roche) in a central laboratory where all the samples were sent. The C-reactive protein detection range corresponds to values of 0.1 to 48 mg/dL, with an interassay variation coefficient of <5%. Measurements were calibrated against CRM 470 standards.
Blood samples for both determinations were taken at least eight hours from the onset of the last episode of ischemic pain.
The double end-point consisted of in-hospital death or AMI, defined as the presence of two of the following three criteria: prolonged ischemic pain (>20 minutes), new Q waves development in 2 or more contiguous leads, and creatine kinase doubling the upper normal value with MB fraction >5 % of total creatine kinase value. The triple end-point included in-hospital death, AMI or refractory angina, which was defined as one or more symptomatic ischemic episodes with ST-T changes on the ECG or hemodynamic instability, lasting at least 10 minutes being the patient treated with aspirin, nitrates, beta blockers or calcium antagonists and heparin (either unfractionated or low-molecular weight) in adequate doses, unless contraindicated.
Data management and statistical analysis
Continuous variables are presented as mean ± SD. In order to develop a risk score, all demographic, clinical, ECG and biochemical variables routinely collected at admission to coronary care units were entered into an univariate analysis and related to the triple end-point. Univariate comparisons were made using the chi-square test for categorical variables and the t-test for continuous ones. Every variable resulting in a p value <0.10 for the triple end-point was entered into a multiple logistic regression analysis to determine which were independently related to the end-points. All statistical comparisons were two-tailed, and a value of p < 0.05 was considered as significant.
The predictive accuracy of the multivariate model was evaluated using the C statistic, an index that reflects the area under the receiver operating characteristic curve.
The odds ratio (OR) values obtained in the multivariate analysis were used to develop the scoring system in the following way: if the OR was between 1 and 1.9, one point was adjudicated; two points if it was between 2 and 2.9; three points between 3 and 3.9 and four points if it exceeded the last value.
Once the risk score was developed we conducted a validation phase to assess its prognostic accuracy in a prospectively collected new sample of patients. Finally, we applied the score to the entire population (from the derivation and validation sets) to assess its value in a larger group of patients with unstable angina.
The overall predictive ability of the risk score was then assessed with the C statistic and compared with that obtained from the multivariate model of the development phase.
Score development phase
Development phase [n:473]
Validation phase [n:242]
Age [mean ± SD]
63.5 ± 11.8
63.7 ± 11.2
Age ≥ 70 y
Recent onset angina
Medication before admission
T wave inversion
Positive troponin test
CRP ≥ 3 mg/L
CRP ≥ 10 mg/L
Results of the univariate analysis
Age (mean ± SD)
68.8 ± 11.3
62.7 ± 11.6
Age ≥ 70 y
1.66 – 5.53
0.40 – 1.33
0.40 – 1.75
0.55 – 2.42
0.29 – 1.04
0.68 – 2.28
0.87 – 2.93
0.45 – 2.58
1.39 – 5.87
Medication before admission
1.14 – 4.45
0.77 – 2.52
1.02 – 3.7
0.46 – 1.79
T wave inversion
2.74 – 9.55
Positive troponin test
1.43 – 4.93
C R P ≥ 3 mg/L
0.75 – 2.39
C R P ≥ 10 mg/L
0.86 – 2.83
Age was a statistically significant factor either used as a continuous (C statistic: 0.65) or as a categorical variable (C statistic: 0.64). The best cut-off point using the receiver operating curve was 70 years. Thus, age ≥ 70 years was chosen for inclusion in the multivariate analysis.
Results of the multivariate analysis.
ST segment deviation
2.21 – 7.36
Age ≥ 70 y
1.26 – 4.16
1.07 – 4.57
Positive troponin test
1.08 – 3.70
Therefore, according to the OR obtained, the scoring system was established as follows: ST deviation (OR = 4.03): 4 points; age ≥ 70 years (OR = 2.29): 2 points; previous bypass surgery (OR = 2.21): 2 points, and positive troponin test (OR = 2): 2 points. As the highest possible score was 10 points, we divided it in tertiles so that we could assign each patient to one of three categories according to the score sum value: low-risk when it was 0 or 2, intermediate-risk when it was 4 or 6 and high-risk when it was 8 or 10.
Fourteen patients (3 %) died, 20 (4.2 %) had an AMI and 33 (7 %) developed refractory angina during in-hospital evolution. The incidence of the triple end-point was 12.3 % and the corresponding figure for the double end-point was 6.6 %.
Outcomes according to risk categorization in the validation phase.
Low-risk [n = 153]
Intermediate-risk [n = 74]
High-risk [n = 15]
16.2 % *
46.6 % §†
8.1 % #
33.3 % ** †
Risk stratification in the entire population
When the already validated scoring system was applied to the entire population (development plus validation phases; n = 715), striking differences were found among the low, intermediate and high risk subsets. Low risk patients comprised 62.8 % of the whole population (n = 449); intermediate risk, 30.6 % (n = 219) and high risk, 6.6 % (n = 47) (Figure 1). Coronary angiography was performed in 47.7 % of the population and revascularization procedures in 27.8 %.
The triple end-point occurred in 6 % of low risk patients, 19.2 % of intermediate risk and 44.7 % of high risk patients [OR for high vs. low risk: 12.6, 95 % CI 5.9 to 26.8, p < 0.001; OR for high vs. intermediate risk: 3.4, 95 % CI 1.6 to 6.9, p < 0.001; OR for intermediate vs. low risk: 3.7, 95 % CI 2.1 to 6.4, p < 0.001].
Outcomes according to risk categorization in the entire population
Low risk n = 449
Intermediate risk n = 219
High risk n = 47
19,2 % *
44,7 % * §
11,4 % *
27,6 % * †
Predictive power of the score, as assessed by the C statistic, was 0.72 (95% CI: 0.66–0.78), similar to that found for the multivariate model.
Although many clinical, electrocardiographic and biochemical markers have been clearly shown to correlate with short and long-term prognosis in UA, few efforts have been made to group them in order to improve their individual predictive power.
The scoring system proposed here is quite simple to obtain and has a good ability to discriminate risk according to the C-statistic value. All the information needed is available at admission or just a few hours after the last episode of pain, it is non-expensive and, what is most important, it has a very good prognostic value. We divided the population studied into three groups: low, intermediate and high-risk, which is a common practice among cardiologists regarding this or other diseases. The probability of developing serious events, not only refractory angina but also acute myocardial infarction or death, is enhanced about ten times from the low to the high risk group.
The TIMI group has reported an interesting proposal of a risk score for UA and non-ST elevation myocardial infarction . They found a significant relationship between the score ranging from 0 to 7 points and outcome. However, some important limitations to this study must be pointed out. It was based on a population derived from trials of therapeutic interventions which had stringent inclusion and exclusion criteria preventing the application of the results to a general population of this kind of patients. Serum cardiac markers were not well defined, so that different markers were used in different patients. Finally, the inclusion of an angiographic variable may not be practical as many patients with UA do not have a previous coronary angiography performed. Therefore, although it seems difficult to make a direct comparison between the TIMI risk score and ours, we think that our proposal may have a similar accuracy to predict risk in the whole population with UA and may be simpler to obtain at admission in the clinical practice. The smaller size of our cohort implies that these results should be reproduced in a larger one in order to make definite comparisons with other scoring systems.
A recently published study coming from Spain, the PEPA registry, deserves some comments. The authors performed a multicenter registry with 4115 unselected patients with suspected non-ST-segment elevation acute coronary syndromes. They found that age, diabetes, peripheral vascular disease, postinfarction angina, Killip class, ECG and cardiac markers were independent predictors of death. They also developed a risk score based on such variables that showed an interesting predictive power. Ninety days mortality ranged from 0.4 % when no risk factor was present to 21 % for patients with more than 4 risk factors .
Since our study is not an interventional trial, inclusion biases are avoided. We did not include postinfarction angina and non-ST elevation myocardial infarctions since biochemical markers (troponins and C-reactive protein) would not be useful in these entities as they should always be elevated in patients with recent myocardial necrosis because of their high sensitivity. Therefore, results apply only to patients with a clinical diagnosis of UA. We must remark that the traditional definition of AMI was used in this study. If the new definition proposed by the ESC / ACC Committee had been used, some patients would be categorized as having an evolving AMI at admission, since 21 % of them had an elevated troponin level .
We chose a qualitative analysis of troponin T because it can be performed bedside, it is not expensive and is widely available in coronary care units from our country and elsewhere . The cut-off value of 0.1 ng/mL has been validated as an useful predictor of major events in UA [7, 18], including a recently published large trial which showed that a positive test doubles the risk of death or AMI . Both troponin T and troponin I have been consistently shown to be powerful prognostic variables in patients with UA [20–23]. As a rather high cut-off level was selected for our trial, it is possible that some "not-low risk" patients were not detected.
C-reactive protein had, in our experience, no predictive value for in-hospital events. This is in agreement with several reports, but may be related to the cut-off values we chose. In fact, there is no agreement about which should be the best cut-off point for C-reactive protein values in order to use it in a risk score. However, although this marker does not seem to be a good short-term prognostic variable, it could be useful for the long-term outcome [7, 24–26]. Although the kind of assay (immunoturbidimetric) performed in this study has been extensively used, it is possible that with newer techniques (high-sensitivity assay) results could be somewhat different.
The high prevalence of patients categorized as low-risk is noteworthy. This figure reflects the non-selected population included in this study. Many low risk patients are excluded from trials of therapeutic interventions as they generally have very stringent entry criteria. We believe our sample is representative of the "real world" patients admitted in our country to coronary care units with UA. The overall outcome of our patients is quite similar to that reported in the OASIS Registry for the short-term follow-up, which included 7987 patients from six different countries, 87 % with UA as admission diagnosis . The very low incidence of serious events (refractory ischemia, infarction or death) seen in the low risk group confirms the risk category in which they were included based on our score.
In the same way, the quite high C statistic reported here may be explained by the non selected population with higher prognostic value of different variables as it has been shown regarding troponin T as a prognostic factor in cohort studies versus clinical trials .
On the other hand, there is a minority of patients classified as high-risk, but they actually have a very poor outcome. The intermediate-risk group is almost in the middle of the other two when in-hospital events are considered. Maybe some factors not included in the score should be used in this particular subset in order to define their prognosis more precisely. We cannot rule out the possibility that the relatively small sample size could have limited the number of variables that showed statistical significance in the multivariate analysis performed.
Our findings should be tested in a larger cohort of patients in order to suggest clinical strategies based on them. If these data were confirmed, a highly aggressive approach should be recommended in high-risk patients and a more conservative one could be reserved for the low-risk group, as has been recently suggested by Solomon et al.  working on the TIMI IIIB study population. Patients categorized as low or very low risk, which comprised more than half of the population, did not show any benefit from an invasive treatment strategy, as did patients categorized as high or very high risk.
The syndrome of UA comprises highly different patients and, consequently, a wide spectrum of clinical outcomes. There is general agreement that a risk stratification strategy is necessary in order to provide the best treatment to each patient. We developed a simple scoring system, easy to obtain at any institution, which showed a good capability to separate patients with UA into low, intermediate and high risk groups. The score proposed here was entirely developed and validated in a prospective fashion, on patients admitted to coronary care units without inclusion biases. Further prospective studies with larger populations are needed to confirm the practical use of this scoring system.
Participating Centers And Investigators
Argerich Hospital: G. Marambio, G. Cestari, A. Piombo.
C.E.M.I.C.: J. Fuselli, J. Guetta, C. Boissonnet.
Fernandez Hospital: K. Crotto, S. Salzberg.
Clínica Bazterrica: C. Barrero, E. Fairman.
Clínica del Sol: J.A. Gagliardi.
I.C.B.A.: A. Alves de Lima, R. Guglielmone.
Clínica Suizo-Argentina: M. Bruno, C. Bruno.
Policlínico Bancario: A. Larraburu, G. Coqui.
Centro Gallego: A. Marinesco.
Rivadavia Hospital: E. Dominé, A. Hirschson.
Statistical analysis: Juan Gagliardi.
We want to thank Bioch. Viviana Correa and Silvia Quiroga for their kind cooperation with the biochemical determinations and the assistance provided by René Baron Research Unit.
- Braunwald E: Unstable angina: a classification. Circulation. 1989, 80: 410-414.View ArticlePubMedGoogle Scholar
- van Miltenburg-van Zijl A, Simoons M, Veerhoek R, et al: Incidence and follow-up of Braunwald subgroups in unstable angina pectoris. J Am Coll Cardiol. 1995, 25: 1286-1292. 10.1016/0735-1097(95)00009-S.View ArticlePubMedGoogle Scholar
- Braunwald E: Unstable angina. An etiologic approach to treatment. Circulation. 1998, 98: 2219-2222.View ArticlePubMedGoogle Scholar
- Langer A, Freeman M, Armstrong P: ST segment shift in unstable angina; pathophysiology and association coronary anatomy and hospital outcome. J Am Coll Cardiol. 1989, 13: 1495-1502.View ArticlePubMedGoogle Scholar
- Liuzzo G, Biasucci L, Gallimore J, et al: The prognostic value of C-reactive protein and serum amyloid A protein in severe unstable angina. N Engl J Med. 1994, 331: 417-424. 10.1056/NEJM199408183310701.View ArticlePubMedGoogle Scholar
- Holmvang L, Clemmensen P, Wagner G, et al: Admission standard electrocardiogram for early risk stratification in patients with unstable coronary artery disease not eligible for acute revascularization therapy: a TRIM substudy. Am Heart J. 1999, 137: 24-33.View ArticlePubMedGoogle Scholar
- Heeschen C, Hamm C, Bruemmer J, et al: Predictive value of C-reactive protein and troponin T in patients with unstable angina: a comparative analysis. J Am Coll Cardiol. 2000, 35: 1535-1542. 10.1016/S0735-1097(00)00581-7.View ArticlePubMedGoogle Scholar
- Newby L, Christenson R, Ohman M, et al: Value of serial troponin T measures for early and late risk stratification in patients with acute coronary syndromes. The GUSTO-IIa Investigators. Circulation. 1998, 98: 1853-1859.View ArticlePubMedGoogle Scholar
- Lindhal B, Venge P, Wallentin L, for the FRISC Study Group: Relation between troponin T and the risk of subsequent cardiac events in unstable coronary artery disease. Circulation. 1996, 93: 1651-1657.View ArticleGoogle Scholar
- Toss H, Lindahl B, Siegbahn A, for the FRISC study group, et al: Prognostic influence of increased fibrinogen and C-reactive protein levels in unstable coronary artery disease. Circulation. 1997, 96: 4204-4210.View ArticlePubMedGoogle Scholar
- Armstrong P, Fu Y, Chang W, for the GUSTO-II b Investigators, et al: Acute coronary syndromes in the GUSTO-IIb trial. Prognostic insights and impact of recurrent ischemia. Circulation. 1998, 98: 1860-1868.View ArticlePubMedGoogle Scholar
- Holmvang L, Lüscher M, Clemmensen P, et al: Very early risk stratification using combined ECG and biochemical assessment in patients with unstable coronary artery disease (A Thrombin Inhibition in Myocardial Ischemia [TRIM] substudy). Circulation. 1998, 98: 2004-2009.View ArticlePubMedGoogle Scholar
- Antman E, Cohen M, Bernink P, et al: The TIMI risk score for unstable angina / non-ST elevation MI. A method for prognostication and therapeutic decision making. JAMA. 2000, 284: 835-842. 10.1001/jama.284.7.835.View ArticlePubMedGoogle Scholar
- Muller-Bardorff M, Freitag H, Scheffold T, et al: Development and characterization of a rapid assay for bedside determinations of cardiac troponin T. Circulation. 1995, 92: 2869-2875.View ArticlePubMedGoogle Scholar
- López de Sá E, López-Sendón J, Anguera I, et al: Prognostic value of clinical variables at presentation in patients with non-ST-segment elevation acute coronary syndromes: results of the Proyecto de Estudio del Pronóstico de la Angina (PEPA). Medicine (Baltimore). 2002, 81: 434-442. 10.1097/00005792-200211000-00004.View ArticleGoogle Scholar
- The Joint European Society of Cardiology/American College of Cardiology Committee: Myocardial infarction redefined-a consensus document of the Joint European Society of Cardiology/American College of Cardiology Committee for the redefinition of myocardial infarction. J Am Coll Cardiol. 2000, 36: 959-969. 10.1016/S0735-1097(00)00804-4.View ArticleGoogle Scholar
- van Domburg R, Cobbaert C, Kimman G, et al: Long-term prognostic value of serial troponin T bedside tests in patients with acute coronary syndromes. Am J Cardiol. 2000, 86: 623-627. 10.1016/S0002-9149(00)01040-7.View ArticlePubMedGoogle Scholar
- Ohman M, Armstrong P, Christenson R, for the GUSTO II-a Investigators, et al: Cardiac troponin T levels for risk stratification in acute myocardial ischemia. N Engl J Med. 1996, 335: 1333-1341. 10.1056/NEJM199610313351801.View ArticlePubMedGoogle Scholar
- The GUSTO IV-ACS Investigators: Effect of glycoprotein IIb/IIIa receptor blocker abciximab on outcome in patients with acute coronary syndromes without early coronary revascularization: the GUSTO IV-ACS randomised trial. Lancet. 2001, 357: 1915-1924. 10.1016/S0140-6736(00)05060-1.View ArticleGoogle Scholar
- Rebuzzi A, Quaranta G, Liuzzo G, et al: Incremental prognostic value of serum levels of troponin T and C-reactive protein on admission in patients with unstable angina pectoris. Am J Cardiol. 1998, 82: 715-719. 10.1016/S0002-9149(98)00458-5.View ArticlePubMedGoogle Scholar
- Antman E, Sacks D, Rifai N, et al: Time to positivity of a rapid bedside assay for cardiac-specific troponin T predicts prognosis in acute coronary syndromes: a Thrombolysis in Myocardial Infarction [TIMI] 11A substudy. J Am Coll Cardiol. 1998, 31: 326-330. 10.1016/S0735-1097(97)00485-3.View ArticlePubMedGoogle Scholar
- Antman E, Tanasijevic M, Thompson B, et al: Cardiac-specific troponin I levels to predict the risk of mortality in patients with acute coronary syndromes. N Engl J Med. 1996, 335: 1342-1349. 10.1056/NEJM199610313351802.View ArticlePubMedGoogle Scholar
- Lindahl B, Toss H, Siegbahn A, et al: Markers of myocardial damage and inflammation in relation to long-term mortality in unstable coronary artery disease. N Engl J Med. 2000, 343: 1139-1147. 10.1056/NEJM200010193431602.View ArticlePubMedGoogle Scholar
- Morrow D, Rifai N, Antman E, et al: C-reactive protein is a potent predictor of mortality independently of and in combination with troponin T in acute coronary syndromes: a TIMI 11A substudy. J Am Coll Cardiol. 1998, 31: 1460-1465. 10.1016/S0735-1097(98)00136-3.View ArticlePubMedGoogle Scholar
- Biasucci L, Liuzzo G, Grillo R, et al: Elevated levels of C-reactive protein at discharge in patients with unstable angina predict recurrent instability. Circulation. 1999, 99: 855-860.View ArticlePubMedGoogle Scholar
- Ferreirós E, Boissonnet C, Pizarro R, et al: Independent prognostic value of elevated C-reactive protein in unstable angina. Circulation. 1999, 100: 1958-1963.View ArticlePubMedGoogle Scholar
- Kaul P, Fu Y, Chang W, et al: Prognostic value of ST segment depression in acute coronary syndromes: insights from PARAGON-A applied to GUSTO-IIb. J Am Coll Cardiol. 2001, 38: 64-71. 10.1016/S0735-1097(01)01307-9.View ArticlePubMedGoogle Scholar
- Yusuf S, Flather M, Pogue J, et al: Variations between countries in invasive cardiac procedures and outcomes in patients with suspected unstable angina or myocardial infarction without initial ST elevation. Lancet. 1998, 352: 507-514. 10.1016/S0140-6736(97)11162-X.View ArticlePubMedGoogle Scholar
- Heidenreich PA, Alloggiamento T, Melsop K, et al: The prognostic value of troponin in patients with non-ST elevation acute coronary syndromes: a meta-analysis. J Am Coll Cardiol. 2001, 38: 478-485. 10.1016/S0735-1097(01)01388-2.View ArticlePubMedGoogle Scholar
- Solomon D, Stone P, Glynn R, et al: Use of risk stratification to identify patients with unstable angina likeliest to benefit from an invasive versus conservative management strategy. J Am Coll Cardiol. 2001, 38: 969-976. 10.1016/S0735-1097(01)01503-0.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2261/3/8/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.