Heart rate n-variability (HRnV) and its application to risk stratification of chest pain patients in the emergency department

Background Chest pain is one of the most common complaints among patients presenting to the emergency department (ED). Causes of chest pain can be benign or life threatening, making accurate risk stratification a critical issue in the ED. In addition to the use of established clinical scores, prior studies have attempted to create predictive models with heart rate variability (HRV). In this study, we proposed heart rate n-variability (HRnV), an alternative representation of beat-to-beat variation in electrocardiogram (ECG), and investigated its association with major adverse cardiac events (MACE) in ED patients with chest pain. Methods We conducted a retrospective analysis of data collected from the ED of a tertiary hospital in Singapore between September 2010 and July 2015. Patients > 20 years old who presented to the ED with chief complaint of chest pain were conveniently recruited. Five to six-minute single-lead ECGs, demographics, medical history, troponin, and other required variables were collected. We developed the HRnV-Calc software to calculate HRnV parameters. The primary outcome was 30-day MACE, which included all-cause death, acute myocardial infarction, and revascularization. Univariable and multivariable logistic regression analyses were conducted to investigate the association between individual risk factors and the outcome. Receiver operating characteristic (ROC) analysis was performed to compare the HRnV model (based on leave-one-out cross-validation) against other clinical scores in predicting 30-day MACE. Results A total of 795 patients were included in the analysis, of which 247 (31%) had MACE within 30 days. The MACE group was older, with a higher proportion being male patients. Twenty-one conventional HRV and 115 HRnV parameters were calculated. In univariable analysis, eleven HRV and 48 HRnV parameters were significantly associated with 30-day MACE. The multivariable stepwise logistic regression identified 16 predictors that were strongly associated with MACE outcome; these predictors consisted of one HRV, seven HRnV parameters, troponin, ST segment changes, and several other factors. The HRnV model outperformed several clinical scores in the ROC analysis. Conclusions The novel HRnV representation demonstrated its value of augmenting HRV and traditional risk factors in designing a robust risk stratification tool for patients with chest pain in the ED.


Background
Chest pain, which may be caused by life-threatening myocardial infarction (MI) or benign musculoskeletal pain, is one of the most common presenting complaints in the emergency department (ED) [1][2][3]. Majority of chest pain patients are subjected to extensive diagnostic tests to rule out acute coronary syndrome (ACS), resulting in oftentimes, prolonged and costly ED admission, with only a small proportion of these patients eventually receiving a diagnosis of ACS [3]. This can strain crowded EDs and reduce availability of resources for patients who need urgent medical attention. Hence, early identification of chest pain patients who are at high-risk of developing adverse cardiac events has been a pressing issue to contend with in the ED. Several established clinical scores have been used for risk stratifying chest pain patients in the ED [4,5], including the History, ECG, Age, Risk factors and Troponin (HEART) [6], the Thrombolysis in Myocardial Infarction (TIMI) [7], and the Global Registry of Acute Coronary Events (GRACE) [8] scores. Of these scores, the HEART score is the most accurate and widely used [5,[9][10][11][12], with recent studies focusing on the development of risk score-based clinical pathways for rapid, yet safe discharge of low-risk patients [1,3,13,14].
In a recent review of clinical scores for ED patients with chest pain [5], heart rate variability (HRV) has demonstrated its capability in building predictive models for accurate risk stratification [15][16][17]. HRV is a widely adopted tool for evaluating changes in cardiac autonomic regulation, and has been shown to be strongly associated with the autonomic nervous system (ANS) [18][19][20]. HRV analysis characterizes the beat-to-beat variation in an electrocardiogram (ECG) by utilizing time and frequency domains, and nonlinear analyses [19]. Reduced HRV has been found to be a significant predictor of adverse cardiac outcomes [21]. Given the complexity of quantifying HRV representation, several tools such as the PhysioNet Cardiovascular Signal Toolbox [22] and Kubios HRV [23] have been developed to standardize HRV analyses.
Based on the principle of parameter calculation on normal R-R intervals (RRIs; in this paper, RRIs are equivalent to normal-to-normal [NN] intervals, in which abnormal beats have been removed), HRV analysis generates only one set of parameters from a fixed length of ECG record. This limits the amount of information that can be extracted from raw ECG signals. In this paper, we proposed a novel representation of beat-to-beat variation, named as heart rate n-variability (HRnV) [24] to characterize RRIs from a different perspective. With the use of HRnV measures, multiple sets of parameters can be calculated from the same ECG record, which significantly increases the amount of extracted information. Our study is the first clinical application and evaluation of the HRnV representation in risk stratification of chest pain patients in the ED. We hypothesized that HRnV, while closely related to conventional HRV, can provide supplementary information associated with adverse cardiac events. We also investigated the potential use of HRnV parameters to develop risk prediction tools.

Study design and setting
We conducted a retrospective analysis of data collected in our previous study on risk stratification of chest pain patients in the ED [9]. A convenience sample of patients was recruited at the ED of Singapore General Hospital, a tertiary hospital with around-the-clock primary percutaneous coronary intervention capabilities and a median door-to-balloon time of 101 min [25], between September 2010 and July 2015. At ED triage, patients are classified using the Patient Acuity Category Scale (PACS), with PACS 1 patients being the most critically ill and requiring immediate medical attention and PACS 4 patients being non-urgent cases. In this study, patients > 20 years old who presented to the ED with chief complaint of chest pain and with PACS of 1 or 2 were included. Patients were excluded from the study if they had ST-elevation myocardial infarction (STEMI) or an obvious non-cardiac etiology of chest pain diagnosed by the primary emergency physician. Patients were also excluded if their ECGs had high level of noise or if they were in non-sinus rhythm; these criteria were applied to ensure the quality of HRV and HRnV analyses. Ethical approval was obtained from the Centralized Institutional Review Board (CIRB, Ref: 2014/584/C) of SingHealth, the largest public healthcare system in Singapore that includes the Singapore General Hospital as a key partner. Patient consent was waived for this study.

Data collection
During the data collection period, five to six-minute single-lead (lead II) ECG recordings were retrieved from the X-Series Monitor (ZOLL Medical Corporation, Chelmsford, MA). The first set of vital signs and troponin values from the recruited patients were extracted from the hospital's electronic health records (EHR). In this study, high-sensitivity troponin-T was used, and an abnormal value was defined as > 0.03 ng/mL [26]; it was further stratified into three groups and coded as 0 if the value was ≤0.03 ng/mL, 1 if the value was between 1 and 3 times the normal limit, and 2 if the value was > 3 times the normal limit. Additionally, patients' first 12-lead ECGs were interpreted by two independent clinical reviewers. Pathologic ST-elevation, ST-depression, T-wave inversions, and Q-waves were recorded. Patient demographics, medical history, and information required for computing the HEART, TIMI, and GRACE scores were retrospectively reviewed and obtained from EHR.
Proposed HRnV representation of beat-to-beat variation in ECG HR n V: a novel measure with non-overlapping RRIs Prior to introducing the new HR n V measure, we define a new type of RRI called RR n I, where 1 ≤ n ≤ N, and N≪N; N is the total number of RRIs. The definition of RR n I is illustrated in Fig. 1a. When n = 1, RR n I is equivalent to conventional RRI. When n > 1, every n adjacent RRI is connected to form a new sequence of RR n Is. By using Fig. 1 a Illustration of R-R intervals (RRIs) and the definition of RR n I where 1 ≤ n ≤ N and N≪N.N is the total number of RRIs; b Illustration of RRIs and the definition of RR n I m where 1 ≤ n ≤ N, 1 ≤ m ≤ N − 1, and N≪N.N is the total number of RRIs and m indicates the non-overlapping portion between two consecutive RR n I m sequences this strategy, we can create a maximum number of (N − 1) new RR n I sequences from conventional single RRI sequence. With these newly generated RR n I sequences, the calculation of HR n V parameters is straightforward and can be accomplished by applying established quantitative methods including time and frequency domain analyses and nonlinear analysis [18,19]. In describing this new measure, we use the term "HR n V" prior to parameter names to indicate that these parameters are calculated from RR n I sequences. As noted in the above, HR n V is a novel measure based on newly generated, non-overlapping RR n Is. The computed HR n V parameters include but are not limited to the following: the average of RR n Is (HR n V mean NN), standard deviation of RR n Is (HR n V SDNN), square root of the mean squared differences between RR n Is (HR n V RMSSD), the number of times that the absolute difference between two successive RR n Is exceeds 50 ms (HR n V NN50), HR n V NN50 divided by the total number of RR n Is (HR n V pNN50), the integral of the RR n I histogram divided by the height of the histogram (HR n V triangular index), low frequency power (HR n V LF power), high frequency power (HR n V HF power), approximate entropy (HR n V ApEn), sample entropy (HR n V SampEn), and detrended fluctuation analysis (HR n V DFA), among others. Notably, two new parameters NN50n and pNN50n are created, where 50 × n ms is set as the threshold to assess the difference between pairs of consecutive RR n Is. HR n V m : a novel measure with overlapping RRIs Like RR n I that is used in HR n V, to define HR n V m measure, we introduce another type of RRI called RR n I m , where 1 ≤ n ≤ N, 1 ≤ m ≤ N − 1, and N≪N. In the RR n I m sequence, m is used to indicate the level of overlap between consecutive RR n I m sequences. As illustrated in Fig. 1b, (n − m) RRIs form the overlapping portions. When m = n, RR n I m becomes RR n I; therefore, the upper limit of m is N − 1. By controlling the overlap among these newly generated RR n I m sequences, we can create a maximum number of (N × (N − 1)/2) RR n I m sequences (excluding the RR n I sequence) from conventional single RRI sequence. For each of the newly created RR n I m sequences, we apply time and frequency domain analyses, and nonlinear analysis to calculate HR n V m parameters. We add the term "HR n V m " prior to the parameters to denote that they are computed from RR n I m sequences. For example, the average RR n I m intervals and the sample entropy are written as HR n V m mean NN and HR n V m SampEn, respectively. The HR n V m measure extracts more information than HR n V, by adopting a strategy of controlling sequence overlap.
The HRnV-Calc software integrates functions from the PhysioNet Cardiovascular Signal Toolbox [22] to perform standardized ECG signal processing and QRS complex detection. Given the short ECG records in this study, the upper limit of n was set as three; thus, six sets of parameters were calculated, namely HRV, HR 2 V, HR 2 V 1 , HR 3 V, HR 3 V 1 , and HR 3 V 2 .

Clinical outcomes
The primary endpoint in this study was a composite outcome of major adverse cardiac events (MACE) [27], including all-cause death, acute myocardial infarction (AMI), and revascularization (coronary artery bypass graft [CABG] or percutaneous coronary intervention [PCI]) within 30 days of ED presentation.

Statistical analysis
Continuous variables were presented as mean and standard deviation and compared between two categories of the primary outcome (MACE) using two-sample t-test. Categorical variables were presented as frequency and percentage and compared between two categories of the primary outcome (MACE) using chi-square test. A statistically significant difference was defined as p < 0.05. To evaluate the HRnV parameters and other risk factors, we conducted univariable and multivariable analyses and subsequently developed simple prediction models using traditional logistic regression. In building the HRnV prediction model, we selected candidate variables with p < 0.2 in the univariable analysis and fed them into the multivariable stepwise logistic regression. To evaluate the predictive performance, we used leave-one-out cross-validation (LOOCV) to conduct the analysis.
Receiver operating characteristic (ROC) analysis [28] was performed to compare prediction performances among the HRnV model, HEART, TIMI and GRACE scores. The area under the ROC curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were reported as predictive measures. Data preparation, descriptive analysis, and predictive model development were performed in R version 3.6.0 (R Foundation, Vienna, Austria); ROC analysis was conducted in MATLAB R2019a (MathWorks, Natick, MA).

Results
A total of 795 patients were selected from the originally recruited 922 patients [9]. Twenty-eight patients were excluded for ECG recording issues, four were excluded for obvious non-cardiac chest pain, and 95 were excluded for irregular rhythm/artifacts. Among the included 795 patients, 247 (31%) had the primary outcome of 30-day MACE. Table 1 shows patient baseline characteristics. Patients with the primary outcome were older (mean age 61 years vs. 59 years, p = 0.035), with a higher proportion being males (76.1% vs. 64.6%, p = 0.002). There was no statistically significant difference between MACE and non-MACE groups in terms of patient ethnicity. Factors such as history of diabetes and current smoking status were significantly more prevalent in the group with MACE.
Descriptive analyses of HRV and HRnV parameters are tabulated in Table 2. In this clinical case study, N was set as 3, thus HR 2 V, HR 2 V 1 , HR 3 V, HR 3 V 1 and HR 3 V 2 parameters were calculated. Among time domain parameters such as mean NN, SDNN and RMSSD, the HR n V and HR n V m values were generally incremental with an increase in n. Notably, HR 2 V NN50 and HR 3 V NN50 were much lower than conventional HRV NN50. Moreover, NN50n and pNN50n are parameters specifically applicable to the HRnV representation. Like time domain parameters, the same trend of changes in frequency domain parameters were observed. The magnitude of increment in VLF power and LF power was larger than that of HF power with increasing n. One exception, however, was the normalized HF power, where HR n V and HR n V m parameters were smaller than that of HRV. In nonlinear analysis, there were marked differences in Poincaré SD2 values between HRV and HRnV parameters. HR 2 V SampEn and HR 3 V SampEn were considerably larger compared to SampEn parameters of HRV, HR 2 V 1 , HR 3 V 1 , and HR 3 V 2 , as their confidence intervals (CIs) were wide. The wide CI was due to insufficient data points of less than 200 [19], as our ECG recordings were only five to six minutes long. HR 2 V 1 , HR 3 V 1 and HR 3 V 2 were free from this issue as they were calculated from overlapping RR n I m sequences where more data points were available. Table 3 presents the results of univariable analyses of HR n V and HR n V m parameters. Eleven out of 21 conventional HRV parameters were statistically significant. Additionally, 13 HR 2 V, six HR 3 V, 11 HR 2 V 1 , seven HR 3 V 1 and 11 HR 3 V 2 parameters were also significant. Overall, additional 115 HRnV parameters were derived, among which 48 showed statistical significances between patients with 30-day MACE and those without. Among all HRV and HRnV parameters, mean NN, SDNN, RMSSD, NN50, pNN50, HF power, Poincaré SD1 and SD2 were statistically significant in at least five out of six measures (i.e., HRV, HR 2 V, HR 2 V 1 , HR 3 V, HR 3 V 1 , and HR 3 V 2 ). Furthermore, skewness, LF power, SampEn, and ApEn, which did not demonstrate statistical significance in conventional HRV analysis, were statistically significant in HRnV representation. Table 4 presents the results of the multivariable analyses of HR n V and HR n V m parameters by adjusting for age and sex. After adjustment, several parameters such as NN50 of HR 3 V and HR 3 V 2 , and triangular index of HRV, HR 2 V, and HR 3 V 2 , became statistically non-significant, while parameters such as ApEn of HR 2 V, HR 2 V 1 , and HR 3 V 2 became statistically significant. Table 5 lists the 16 variables that were selected through multivariable stepwise logistic regression, among which there were one conventional HRV parameter and seven HRnV parameters. In addition to traditional predictors of adverse cardiac outcomes such as ST segment changes and troponin, HR 2 V ApEn (OR = 0.095; 95% CI 0.014-0.628), HR 2 V 1 ApEn (OR = 19.700; 95% CI 2.942-131.900) and HR 3 V skewness (1.560; 95% CI 1.116-2.181) also demonstrated strong predictive power in assessing the risk of 30-day MACE. Figure 2 shows the ROC curves and Table 6 presents the results of ROC analysis in evaluating the predictive performance of the HRnV model (based on LOOCV), HEART, TIMI, and GRACE scores. The HRnV model achieved the highest AUC value and outperformed HEART, TIMI, and GRACE scores in terms of specificity, PPV, and NPV at the optimal cut-off scores, defined as the points nearest to the upper-left corner of the ROC curves.

Discussion
HRV has generated significant research interest in the past decades [18,19,29], with majority of studies focusing on development of advanced nonlinear techniques to derive novel parameters [30,31]. There is, however, a paucity of research on alternative approaches to analyze RRIs. Vollmer [32] used relative RRIs to describe the relative variation of consecutive RRIs, with which HRV was analyzed. Likewise, we proposed a novel HRnV representation, providing more HRnV parameters than conventional HRV analysis. In this paper, we introduced two measures of HRnV, namely HR n V and HR n V m . HR n V was calculated based on non-overlapping RR n I sequences, while HR n V m was computed from overlapping Table 2 Descriptive analyses of heart rate variability (HRV) and heart rate n-variability (HRnV) parameters HRV Heart rate variability, mean NN Average of R-R intervals, SDNN Standard deviation of R-R intervals, RMSSD Square root of the mean squared differences between R-R intervals; NN50, the number of times that the absolute difference between 2 successive R-R intervals exceeds 50 ms; pNN50, NN50 divided by the total number of R-R intervals; NN50n, the number of times that the absolute difference between 2 successive RR n I/RR n I m sequences exceeds 50 × n ms; pNN50n, NN50n divided by the total number of RR n I/RR n I m sequences; VLF Very low frequency, LF Low frequency, HF High frequency, SD Standard deviation, SampEn Sample entropy, ApEn Approximate entropy, DFA Detrended fluctuation analysis In our clinical case study, we investigated the predictive value of HRnV parameters in assessing the risk of 30-day MACE for chest pain patients in the ED. In addition to 21 HRV parameters, 115 HRnV parameters were derived, of which 48 were found to be statistically significant in their association with the primary outcome. Notably, even with a small n (three in our study), newly generated HRnV parameters greatly boosted the number of candidate predictors. When longer ECG records are available, more HRnV parameters can be calculated. With HRnV parameters, HRV parameters, vital signs, and several established risk factors, we conducted multivariable logistic regression analysis and selected age, diastolic BP, pain score, ST-elevation, STdepression, Q wave, cardiac history, troponin, HRV NN50, and seven HRnV parameters. In addition to traditional risk factors such as ST segment changes, HR 2 V ApEn, HR 2 V 1 ApEn, and HR 3 V skewness were found to be strong predictors for 30-day MACE. Compared to the HEART, TIMI, and GRACE scores, the HRnV model achieved the highest AUC, specificity, PPV, and NPV values at the optimal cut-off points in ROC analysis. This demonstrated the clinical utility of HRnV in determining the risk of 30-day MACE for ED patients with chest pain.
Due to the wide differential diagnosis for chest pain, accurate stratification is vital, particularly for preventing low-risk patients from obtaining expensive and unnecessary medical testing and intervention [3]. Although the TIMI and GRACE scores have been validated for risk prediction of patients with chest pain in the ED [4,33,34], some criteria used in these scores may be inappropriate for undifferentiated chest pain cohorts in the ED, as they were originally developed for post-acute myocardial infarction patients [1]. In comparison, the HEART score was derived from a population of ED patients with chest pain, and has been extensively validated worldwide [10,13,27,35]. It has demonstrated its utility in identifying both low-risk patients for possible early discharge and high-risk patients for urgent intervention. Built upon established scores, several chest pain pathways [14,[36][37][38] have been implemented and tested, particularly for the management of low-risk patients. Than et al. [38] evaluated a TIMI score-based accelerated diagnostic protocol (ADP) with a reported sensitivity of 99.3% and NPV of 99.1%. Similarly, a systematic review by Laureano-Phillips et al. [39] reported that the HEART score achieved both sensitivity and NPV of 100% in several validation studies. Furthermore, a cost-effectiveness study conducted in Brisbane, Australia reported economic benefits by adopting an ADP in the ED, with reduction in expected cost and length of stay amongst patients with chest pain [40].
Most established clinical scores use conventional risk factors such as biomarkers, medical history, and presenting vital signs. However, patient history can sometimes be subjective and blood tests, such as troponin, require waiting time. HRV, as a noninvasive measure, can be easily calculated from ECGs; it is an objective tool to assess the activities of the ANS [19]. It also has the advantage of requiring only several minutes to acquire (five to six minutes in our protocol), which is much faster than serum biomarkers. Over the past decades, HRV has been widely investigated in a broad range of clinical applications, particularly in cardiovascular research. Apart from being associated with sudden cardiac death [18], HRV also showed significant correlations with adverse clinical outcomes in prehospital setting [41] and with MACE outcomes in ED patients with chest pain [17]. HRV parameters have been integrated with other risk factors into machine learning algorithms to predict adverse outcomes [42,43]. These promising results motivated the use of HRV to develop objective and computerized risk stratification tools for chest pain patients [44,45]. In an updated review of clinical scores for chest pain, Liu et al. [5] summarized several studies which aimed to develop alternative techniques for risk stratification.
This study aimed to present novel HRnV representation and its measures and investigate their association HRV Heart rate variability, OR Odds ratio, CI Confidence interval, mean NN Average of R-R intervals, SDNN Standard deviation of R-R intervals, RMSSD Square root of the mean squared differences between R-R intervals, NN50 The number of times that the absolute difference between 2 successive R-R intervals exceeds 50 ms, pNN50, NN50 divided by the total number of R-R intervals; NN50n, the number of times that the absolute difference between 2 successive RR n I/RR n I m sequences exceeds 50 × n ms; pNN50n, NN50n divided by the total number of RR n I/RR n I m sequences; VLF Very low frequency, LF Low frequency, HF High frequency, SD Standard deviation, SampEn Sample entropy, ApEn Approximate entropy, DFA Detrended fluctuation analysis * p < 0.05 In this study, with five to six-minute ECG recording and n = 3, five-fold more HRnV parameters were calculated compared to HRV alone. When longer ECG recordings are available and parameter n is larger, more HRnV parameters can be derived. To build a HRnV-based risk stratification tool, a systematic approach is needed to derive a point-based, consistent score to ease its clinical application and practical implementation. As a natural extension of conventional HRV, HRnV representation creates the opportunity to generate additional parameters. This representation could also serve as a smoother for RRIs, making them less sensitive to sudden changes caused by abnormal heart beats (e.g. very short or very long RRI). However, since HRnV is a novel representation of beat-to-beat variations in ECG, many technical issues need to be addressed in future research. For instance, as shown in Table 2, SampEn became larger when the available number of data points was less than 200 [19], suggesting that additional research is required to investigate its applicability to short ECG records. Moreover, parameters NN50n and pNN50n are newly introduced in HRnV representation only. They characterize the number of times that the absolute difference between two successive RR n I sequences exceeds 50 × n ms, by assuming that the absolute difference may be magnified when the corresponding RR n I is n times longer than RRI. Thus, in-depth investigations are required in the selection of appropriate thresholds. More importantly, physiological interpretations of the HRnV parameters and their normal values [29] need to be determined through numerous research. One example is the identification of frequency bands that correlate with certain physiological phenomenon. In the current analysis, the conventional cut-off values were adopted (i.e., ≤0.04 Hz as very low frequency range; 0.04-0.15 Hz as low frequency range; 0.15-0.4 Hz as high frequency range). With the increase in n, frequency domain analysis may need to be changed accordingly.
Beyond its use in risk stratification of ED patients with chest pain, HRnV can potentially be used in other clinical domains, where conventional HRV has been extensively investigated [46][47][48][49]. With the augmented RR n I and RR n I m sequences, HRnV could possibly capture more dynamic changes in cardiac rhythms than HRV. This capability enables the extraction of additional information HRV Heart rate variability, OR Odds ratio, CI Confidence interval, mean NN average of R-R intervals, SDNN Standard deviation of R-R intervals, RMSSD Square root of the mean squared differences between R-R intervals, NN50, the number of times that the absolute difference between 2 successive R-R intervals exceeds 50 ms; pNN50, NN50 divided by the total number of R-R intervals; NN50n, the number of times that the absolute difference between 2 successive RR n I/RR n I m sequences exceeds 50 × n ms; pNN50n, NN50n divided by the total number of RR n I/RR n I m sequences; VLF Very low frequency, LF Low frequency, HF High frequency, SD Standard deviation, SampEn Sample entropy, ApEn Approximate entropy, DFA Detrended fluctuation analysis * p < 0.05  Fig. 2 The receiver operating characteristic (ROC) curves produced by heart rate n-variability (HRnV) model (performance was based on leaveone-out cross-validation), the History, ECG, Age, Risk factors and Troponin (HEART) score, the Thrombolysis in Myocardial Infarction (TIMI) score, and the Global Registry of Acute Coronary Events (GRACE) score from limited raw ECGs. This study utilized HRnV parameters as independent risk factors and analyzed them with traditional biostatistical methods. There are multiple ways to use HRnV parameters, e.g. each set of HRnV parameters can be analyzed individually and subsequently combined with an ensemble learning [50] (a special type of machine learning algorithm) architecture to reach a decision. However, artificial intelligence and machine learning methods generally create black-box predictive models, making interpretation a challenge [51].

Limitations
This study has several limitations. First, we did not develop a scoring tool for practical clinical use. The primary aim of this study was to demonstrate the feasibility of using HRnV parameters and common risk factors to build predictive models. Second, the HRnV model was evaluated with LOOCV strategy due to the small sample size. Ideally, separate patient cohorts are needed to train and test prediction models. When a new scoring tool is developed, it is necessary to conduct external validations on cohorts with diverse patient characteristics. Furthermore, properly designed clinical pathways are needed as well. Third, the patients included in this study were mainly from the high acuity group, resulting in a higher 30-day MACE rate (i.e., 31%) compared to other similar studies [10,39]. As a result, the generalizability of the HRnV model developed in this study may be uncertain in other patient cohorts. Fourth, the calculated HRnV and HRV parameters depended on the choice of tools and methods for ECG signal analysis. Thus, the values of these parameters may vary across studies. Last, the physiology of HRnV and interpretations of its measures are mostly unknown; calculation of some parameters also needs to be standardized. All these require future collaborative research efforts between clinicians and scientists to address.