ACTION (acute coronary treatment and intervention outcomes network) registry-GWTG (get with the guidelines) risk score predicts long-term mortality in acute myocardial infarction

This study aimed to test the predictive performance of the updated ACTION, GRACE, and CADILLAC risk scores (RS’s) for long-term mortality in patients with acute myocardial infarction (AMI) undergoing percutaneous coronary intervention (PCI). The study included individuals from 2 independent cohorts: derivation cohort (N = 1901) and validation cohort (N = 728). From the derivation cohort, we used Cox regression analysis to determine that the updated ACTION, GRACE, and CADILLAC RS's were associated with long-term mortality. The concordance (C) statistics of the 3 RS's were 0.682, 0.703 and 0.734, respectively. We used the validation cohort to validate the results. Moreover, the discriminatory performance of the updated ACTION RS for predicting long-term mortality in both the respective derivation and validation cohorts was similar to the discriminatory performance of the GRACE and CADILLAC RS's (ACTION vs. GRACE: z = 0.684, p = 0.494; ACTION vs. CADILLAC: z = 1.638, p = 0.101) and (ACTION vs. GRACE: z = 0.460, p = 0.646; ACTION vs. CADILLAC: z = 0.290, p = 0.772). Despite their development over a decade ago, GRACE and CADILLAC RS's maintain good performance for predicting the long-term mortality of AMI patients undergoing PCI. As a new risk model, the updated ACTION RS also predicts long-term mortality, and its discriminatory performance is similar to that of the GRACE and CADILLAC RS’s.


INTRODUCTION
Although advances have been made in the treatment strategies and management of patients with acute myocardial infarction (AMI), some patients still have poor outcomes [1][2][3][4]. Early risk stratification is important for such patients. Patients found to be at high risk by early risk stratification should receive appropriate clinical treatment and long-term follow up. The Acute Coronary Treatment and Intervention Outcomes Network (ACTION) Registry-Get With The Guidelines (GWTG) AMI mortality model and risk score (ACTION) were introduced in 2011 to predict in-hospital mortality [5]. The simple risk model was subsequently externally validated [6]. The existing ACTION risk model was updated in 2016 [7]. The C statistic of the updated ACTION risk model was 0.88, with good calibration. However, no published study has focused on the value of the risk model for predicting long-term mortality. An extensive assessment of the prognostic value of the risk model for patients with AMI undergoing percutaneous coronary intervention (PCI) that validates the risk model has not been performed.
However, over the last few decades, many other risk models that facilitate assessment of the long-term outcomes of AMI patients with or without PCI have been introduced. Among these risk models, the Global Registry of Acute Coronary Events (GRACE) clinical risk scoring system is most frequently used and recommended [1][2][3][4]. It was derived from a very large 'real-world', worldwide study of patients with the entire spectrum of acute coronary syndromes (ACS's), and can predict in-hospital [8], 6-month [9], and even longer-term (up to 4 years) [10] mortality after ACS to help us identify high-risk patients. The CADILLAC risk score was derived from participants of the Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications (CADILLAC)

Clinical Research Paper
Oncotarget 102560 www.impactjournals.com/oncotarget trial [11]. It was found to provide accurate predictions of in-hospital and 1-year mortality of AMI patients undergoing primary PCI. Both risk models have been in use for more than a decade; however, the treatment and management of patients with AMI has greatly improved over this past decade. Therefore, these older risk models should be validated in recent patient cohorts. In addition, to the best of our knowledge, no study has been published that compares the prognostic accuracy of GRACE and CADILLAC with new risk models, such as the updated ACTION risk model, for the long-term outcome of patients with AMI who PCI.
In this retrospective cohort study, we aimed to determine if the updated ACTION risk score is a useful method for predicting long-term mortality in AMI patients undergoing PCI. We then used an independent prospective cohort to validate our findings. We also used both cohorts to test the predictive performance of the GRACE and CADILLAC risk scores and to compare the prognostic accuracy of the updated ACTION risk score with the prognostic accuracy of the GRACE and CADILLAC risk scores.

Derivation and validation cohorts
As shown by the population flow diagram (Figure 1), the final numbers of patients in the derivation and validation cohorts were 1901 and 728 patients, respectively. Table 1 shows the baseline characteristics of the 2 cohorts. Participants in the validation cohort had higher left ventricular ejection fractions (LVEFs) and greater creatinine clearance and lower ACTION, GRACE, or CADILLAC risk scores than participants in the derivation cohort. The validation cohort had a lower prevalence of anemia, three-vessel disease, use of intra-aortic balloon pump, use of angiotensin-converting enzyme inhibitors/ angiotensin receptor blockers, and use of beta-blockers than the derivation cohort. The validation cohort showed higher prevalence of prior MI, prior PCI, use of glycoprotein IIb/IIIa inhibitors, and statins than the derivation cohort. However, other characteristics such as age and gender distributions were comparable between cohorts ( Table 1).

Development of updated ACTION risk score as a predictive parameter of outcome
Over a mean follow up of 32 months, there were 64 events (3.4% of 1901 cases) of all-cause mortality in the derivation cohort. Cox regression analysis of the updated ACTION, GRACE, and CADILLAC risk scores, which were used as continuous variables, were significantly predictive of long-term mortality ( Table 2). When categorized into groups, all 3 risk scores remained significantly predictive of long-term mortality ( Table 2).
Compared with the very-low-risk ACTION score, the patient groups with low-risk, intermediate-risk, high-risk, and very-high-risk ACTION scores had a substantially higher risk of death (Table 2). Compared with the low-risk GRACE and CADILLAC scores, the respective patient groups with GRACE and CADILLAC intermediate-risk and high-risk scores had a substantially higher risk of death.
The Hosmer-Lemeshow (HL) goodness-of-fit test indicated that the 3 risk models all showed good calibration for predicting death in the derivation cohort ( Table 3). The C-statistics of the updated ACTION, GRACE, and CADILLAC risk scores for long-term mortality were 0.682, 0.703 and 0.734, respectively (Table 3 and Figure 2).

Validation of updated ACTION risk score as a predictive parameter for outcome
The all-cause mortality of the validation cohort over a 1-year follow up was 4.0% (29 of 728 cases). Cox regression analysis of the updated ACTION, GRACE, and CADILLAC risk scores, which were used as continuous variables were significantly predictive of long-term mortality ( Table 2). When categorized into groups, all 3 risk scores remained significantly predictive of longterm mortality (Table 2). Compared with the very-lowrisk ACTION score, the patient groups with low-risk, intermediate-risk, high-risk, and very-high-risk ACTION scores had a substantially higher risk of death (Table 2). Compared with the low-risk GRACE and CADILLAC scores, the respective patient groups with GRACE and CADILLAC intermediate-risk and high-risk scores had a substantially higher risk of death.
The HL goodness-of-fit test indicated that the 3 risk models all showed good calibration for predicting death in the validation cohort ( Table 3). The C-statistics of the updated ACTION, GRACE and CADILLAC risk scores for long-term mortality were 0.713, 0.741 and 0.731, respectively (Table 3 and Figure 3).

Prognostic accuracy of updated ACTION, GRACE, and CADILLAC risk scores
The stratified risk scores of the 3 systems could all identify the high-risk patients in both cohorts as follows: updated ACTION risk score, Figure 4 shows 5 subgroups in the derivation cohort (p < 0.001) and 5 subgroups in the validation cohort, (p < 0.001); GRACE risk score, Figure 5 shows 3 subgroups in the derivation cohort (p < 0.001) and 3 subgroups in the validation cohort (p < 0.001); CADILLAC risk score, Figure 6 shows 3 subgroups in the derivation cohort (p < 0.001) and 3 subgroups in the validation cohort (p < 0.001).
For the derivation cohort, the discriminatory performance of the updated ACTION risk score was similar to the discriminatory performances of the GRACE and CADILLAC risk scores for predicting long-term www.impactjournals.com/oncotarget all-cause mortality (ACTION vs. GRACE: z = 0.684, p = 0.494; ACTION vs. CADILLAC: z = 1.638, p = 0.101) ( Table 4). For the validation cohort, the discriminatory performance of the updated ACTION risk score was similar to the discriminatory performances of the GRACE and CADILLAC risk scores for predicting long-term all-cause mortality (ACTION vs. GRACE: z = 0.460, p = 0.646; ACTION vs. CADILLAC: z = 0.290, p = 0.772) ( Table 4).
For both cohorts, the discriminatory performance of the GRACE risk score was similar to the discriminatory performance of the CADILLAC risk score for predicting long-term all-cause mortality (derivation cohort, GRACE vs. CADILLAC: z = 0.788, p = 0.431; validation cohort, GRACE vs. CADILLAC: z = 0.224, p = 0.823) ( Table 4).

Predicted and observed mortality rates of the different patient subgroups in the validation cohort
The predicted and observed mortality rates of the different patient subgroups in the validation cohort were compared (Table 5). These subgroups included the following: age < 65 yrs, age ≥ 65 yrs, male, female, diabetes mellitus, no diabetes mellitus, hypertension, no hypertension, ST-segment elevation myocardial infarction (STEMI), non-ST-segment elevation myocardial infarction (NSTEMI), three-vessel disease, and no three-vessel disease. There were no significant differences between the predicted and observed mortality rates of all the subgroups for the updated ACTION, GRACE and CADILLAC risk models (Table 5).

DISCUSSION
To the best of our knowledge, this is the first study to investigate the predictive value of the updated ACTION risk model for long-term mortality in AMI patients undergoing PCI. The main findings were as follows: (1) the updated ACTION risk model predicted the long-term mortality of patients with AMI undergoing PCI; (2) despite their development over a decade ago, the GRACE and CADILLAC risk scores still maintain good performance for predicting the long-term risk of mortality of AMI patients undergoing PCI in 2 recent cohorts; (3) the discriminatory performance of the updated ACTION risk model was similar to the discriminatory performance of the GRACE and CADILLAC scoring systems for predicting long-term mortality. The ACTION risk model was developed and validated to predict the risk of in-hospital mortality in 2011 in order to improve performance quality and guide investigators studying patients with STEMI or NSTEMI who were treated in routine clinical practice [5]. The ACTION risk model was developed from data in the ACTION Registry-GWTG database, which included a very large number of consecutive AMI patients from various institutions and regions across the United States. The developers found that the following factors showed the highest independent significance for in-hospital mortality: age, baseline serum creatinine, systolic blood  patients admitted to a single center in Spain [6]. The risk scores all showed good discrimination for in-hospital death (C-statistics: 0.91, 0.90 and 0.90, respectively) with optimal calibration (HL p: 0.42, 0.50, and 0.47, respectively). In 2016, data from January 2012 through December 2013 in the ACTION Registry-GWTG database was used to update the ACTION risk model [7]. Several changes in risk adjustment were incorporated into the new model. Firstly, the updated model included presentation after cardiac arrest, which was not available at the time the  Table 2, the ACTION risk score still maintained an independent and significant positive correlation with all-cause mortality as a categorical variable. Based on the updated ACTION risk model, a simplified integer score performed well and could easily identify the high-risk patients; patients in the very-highrisk ACTION risk score group had the highest all-cause mortality in both cohorts (Figure 4). The model showed  The GRACE risk score was first developed to assessing in-hospital [8] and 6-month [9] mortality after ACS. It was soon verified by many external cohorts from  various countries and regions [6,[18][19][20][21][22]. Furthermore, it was extended to predict angiographic severity and contrast-induced nephropathy in patients with ACS [23,24]. It could also identify those high-risk patients who required urgent or elective PCI [25]. Our study again confirmed that the GRACE risk score could predict the long-term mortality of patients with AMI undergoing PCI (derivation cohort: HR = 1.022, 95% CI 1.015 -1.028, p < 0.001; validation cohort: HR = 1.024, 95% CI 1.014-1.035, p < 0.001). Also, as a categorical variable, the GRACE risk score was independently associated with all-cause mortality and could easily identify highrisk patients; patients in the high-risk GRACE risk score groups of both cohorts showed the highest all-cause mortality ( Table 2 and Figure 5). In our study, the GRACE risk score still had good prognostic accuracy with good calibration in both the validation and derivation cohorts (C-statistics: 0.703, 0.741, respectively; H-L p-Value = 0.135, 0.297, respectively) ( Tables 3 and 5). The good discriminatory performance of GRACE was expected. The GRACE risk scoring system is unparalleled, since it was derived from a large worldwide registry of 'real-world' patients with the entire spectrum of ACS. However, in our study, the predictive accuracy of GRACE was higher for the validation cohort than for the derivation cohort. One explanation might be that the follow-up periods of the 2 cohorts were different (the patients in the derivation cohort were followed for a mean duration of 32 months; the patients in the validation cohort received a 1-year follow up). Another study showed a similar finding, the longer the follow-up, the smaller the C-statistic [26]. The CADILLAC risk score was also found to be independently associated with the long-term mortality of AMI patients undergoing PCI (derivation cohort: HR = 1.334, 95% CI 1.243-1.432, p < 0.001; validation cohort: HR = 1.394, 95% CI 1.207-1.611, p < 0.001). Also, as a categorical variable, a simplified integer CADILLAC risk score could indicate risk stratification for patients with different scores (Table 2 and Figure 6). In our study, the CADILLAC risk score still showed good prognostic accuracy with good calibration for both the validation and derivation cohorts (C-statistics: 0.734, 0.731, respectively; H-L p-value = 0.946, 0.107, respectively) ( Tables 3  and 5). In contrast with the GRACE risk scoring system, the CADILLAC risk scoring system was based on 2 multicenter randomized AMI trials of patients undergoing primary PCI, and contains both clinical and angiographic variables. Although it was not derived from 'real-world' data, the angiographic variables have been confirmed as important prognostic information [27,28]. The CADILLAC risk score was also verified by other external cohorts [26,27,29,30]. In our study, the CADILLAC risk score also showed good prognostic accuracy for both cohorts. Interestingly, unlike the GRACE risk score, the CADILLAC risk score yielded almost the same C-statistics for both cohorts. Another study by Brkovic et al found that the SYNTAX score, which grades the complexity of coronary angiographic anatomy, improved the prognostic performance of the GRACE risk score, but not the CADILLAC risk score. The long-term survival of patients after STEMI determined by the CADILLAC risk score mainly depended on clinical characteristics, myocardial function, and basic angiographic findings [27]. This might account for the consistency of the C-statistics for the CADILLAC risk scores of both of our cohorts. However, the CADILLAC risk scoring system performed worse in our study (C-statistics: 0.734 in the derivation cohort; 0.731 in the validation cohort) than in the derivation and validation cohorts from the CADILLAC risk scores [11], respectively, with C-statistics of 0.79 and 0.78, respectively. The most important reason for the discrepancy in results is the difference between the participants of the the CADILLAC risk scores and the participants of our study. The CADILLAC risk score was only derived from patients with STEMI undergoing primary PCI, and patients with NSTEMI, cardiogenic shock, and other severe diseases were excluded [11]. In contrast, our study was based on 'real-world' patients with AMI who underwent PCI.

Limitations
This study had several limitations. Firstly, the cohorts were both from a single center. Our study only included AMI patients undergoing PCI, and did not include patients receiving conservative medical treatment only. The derivation cohort was retrospective. So, potential confounders and selection bias could not be completely adjusted. Secondly, compared with the original study cohorts [5-9], our study cohorts were relatively small. Thirdly, our study could not test other powerful mortality prediction models because of missing data. For example, our cohorts had no data for the Zwolle risk score on ischemia time in NSTEMI patients [31]; for the TIMI risk score, use of aspirin over the last 7 days in NSTEMI patients [32]; for the PURSUIT risk score, worst Canadian Cardiovascular Society (CCS) class of the 6 weeks [33]; for the PAMI risk score, left bundle branch block [34]. Therefore, comparing the ACTION risk score with other risk scores becomes difficult. Fourthly, our study found that the C-statistics of all 3 risk scores were lower than those in the original cohorts. This is mainly accounted for by the very low mortality in our study. The lower mortality in our cohorts is due to the lack of adequate medical emergency infrastructure and a large number of low-income people. In remote districts or rural areas, many high-risk AMI patients cannot be rapidly transferred to a hospital that offers PCI because of lack of adequate medical emergency infrastructure. In addition, high-risk AMI patients usually need more treatment time and more expensive treatment, and their outcomes are worse. Under the conditions described, low-income, high-risk AMI patients usually refuse expensive PCI, and request conservative medical treatment only. Finally, we www.impactjournals.com/oncotarget did not test the predictive ability of the updated ACTION risk model for in-hospital mortality and excluded patients who died during the index hospitalization, since the updated ACTION risk model was originally designed for predicting in-hospital mortality.

Study Design and Setting Derivation Cohort and Validation Cohort
We performed this study using 2 independent cohorts. The derivation cohort consisted of a retrospective cohort study for which the rationale and design were previously described [12]. In brief, from January 1, 2010 to October 31, 2014, consecutive AMI patients, who were hospitalized and underwent PCI at a large hospital in Northeast China (Shengjing Hospital of China Medical University, Shenyang, China), were included. The clinical and procedural data of all cases were obtained by the investigators from electronic medical records and a picture archiving and communication system (PACS), which stores data from interventional imaging and the surgical records of PCI cases. Left ventricular ejection fractions (LVEFs) were determined during hospitalization by echocardiography. Thrombolysis in myocardial infarction (TIMI) flow scores were determined as defined previously [13]. The initial cohort consisted of 2060 patients. The following patients were then excluded: (1) missing ACTION, GRACE, or CADILLAC risk scores (80 cases), (2) lost to follow up (62 cases), (3) died during the index hospitalization (17 cases). The final cohort consisted of 1901 AMI patients undergoing PCI. Clinical follow-up was performed in October 2015 by phone interviews with the patient's general practitioner/cardiologist, patient him-/ herself, or patient's family. All patients were followed for a mean duration of 32 months (12 to 67 months).
The validation cohort consisted of an ongoing, prospective, observational cohort-P-PUSH (prospective evaluation of prognosis of Pci patients) using network data from Shengjing Hospital of China Medical University. The P-PUSH database contained comprehensive clinical and procedural data from all consecutive AMI patients who underwent PCI from January 1, 2015, at Shengjing Hospital of China Medical University. We constructed patient care report forms (CRFs) that contained 332 discrete items that were subdivided, and included the following: demographic characteristics, past history, clinical characteristics on admission, laboratory measurements, procedure-related complications, and relevant therapeutic agents. The attending physician used the electronic medical records to complete the CRFs after discharge. The diagnoses of patients were classified by attending cardiologists according to guidelines [1][2][3][4]. LVEFs were determined during hospitalization by echocardiography. TIMI flow scores were determined as defined previously [13]. Prospective clinical follow-up after discharge was performed regularly during hospital visits or phone interviews with the patient's general practitioner/cardiologist, patient her-/himself, or his or her family in all participants. All events were adjudicated and classified by 2 cardiologists. The exclusion criteria of the validation cohort included the following: (1) missing ACTION, GRACE, or CADILLAC risk scores (21 cases), (2) lost to follow up (60 cases), (3) died during the index hospitalization (5 cases). The final validation cohort consisted of 728 AMI patients who underwent PCI and were followed for 1 year, from January 1, 2015 to January 1, 2016.
All-cause mortality was identified from the patients' medical records or each patient's referring hospital physician. All events were validated by 2 independent physicians. This study complied with the Declaration of Helsinki, and the Shengjing Hospital of China Medical University Research Ethics Committee approved the research protocol. Written informed consent was obtained from all participants.

Participants and procedures
AMI was defined according to current guidelines [1][2][3][4]. Briefly, non-ST-segment elevation myocardial infarction (NSTEMI) was defined as follows: (1) chest discomfort or anginal equivalent, (2) ST-segment depression, transitory ST-segment elevation or prominent T-wave inversion, and (3) positive biomarkers of necrosis (creatine kinase-myocardial band, cardiac troponin-I); ST-segment elevation myocardial infarction (STEMI) was defined as (1) duration of chest pain less than 12 hours from onset of pain to time of catheterization, (2) notable ST-segment elevation (at least 0.1 mV in 2 or more standard leads or at least 0.2 mV in 2 or more contiguous precordial leads), or (3) a new left bundle branch block.

Statistical analysis
Quantitative variables were represented as means ± standard deviation (SD) or medians (interquartile range [IQR]), and categorical variables were represented as counts and proportions (%). Cox proportional hazards regression modeling by the forward stepwise procedure was used to analyze the effect of variables on event-free survival. Results were reported as hazard ratios (HRs) with associated 95% confidence intervals (CIs). The performance of the various risk scores was assessed by indices of calibration and discrimination. Each risk score was entered into a logistic regression model to obtain individual risk probability of all-cause death. The Hosmer-Lemeshow (HL) statistic from regression modeling was used as an indicator of goodness-of-fit of each risk score and to assess calibration ability [14]. In the HL Goodnessof-Fit test, higher p values indicate better calibration. The discriminative power of each risk score was assessed by the concordance (C) statistic, which was defined by the area under the receiver operating characteristic (AUROC) curve in relation to long-term all-cause mortality [15]. The C statistics for the 3 risk scores were compared by a nonparametric test developed by DeLong et al. [16,17], with the use of MedCalc software for Windows, version 11.4.2.0 (MedCalc Software, Mariakerke, Belgium). All tests were two-sided, and statistical significance was defined as p < 0.05. Statistical analysis was performed by SPSS version 19 (SPSS Inc., Chicago, Illinois, USA).

CONCLUSIONS
The updated ACTION risk model predicts the long-term mortality of patients with AMI who undergo PCI. Despite their development over a decade ago, the GRACE and CADILLAC risk models still maintain good performance for predicting the long-term risk of death in AMI patients undergoing PCI from 2 recent cohorts. The discriminatory performance of the updated ACTION risk model for predicting long-term mortality was similar to the discriminatory performances of the GRACE and CADILLAC scoring systems.

Author contributions
Zhaoqing Sun conceived and designed the experiments. Tongtong Yu, Chunyang Tian, Jia Song, Dongxu He and Zhijun Sun and performed the experiments. Tongtong Yu analyzed the data and wrote the paper. Zhaoqing Sun revised the paper. All authors had reviewed and agreed on the contents of this paper.