Evaluation of risk equations for prediction of short-term coronary heart disease events in patients with long-standing type 2 diabetes: the Translating Research into Action for Diabetes (TRIAD) study

  • Shou-En Lu1Email author,

    Affiliated with

    • Gloria L Beckles2,

      Affiliated with

      • Jesse C Crosson3,

        Affiliated with

        • Dorian Bilik4,

          Affiliated with

          • Andrew J Karter5,

            Affiliated with

            • Robert B Gerzoff2,

              Affiliated with

              • Yong Lin1,

                Affiliated with

                • Sonja V Ross3,

                  Affiliated with

                  • Laura N McEwen4,

                    Affiliated with

                    • Beth E Waitzfelder6, 7,

                      Affiliated with

                      • David Marrero8,

                        Affiliated with

                        • Norman Lasser9 and

                          Affiliated with

                          • Arleen F Brown10

                            Affiliated with

                            BMC Endocrine Disorders201212:12

                            DOI: 10.1186/1472-6823-12-12

                            Received: 28 March 2012

                            Accepted: 22 June 2012

                            Published: 9 July 2012

                            Abstract

                            Background

                            To evaluate the U.K. Prospective Diabetes Study (UKPDS) and Framingham risk equations for predicting short-term risk of coronary heart disease (CHD) events among adults with long-standing type 2 diabetes, including those with and without preexisting CHD.

                            Methods

                            Prospective cohort of U.S. managed care enrollees aged ≥ 18 years and mean diabetes duration of more than 10 years, participating in the Translating Research into Action for Diabetes (TRIAD) study, was followed for the first occurrence of CHD events from 2000 to 2003. The UKPDS and Framingham risk equations were evaluated for discriminating power and calibration.

                            Results

                            A total of 8303 TRIAD participants, were identified to evaluate the UKPDS (n = 5914, 120 events), Framingham-initial (n = 5914, 218 events) and Framingham-secondary (n = 2389, 374 events) risk equations, according to their prior CHD history. All of these equations exhibited low discriminating power with Harrell’s c-index <0.65. All except the Framingham-initial equation for women and the Framingham-secondary equation for men had low levels of calibration. After adjsusting for the average values of predictors and event rates in the TRIAD population, the calibration of these equations greatly improved.

                            Conclusions

                            The UKPDS and Framingham risk equations may be inappropriate for predicting the short-term risk of CHD events in patients with long-standing type 2 diabetes, partly due to changes in medications used by patients with diabetes and other improvements in clinical care since the Frmaingham and UKPDS studies were conducted. Refinement of these equations to reflect contemporary CHD profiles, diagnostics and therapies are needed to provide reliable risk estimates to inform effective treatment.

                            Background

                            Adults with diabetes have an increased risk of coronary heart disease (CHD) [1, 2]. Their risks of having an initial CHD event and the prediction equations used to determine this risk have been extensively studied [312]. These risk estimates are helpful for clinical consultation and identifying high risk populations for effective treatment. Recent studies even recommended that the initiation of cardio-protective treatment for diabetic patients be based on personalized CHD risk estimates to reduce harms from overly aggressive risk factor modification [13, 14]. Several equations are currently available to estimate CHD risk. Among them are algorithms derived for use in the general population such as the Systematic Coronary Risk Evaluation (SCORE) [15], QRISK [16, 17], the Reynolds Risk Score for women [18] and the Framingham risk equations [4, 6]. Most of these algorithms include diabetes as a risk factor when determining CHD risk. Algorithms particularly developed for use in diabetic populations include the United Kingdom Prospective Diabetes Study (UKPDS) risk equations [3] and a recent algorithm developed by Donnan et al.[10]. Among these algorithms, the UKPDS and Framingham equations [3, 4] are frequently used to predict the risk of an initial CHD event for diabetic patients. Their use has also been recommended in national guidelines in various regions [1923]. However, previous studies have shown that both the UKPDS and Framingham risk equations can give unreliable risk estimates of an initial CHD event for diabetic patients in some European and Australian cohorts [7, 9, 11, 24, 25]. Given the variation in practice patterns, patient racial/ethnic composition between these regions and the U.S., as well as changes in the standards of clinical care for patients with diabetes over the last few decades, it is not clear whether these equations can provide reliable risk estimates for adults with long-standing diabetes in the U.S. Recent estimates showed that approximately 785,000 Americans will have a new coronary event each year, with approximately 470,000 of them a recurrent event [26]. It is of both clinical and public health importance to understand and re-evaluate the risk stratifications for patients with long-standing type 2 diabetes.

                            In this paper, we evaluate the performance of the UKPDS [3] and Framingham initial CHD risk equations [4] in predicting CHD occurrence for adults with long-standing type 2 diabetes without an established CHD history. We also evaluated the performance of the secondary Framingham risk equation [27] in predicting CHD events among those with an established CHD history as this has not been studied among adults who have diabetes. Because adults with long-standing diabetes are known to have higher CHD risk than those without diabetes or newly diagnosed with diabetes, we focused on estimates of short-term CHD event risk using data from the Translating Research Into Action for Diabetes (TRIAD) study, a large multi-center, population-based, prospective study of diabetic adults [28].

                            Methods

                            The TRIAD study has been described in detail elsewhere [28]. In brief, the initial TRIAD cohort consisted of 11927 community-dwelling adults with diabetes ages 18 years and older, and continuously enrolled between July 2000 and August 2001 in one of 10 managed care plans in 7 states. TRIAD data included patient surveys, medical record reviews, health plan administrative claims (inpatient, outpatient and emergency room claims), and National Death Index (NDI) data. In our analyses, we limited our sample to the 8820 TRIAD participants for whom we had medical record data 18 months prior to the baseline survey. In addition, we excluded those whose age at diabetes diagnosis was less than 30 years and were treated with insulin only at the time of the baseline survey since it is likely they had type 1 diabetes. Institutional review boards at each participating site approved the study and all participants provided informed consent.

                            Risk predictors such as patient age, diabetes duration (years since diagnosis), and smoking status were obtained from survey responses. Hemoglobin A1C, systolic blood pressure, diastolic blood pressure, total cholesterol, HDL, and LDL were obtained from medical records, and only the most recent values within the 18 months prior to the baseline survey were used. Patients were defined as having a CHD history if at least one of the following conditions was documented in their medical records within 3 years prior to the baseline survey: angina, MI, coronary heart disease, coronary artery disease, coronary angioplasty or coronary bypass. We also obtained additional baseline information on diabetes treatment from patient surveys, and determined the use of hypertension medication, statins and co-morbid conditions (measured by the Charlson’s score [29, 30]) from medical records from 5 out of the original 6 research centers where the data were available to us.

                            Outcome variables

                            For evaluation of each risk equation, we used the CHD event definition used in the study that provided the equations. To evaluate the UKPDS risk equations, we defined a CHD event as: a fatal or nonfatal MI (ICD-9-CM code of 410.xx administrative data; ICD-10 of I21-I22 NDI data); to evaluate the Framingham risk equations, we defined a CHD event as: angina pectoris, MI, coronary insufficiency, sudden death, or CHD death (ICD-9-CM code of 410.xx, 413, 411.89, 414.8 administrative data; ICD-10 of I20-I22 and I46.1 NDI data). For each CHD event, we calculated the “CHD event time” as the time from the TRIAD baseline survey to the first CHD event. Observations were censored at the end of 2003, date of death from another cause, or the date of the first health plan enrollment gap of more than two months, whichever came first.

                            UKPDS and Framingham CHD risk equations

                            We evaluated various versions of UKPDS, Framingham initial and Framingham secondary CHD risk equations (Table 1). Specifically, two UKPDS risk equations were evaluated: The first equation predicts the risk of an initial CHD event for a patient with newly diagnosed diabetes and we referred to it as the incident UKPDS risk equation. The second equation takes into account diabetes duration when predicting the risk of an initial CHD event and we thus referred to it as the duration UKPDS risk equation. Framingham risk equations are gender-specific and were thus evaluated separately for men and women. We evaluated the risk equations for predicting an initial CHD event using total cholesterol categories and referred to it as Framingham-initial. We also evaluated the performance of the equations for predicting a subsequent CHD event and referred to it as the Framingham-secondary.
                            Table 1

                            UKPDS and framingham risk equations

                            Equations

                             

                            Formula for t-year CHD risk

                            UKPDS Incident Stevens et al. [3]

                             

                            http://static-content.springer.com/image/art%3A10.1186%2F1472-6823-12-12/MediaObjects/12902_2012_133_IEq1_HTML.gif where d = 1.078 and log e q = −4.4918+ 0.0573× (age-55)0.6444× female-0.9416× Afro-Caribbean + 0.3001× smoking + 0.1681× (HbA1c-6.72) +0.0843× (SPB-135.7)/10 + 1.3468× {log e (TC a /HDL)-1.59}.

                            UKPDS Duration

                             

                            http://static-content.springer.com/image/art%3A10.1186%2F1472-6823-12-12/MediaObjects/12902_2012_133_IEq2_HTML.gif where T = diabetes duration in years, and d and q were defined as above.

                            Framingham-Initial

                            Table 6 in Wilson et al. [4]

                            Male

                            http://static-content.springer.com/image/art%3A10.1186%2F1472-6823-12-12/MediaObjects/12902_2012_133_IEq3_HTML.gif where m = 0.0483×Age-0.6595× (TC a < 160 mg/dL) + 0.1769×(TC 200-239 mg/dL) + 0.5054×(TC 240-279 mg/dL) + 0.6571×(TC> = 280 mg/dL) + 0.4974×(HDL < 35 mg/dL) + 0.2431× (HDL 35-44 mg/dL)-0.0511× (HDL 50-59 mg/dL)-0.4866× (HDL > =60 mg/dL)-0.0023×(BP Optimal) + 0.2832× (BP High-normal) + 0.5217×(BP Stage-I-hypertension) + 0.6186×(BP Stage-II-IV-hypertension) + 0.4284×Diabetes + 0.5234× (Smoker)-3.0975

                              

                            S 0 (1) b =0.9946, S 0 (2) = 0.9850, S 0 (3) = 0.9770, S 0 (4) = 0.9622, S 0 (5) = 0.95592.

                            Female

                            http://static-content.springer.com/image/art%3A10.1186%2F1472-6823-12-12/MediaObjects/12902_2012_133_IEq4_HTML.gif where m = 0.3377×Age-0.0027×age 2 -0.2614×(TC a < 160 mg/dL) + 0.2077×(TC 200-239 mg/dL)+0.2439×(TC 240-279 mg/dL) + 0.5351×(TC > =280 mg/dL)+0.8431×(HDL < 35 mg/dL) + 0.3780×(HDL 35-44 mg/dL)+0.1979×(HDL 45-49 mg/dL)-0.4295×(HDL > =60 mg/dL)0.5336×(BP Optimal)-0.0677×(BP High-normal)+0.2629×(BP Stage-I-hypertension) + 0.4657×(BPStage-II-IV-hypertension) + 0.5963×Diabetes + 0.2925×(Smoker)-9.9255

                              

                            S 0 (1) b =0.9984, S 0 (2) = 0.9933, S 0 (3) = 0.9909, S 0 (4) = 0.9858, S 0 (5) = 0.98297.

                            Framingham-Secondary

                            Male

                            http://static-content.springer.com/image/art%3A10.1186%2F1472-6823-12-12/MediaObjects/12902_2012_133_IEq5_HTML.gif where m = 4.995–0.0145×age–0.6738× log e (TC a /HDL)-0.3042×Diabetes.

                            D’Agostino et al. [27]

                              
                             

                            Female

                            http://static-content.springer.com/image/art%3A10.1186%2F1472-6823-12-12/MediaObjects/12902_2012_133_IEq6_HTML.gif where m = 13.537-0.0225×age0.834×log e (TC a /HDL)-1.3713×ln(SBP)-0.7829×Diabetes-0.3669×smoker.

                            a TC = Total Cholesterol.

                            b S 0 (t) represents the estimate of the t-year baseline survival rate, provided by Framingham investigators, for t = 1,2,…,5 years.

                            Risk score calculation and statistical methods

                            For each eligible participant, we calculated the absolute risk of a CHD event using each equation. Because the racial/ethnic composition of the TRIAD cohort differed from that in the UKPDS cohort, we used the “Afro-Caribbean” risk adjustment for African American patients and the “Caucasian or Asian-Indian” calculation adjustment for the remaining participants. Framingham risk equations were not adjusted for race/ethnicity. Because the Framingham-initial equations were published with the 10-year baseline survival rates, we obtained the 1–5 year baseline survival rates directly from the Framingham investigators.

                            We evaluated the risk equations for 1) how well they separate individuals who develop a CHD event from those who do not (discrimination) and 2) how close predicted risks are to observed risks [6, 31] (calibration, or goodness-of-fit (GOF)). When we examined the performance of the UKPDS and Framingham-initial CHD equations, we only included patients without a CHD history; when we examined the performance of the Framingham-secondary equations, we only included patients with a CHD history.

                            Discrimination was evaluated using the Harrell’s c-index for censored data (R package HMISC available on CRAN at http://​cran.​r-project.​org), a statistic similar to the area under a receiver operating characteristic curve [32]. In general, a c-index greater than 0.7 indicates good discrimination while a value of 0.5 indicates discrimination equivalent to chance. Intermediate values indicate limited discriminating utility. Calibration plots were generated and Hosmer-Lemeshow-type chi-square statistics [6, 33] were calculated to compare differences between predicted and observed risks based on deciles of risk scores. We conservatively defined lack of calibration as chi-square values greater than 23.2 (the 99th percentile of chi-square distribution with 10 degrees of freedom). We also recalibrated the UKPDS and Framingham risk equations by replacing the average values of predictors and event rates in the original populations by those in the TRIAD population. Specifically, we used the method of D’Agostino et al.[6] to recalibrate the Framingham-initial equations and the method of van Houwelingen [34] to recalibrate the UKPDS and Framingham-secondary equations because the latter were parametric models.

                            To investigate the difference between study populations with regard to the effect of risk predictors, we fitted each of these equations on TRIAD data and compared the estimates of relative risk (hazard ratio) using the method described in D’Agostino et al. [6]. Specifically, we fitted the Cox regression models and used the same CHD event definition as well as the risk predictors in the original equations. For simplicity, the models using the TRIAD data were all referred to as the TRIAD models. Regression coefficients, hazard ratio (HR) estimates, Harrell’s c-index and GOF statistics [35] were calculated.

                            Missing data ranged from 1.3% (smoking) to 20.7% (HDL), and was handled in the data analysis using multiple imputation. Imputations were generated using a sequential regression imputation method via the software package IVEware, and results were combined using Rubin’s rule implemented in SAS v9.2 MIANALYZE procedure [3638].

                            Results and discussion

                            Results

                            The follow-up time ranged from 0–3.5 years with a median of 2.7 years (inter-quartile range = 0.9). In the overall TRIAD sample (n = 8303 subjects), there were 319 UKPDS-defined and 592 Framingham-defined CHD events. Among those without a history of CHD (n = 5914), there were 120 UKPDS-defined and 218 Framingham-defined CHD events over the analysis period, with a corresponding 3.5-year CHD event rate (Kaplan-Meier estimate) of 3.0% (95%CI: 2.3%, 3.8%) and 5.1% (95%CI: 4.2%, 6.0%), respectively. Mean age at baseline was 59.8 (SD = 12.3) years with a mean diabetes duration of 10.6 (SD = 9.1) years; 56.7% were female, 39.7% were non-Hispanic White, and nearly 18% were smokers. Of those for whom we had information on medication use and co-morbid burden (n = 4602), 78.4% took oral medication, 26.4% took insulin, 68.7% took hypertension medication, 28% took statins, and 50% had a Charlson score > =2. For those with a prior CHD history (n = 2389), there were 199 UKPDS-defined and 374 Framingham-defined CHD events, with a corresponding 3.5-year CHD event rate of 11.9% (95%CI: 9.6%, 14.2%) and 20.9% (95%CI: 17.9%, 23.9%), respectively. Compared to those without a CHD history, they were generally older with longer diabetes duration and better cholesterol control; they also took more hypertension medication, statins, and suffered from a greater number of comorbid conditions (Table 2).
                            Table 2

                            Demographics and clinical characteristics of TRIAD participants

                             

                            Without CHD History

                            With CHD History

                            N

                            5914

                            2389

                            Age at baseline survey (SD) (years)

                            59.8 (12.3)

                            66.1 (10.4)

                            Female (%)

                            56.7

                            44.9

                            Race/Ethnicity (%)

                              

                              Non- Hispanic white

                            39.7

                            51.0

                              Non- Hispanic black

                            17.3

                            15.9

                              Hispanic

                            17.7

                            14.4

                              Hawaiian/Pacific Islander

                            16.6

                            10.5

                              Other

                            8.8

                            8.2

                            Duration (SD) (years)

                            10.6 (9.1)

                            13.6 (10.2)

                            History of CHD eventa (%)

                            0.0

                            100.0

                            HbA1C (SD)

                            8(1.9)

                            7.9(1.8)

                            Total Cholesterol (SD) (mg/dL)

                            200.8 (42.1)

                            191.9 (45.5)

                            HDL Cholesterol (SD) (mg/dL)

                            47.5 (12.9)

                            44.7 (12.8)

                            LDL Cholesterol (SD) (mg/dL)

                            116.1 (34.9)

                            108.8 (36.7)

                            Systolic Blood Pressure (SD) (mmHg)

                            136.4 (18.4)

                            136.7 (19.8)

                            Smoker (%)

                            18.5

                            17.5

                            Diabetes Treatmentb (%)

                              

                              Diet only

                            7.6

                            6.2

                              Oral medication

                            66.0

                            57.5

                              Insulin

                            14.0

                            20.1

                              Insulin and oral medication

                            12.4

                            15.4

                            Other medicationb (%)

                              

                              Hypertension

                            68.7

                            88.5

                              Statin

                            28.0

                            54.6

                            Co-morbidity Charlson’s scoreb (%)

                              

                              <1

                            5.2

                            1.9

                              > = 1 to 2

                            44.0

                            17.4

                              > = 2 to 3

                            28.9

                            24.1

                              > = 3

                            21.9

                            54.6

                            Number of incident UKPDS CHD eventsc

                            120

                            199

                            Number of incident Framingham CHD eventsd

                            218

                            374

                            3.5-year UKPDS CHD event rate (95%CI)

                            3.0%

                            11.9%

                             

                            (2.3%, 3.8%)

                            (9.6%, 14.2%)

                            3.5-year Framingham CHD event rate (95%CI)

                            5.1%

                            20.9%

                             

                            (4.2%, 6.0%)

                            (17.9%, 23.9%)

                            aCHD history was identified in the medical record documentation, if at least one of the following conditions occurred in medical records 3 years prior to the baseline survey: angina, MI, other coronary heart disease or coronary artery disease, coronary angioplasty or bypass.

                            bData were obtained from 4602 patients without a CHD history and 2029 patients with a CHD history.

                            cUKPDS CHD event is defined as fatal or non-fatal MI.

                            dFramingham CHD event is defined as angina pectoris, MI, coronary insufficiency, sudden or non-sudden CHD death.

                            Evaluation of discrimination and calibration of the UKPDS and Framingham risk equations is summarized in Table 3. The Harrell’s c-index of discrimination for the UKPDS risk equations was generally low: 0.63 (95%CI: 0.58, 0.68) for the incident equation and 0.64 (95%CI: 0.59, 0.69) for the duration equation. The (unadjusted) GOF chi-square value for both UKPDS equations exceeded the established cutoff, indicating a lack of calibration. Specifically, these equations tended to over-estimate CHD risk (Figure1). Similar results were found in our evaluation of Framingham-initial and Framingham-secondary risk equations. Specifically, we found that discrimination was generally low, and so was the calibration except the Framingham-initial equation for women and the Framingham-secondary equation for men. When using these equations, risk tended to be over-estimated for men without a CHD history, and under-estimated for women with a CHD history. After recalibration, the goodness-of-fit of all of the risk equations greatly improved as indicated by the adjusted GOF chi-square statistics that are below the cutoff values (Table 3 and Figure1).
                            Table 3

                            Discrimination and calibration of UKPDS and Framingham CHD Risk Equations

                              

                            N (Number of CHD events)

                            Discrimination Harrell’s c-index (95%CI)

                            Calibration GOF Chi-Square Statistics

                                

                            Unadjusteda

                            Adjusteda

                            UKPDS

                            Incident

                            5914 (120)

                            0.63 (0.58, 0.68)

                            55.64

                            2.18

                             

                            Duration

                            5914 (120)

                            0.64 (0.59, 0.69)

                            446.73

                            7.65

                            Framingham-Initial

                            Men

                            2560 (97)

                            0.61 (0.55, 0.67)

                            24.28

                            0.88

                             

                            Women

                            3354 (121)

                            0.59 (0.54, 0.64)

                            17.27

                            5.85

                            Framingham-Secondary

                            Men

                            1317 (209)

                            0.55 (0.51, 0.59)

                            19.45

                            8.52

                             

                            Women

                            1072 (165)

                            0.54 (0.49, 0.58)

                            77.59

                            5.23

                            aUnadjusted and adjusted refer to the GOF chi-square statistics before vs. after recalibration, respectively.

                            http://static-content.springer.com/image/art%3A10.1186%2F1472-6823-12-12/MediaObjects/12902_2012_133_Fig1_HTML.jpg
                            Figure 1

                            Calibration plot of UKPDS and Framingham CHD risk equations. X-axes refer to deciles of predicted risk scores using the UKPDS and Framingham CHD risk equations, where appropriate. Each bar in the graph represents the average of the observed and predicted risk scores from the UKPDS and Framingham risk equations.

                            For the TRIAD version of the UKPDS equations (Table 4), only age and systolic blood pressure remained significantly associated with CHD (p<0.05). HRs associated with age, gender, racial difference, loge(total cholesterol/HDL) and diabetes duration were significantly different, mostly smaller, than those in the original equations (p<0.05). For the TRIAD versions of Framingham initial and secondary equations, almost none of the risk predictors were significantly associated with CHD. Most HR estimates did not significantly differ from those from the original estimates, except for age and HDL for women in the initial equation. However, because the standard errors of regression coefficient estimates for the original Framingham-initial equations were not provided in Wilson et al. [4], thus they were not included in these calculations and the actual p-value might be slightly bigger. The goodness-of-fit of these TRIAD models is generally good (p<0.05), but the discriminating power was low (all the c-indexes were < 0.65).
                            Table 4

                            Regression coefficients, hazard ratio (HR), and goodness-of-fit of TRIAD Models and comparisons of the hazard ratios with the original risk equations

                              

                            TRIAD models

                            Original

                            Model

                            Variable

                            Coeff

                            Standard Error or 95%CI

                            p-value

                            HR

                            HR

                            Incident

                            Age (year)

                            0.038

                            0.008

                            <.0001

                            1.038*

                            1.059

                            UKPDS

                            Female (yes/no)

                            −0.109

                            0.188

                            0.561

                            0.896*

                            0.525

                             

                            African American (yes/no)

                            −0.154

                            0.281

                            0.585

                            0.858*

                            0.390

                             

                            (yes/no)

                            0.440

                            0.235

                            0.061

                            1.552

                            1.350

                             

                            HbA1c

                            0.075

                            0.054

                            0.170

                            1.077

                            1.183

                             

                            SBP

                            0.010

                            0.005

                            0.034

                            1.010

                            1.008

                             

                            loge (TCa/HDL)

                            0.588

                            0.359

                            0.102

                            1.801*

                            3.845

                             

                            C index

                            0.66

                            (0.61, 0.71)

                               
                             

                            p-value for GOF

                            0.48

                                

                            UKPDS

                            Age (year)

                            0.035

                            0.009

                            <.0001

                            1.035*

                            1.059

                             

                            Female (yes/no)

                            −0.122

                            0.188

                            0.517

                            0.885*

                            0.525

                             

                            African American (yes/no)

                            −0.160

                            0.281

                            0.569

                            0.852*

                            0.390

                             

                            Smoker (yes/no)

                            0.450

                            0.235

                            0.055

                            1.568

                            1.350

                             

                            A1c

                            0.066

                            0.055

                            0.234

                            1.068

                            1.183

                             

                            SBP

                            0.010

                            0.005

                            0.037

                            1.010

                            1.008

                             

                            loge (TCa/HDL)

                            0.612

                            0.358

                            0.089

                            1.844

                            3.845

                             

                            Duration (year)

                            0.012

                            0.009

                            0.177

                            1.012*

                            1.078

                             

                            C index

                            0.66

                            (0.61, 0.71)

                               
                             

                            p-value for GOF

                            0.62

                                

                            Framingham

                            -Initial

                            Age (year)

                            0.037

                            0.009

                            <.0001

                            1.038

                            1.049

                            Male

                            TC, mg/dL

                                 
                             

                             <160

                            −0.154

                            0.385

                            0.690

                            0.857

                            0.517

                             

                             160-199

                            -

                            -

                            -

                            -

                            -

                             

                             200–239

                            0.415

                            0.239

                            0.083

                            1.514

                            1.194

                             

                             240-279

                            0.052

                            0.422

                            0.903

                            1.053

                            1.658

                             

                             ≥280

                            −0.228

                            1.055

                            0.829

                            0.796

                            1.929

                             

                            HDL, mg/dL

                                 
                             

                             <35

                            −0.004

                            0.355

                            0.990

                            0.996

                            1.645

                             

                             35-44

                            −0.179

                            0.306

                            0.560

                            0.836

                            1.275

                             

                             45-59

                            -

                            -

                            -

                            -

                            -

                             

                             50-59

                            −0.297

                            0.397

                            0.455

                            0.743

                            0.950

                             

                             ≥60

                            −0.272

                            0.465

                            0.558

                            0.762

                            0.615

                             

                            Blood Pressureb

                                 
                             

                             Optimal

                            −0.513

                            0.459

                            0.264

                            0.599

                            0.998

                             

                             Normal

                            -

                            -

                            -

                            -

                            -

                             

                             High normal

                            −0.059

                            0.326

                            0.857

                            0.943

                            1.327

                             

                             Hypertension stage I

                            0.037

                            0.299

                            0.903

                            1.037

                            1.685

                             

                             Hypertension stage II–IV

                            −0.009

                            0.376

                            0.982

                            0.991

                            1.856

                             

                            Smoker

                            0.216

                            0.273

                            0.429

                            1.241

                            1.688

                             

                            C index

                            0.65

                            (0.59, 0.70)

                               
                             

                            p-value for GOF

                            0.53

                                

                            Framingham

                            -Initial

                            Age (year)

                            0.157

                            0.080

                            0.050

                            1.170*

                            1.402

                            Female

                            Age2 (year)

                            −0.001

                            0.001

                            0.103

                            0.999

                            0.997

                             

                            TC, mg/dL

                                 
                             

                             <160

                            −0.759

                            0.547

                            0.168

                            0.468

                            0.77

                             

                             160-199

                            -

                            -

                            -

                            -

                            -

                             

                             200–239

                            −0.072

                            0.233

                            0.756

                            0.930

                            1.231

                             

                             240-279

                            0.302

                            0.302

                            0.318

                            1.353

                            1.276

                             

                             ≥280

                            0.693

                            0.356

                            0.052

                            1.999

                            1.708

                             

                            HDL, mg/dL

                                 
                             

                             <35

                            0.415

                            0.461

                            0.369

                            1.515

                            2.324

                             

                             35-44

                            0.132

                            0.291

                            0.652

                            1.141

                            1.459

                             

                             50-59

                            0.210

                            0.336

                            0.534

                            1.233

                            1.219

                             

                             50-59

                            -

                            -

                            -

                            -

                            -

                             

                             ≥60

                            0.372

                            0.269

                            0.167

                            1.451**

                            0.651

                             

                            Blood Pressureb

                                 
                             

                             Optimal

                            −0.638

                            0.469

                            0.174

                            0.528

                            0.586

                             

                             Normal

                            -

                            -

                            -

                            -

                            -

                             

                             High normal

                            −0.298

                            0.318

                            0.348

                            0.742

                            0.935

                             

                             Hypertension stage I

                            0.119

                            0.270

                            0.660

                            1.126

                            1.301

                             

                             Hypertension stage II–IV

                            0.340

                            0.301

                            0.259

                            1.404

                            1.593

                             

                            Smoker

                            0.440

                            0.236

                            0.062

                            1.553

                            1.340

                             

                            C index

                            0.67

                            (0.63, 0.72)

                               
                             

                            p-value for GOF

                            0.62

                                

                            -Secondary

                            Age (year)

                            0.006

                            0.007

                            0.426

                            1.005

                            1.015

                            Male

                            loge(TC/HDL)

                            0.604

                            0.261

                            0.023

                            1.827

                            1.962

                             

                            C index

                            0.55

                            (0.51, 0.59)

                               
                             

                            p-value for GOF

                            0.27

                                

                            -Secondary

                            Age (year)

                            0.012

                            0.008

                            0.131

                            1.012

                            1.022

                            Female

                            loge (TC/HDL)

                            0.336

                            0.335

                            0.319

                            1.395

                            2.245

                             

                            loge (SBP)

                            0.793

                            0.560

                            0.157

                            2.248

                            3.780

                             

                            Smoker (yes/no)

                            −0.266

                            0.240

                            0.269

                            0.764

                            1.427

                             

                            C index

                            0.55

                            (0.50, 0.59)

                               
                             

                            p-value for GOF

                            0.23

                                

                            aTC = Total Cholesterol.

                            bBlood Pressure categories (mmHg): Optimal: Systolic < 120, Diastolic <80; Normal: Systolic 120–129, Diastolic <80-84; High normal: Systolic 130–139, Diastolic 85–89; Hypertension stage I: Systolic 140–159, Diastolic 90–99, Hypertension stage II–IV: Systolic ≥ 160, Diastolic ≥100 [4].

                            *p<0.05, **p<0.01, ***P<0.001 in the HR comparisons of the TRIAD models with original models. Note that in comparing HRs with the Framingham-initial equations, the actual p-values may be slightly bigger than those reported here because the standard errors of the original regression coefficient estimates were not provided in original study [4] and they were not included in these calculations.

                            Discussion

                            Our study showed that the UKPDS and Framingham risk equations may be inappropriate for predicting short-term risk of CHD events for adults with long-standing type-2 diabetes. All of these equations exhibited low discriminating power. All except the Framingham-initial equation for women and Framingham-secondary equation for men had low levels of calibration. Our findings were similar to those found in other studies, including van Dieren et al.[9] that evaluated the performance of the UKPDS risk equations by looking at the CHD event risk at 4, 5, 6 and 8 years, and the ADVANCE trial that evaluated the 4-year CHD risk [11]. Separate analyses also showed that the UKPDS and Framingham-initial equations tended to greatly underestimate the risk of a CHD event for patients with a CHD history and discriminating power was consistently low (data not shown).

                            Several factors may explain our findings of low discrimination and calibration of these equations. First, our study participants had an average of diabetes duration longer than ten years and were community-based health plan enrollees. In contrast, the UKPDS cohort was derived from a clinical trial that only included individuals newly diagnosed with diabetes, while the Framingham cohort only included a small proportion of individuals with diabetes. The general health status, patterns of medication use, and presence of other CHD risk factors in the TRIAD cohort (Table 1) likely differ from those in the previous studies. For instance, at least 68% of TRIAD participants received anti-hypertensive therapy, while fewer than 30% of UKPDS participants and fewer than 10% of Framingham participants had received anti-hypertensive therapy [4, 39]. Moreover, the participants in TRIAD were on average 10 years older than those in the UKPDS and Framingham cohorts [3, 4]. Risk estimates of these equations may need adjustment when applied to older patients because older patients generally are more susceptible to cardiovascular risks, such as higher blood pressure and declining levels of physical activity. They also tend to have more co-morbid conditions, both physically and mentally; some co-morbid conditions may even lead to non-cardiovascular deaths as competing risks [4042]. In addition, the racial/ethnic composition of our cohort included greater numbers of non-white patients than the UKPDS or Framingham cohorts. Risk profiles of minority groups not specified by the algorithms (e.g., Hispanics and Asians other than South Asians) may be significantly different. Finally, UKPDS and Framingham risk equations were developed from cohorts formed between the 1970s and the 1990s [3, 4]. Treatment of type 2 diabetes and management of cardiovascular risk among diabetic adults have improved substantially since then [4346].

                            In investigating the relations of these risk equations with the risk of CHD events in the TRIAD cohort, we found that most predictors were not statistically significant in the TRIAD models. This may be caused by patterns of medication use (e.g., hypertensive drugs, diabetes treatment and statins; Table 1), comorbid conditions, or other factors unaccounted for in the risk equations. Some recent algorithms do include medication as a predictor, such as the prediction model of Donnan et al. [10]. To improve the discriminating power of these models, additional variables, such as medications, family history, life-style related risk factors, socioeconomic status, co-morbid conditions, and novel biomarkers [20, 26, 45, 47, 48], may need to be incorporated in the risk equations.

                            Recent studies suggest that using a “blanket” approach or aggressive risk factor modification (e.g., lowering LDL and/or blood pressure), based on the public notion that diabetes is a CHD risk equivalent, may lead to an overly aggressive treatment and thus offset a patient’s net benefit from treatment [13, 14, 49]. These authors instead recommend that patients be treated based on appropriate personalized CHD risk estimates. Our analysis showed that the UKPDS and the Framingham-initial equation for men tend to overestimate the initial CHD risk for diabetic patients in a contemporary cohort. Refinement of these equations to reflect the CHD prognostics in a modern diabetic cohort are needed to provide reliable risk estimates to inform effective treatment.

                            The strength of our study is the large sample size. However, our study has some limitations. The longest follow-up time for CHD events in our study was 3.5 years, thus limiting our ability to evaluate the use of these equations to predict longer-term CHD event risk. With longer follow-up (e.g., 10 years), it is possible that these equations may provide better predictions of CHD risks. However, our study population is more susceptible to CHD than the general population, and the average age of our study population tends to be older (> = 60 years). Evaluating short-term CHD risk in this population can provide useful insights for disease management and treatment. Since CHD events were identified mostly through health plan administrative data, identification of these events may not be complete [50], particularly for patients with “silent” infarction who do not seek health care and thus are not represented in claims data. As a consequence, while the numbers of CHD events represent what a health plan should expect from a typical diabetic patient population, it will miss events that are not clinically recognized.

                            Conclusion

                            Our study shows that UKPDS and Framingham CHD risk equations may have limited utility to predict CHD risk for adults with long-standing type-2 diabetes in a U.S. population. It is of both clinical and public health importance to understand the risk levels, risk factors, effective treatment and prevention of the occurrence of a CHD event. Evaluation of these commonly used risk equations for predicting short-term risk of CHD events in this cohort is important in that risk-stratification is frequently used for clinical decision-making, and use of these risk equations are likely to give unreliable risk estimates. In addition, given the high rates of CHD and recurrent CHD events in adults with diabetes, refinement of these risk equations may help to identify high-risk populations that can benefit from public health approaches to risk reduction. The number of adults with long-standing diabetes and associated CHD in the U.S. is high [26, 51]. Our findings highlight the need for new or more refined CHD risk equations to re-assess the CHD event risk and understand factors that influence CHD event risk in adults with prevalent diabetes in a modern U.S. cohort.

                            Declarations

                            Acknowledgements

                            The findings and conclusions in this report are those of the authors and do not necessarily represent the official positions of the Centers for Disease Control or the National Institute of Diabetes and Digestive and Kidney Diseases. Members of the TRIAD STUDY GROUP made significant contributions to this study. We acknowledge the participation of our health plan partners.

                            Authors’ Affiliations

                            (1)
                            School of Public Health, University of Medicine and Dentistry of New Jersey
                            (2)
                            Centers for Disease Control and Prevention
                            (3)
                            Department of Family Medicine and Community Health – Research Division, UMDNJ-Robert Wood Johnson Medical School
                            (4)
                            Department of Internal Medicine/Metabolism, Endocrinology and Diabetes, University of Michigan
                            (5)
                            Division of Research, Kaiser Permanente
                            (6)
                            Pacific Health Research Institute
                            (7)
                            Kaiser Center for Health Research Hawaii
                            (8)
                            Department of Medicine, School of Medicine, Indiana University
                            (9)
                            Department of Medicine, New Jersey Medical School, University of Medicine and Dentistry of New Jersey
                            (10)
                            Division of General Internal Medicine and Health Services Research, School of Medicine, University of California, Los Angeles

                            References

                            1. Kannel WB, McGee DL: Diabetes and glucose tolerance as risk factors for cardiovascular disease: the Framingham study. Diabetes Care 1979,2(2):120–126.PubMedView Article
                            2. Jacoby RM, Nesto RW: Acute myocardial infarction in the diabetic patient: pathophysiology, clinical course and prognosis. J Am Coll Cardiol 1992,20(3):736–744.PubMedView Article
                            3. Stevens RJ, Kothari V, Adler AI, Stratton IM: The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56). Clin Sci (Lond) 2001,101(6):671–679.View Article
                            4. Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB: Prediction of coronary heart disease using risk factor categories. Circulation 1998,97(18):1837–1847.PubMedView Article
                            5. Anderson KM, Wilson PW, Odell PM, Kannel WB: An updated coronary risk profile. A statement for health professionals. Circulation 1991,83(1):356–362.PubMedView Article
                            6. D'Agostino RB, Grundy S, Sullivan LM, Wilson P: Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA 2001,286(2):180–187.PubMedView Article
                            7. Coleman RL, Stevens RJ, Retnakaran R, Holman RR: Framingham, SCORE, and DECODE risk equations do not provide reliable cardiovascular risk estimates in type 2 diabetes. Diabetes Care 2007,30(5):1292–1293.PubMedView Article
                            8. Simmons RK, Coleman RL, Price HC, Holman RR, Khaw KT, Wareham NJ, Griffin SJ: Performance of the UK Prospective Diabetes Study Risk Engine and the Framingham Risk Equations in Estimating Cardiovascular Disease in the EPIC- Norfolk Cohort. Diabetes Care 2009,32(4):708–713.PubMedView Article
                            9. van Dieren S, Peelen LM, Nothlings U, van der Schouw YT, Rutten GE, Spijkerman AM, van der AD, Sluik D, Boeing H, Moons KG, et al.: External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes. Diabetologia 2011,54(2):264–270.PubMedView Article
                            10. Donnan PT, Donnelly L, New JP, Morris AD: Derivation and validation of a prediction score for major coronary heart disease events in a U.K. type 2 diabetic population. Diabetes Care 2006,29(6):1231–1236.PubMedView Article
                            11. Kengne AP, Patel A, Colagiuri S, Heller S, Hamet P, Marre M, Pan CY, Zoungas S, Grobbee DE, Neal B, et al.: The Framingham and UK Prospective Diabetes Study (UKPDS) risk equations do not reliably estimate the probability of cardiovascular events in a large ethnically diverse sample of patients with diabetes: the Action in Diabetes and Vascular Disease: Preterax and Diamicron-MR Controlled Evaluation (ADVANCE) Study. Diabetologia 2010,53(5):821–831.PubMedView Article
                            12. van Dieren S, Beulens JW, Kengne AP, Peelen LM, Rutten GE, Woodward M, van der Schouw YT, Moons KG: Prediction models for the risk of cardiovascular disease in patients with type 2 diabetes: a systematic review. Heart 2012,98(5):360–369.PubMedView Article
                            13. Bulugahapitiya U, Siyambalapitiya S, Sithole J, Idris I: Is diabetes a coronary risk equivalent? Systematic review and meta-analysis. Diabet Med 2009,26(2):142–148.PubMedView Article
                            14. Timbie JW, Hayward RA, Vijan S: Variation in the net benefit of aggressive cardiovascular risk factor control across the US population of patients with diabetes mellitus. Arch Intern Med 2010,170(12):1037–1044.PubMedView Article
                            15. Conroy RM, Pyorala K, Fitzgerald AP, Sans S, Menotti A, De Backer G, De Bacquer D, Ducimetiere P, Jousilahti P, Keil U, et al.: Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 2003,24(11):987–1003.PubMedView Article
                            16. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P: Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ 2007,335(7611):136.PubMedView Article
                            17. Hippisley-Cox J, Coupland C, Robson J, Brindle P: Derivation, validation, and evaluation of a new QRISK model to estimate lifetime risk of cardiovascular disease: cohort study using QResearch database. BMJ 2010, 341:c6624.PubMedView Article
                            18. Ridker PM, Buring JE, Rifai N, Cook NR: Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 2007,297(6):611–619.PubMedView Article
                            19. Grundy SM, Balady GJ, Criqui MH, Fletcher G, Greenland P, Hiratzka LF, Houston-Miller N, Kris-Etherton P, Krumholz HM, LaRosa J, et al.: Primary prevention of coronary heart disease: guidance from Framingham: a statement for healthcare professionals from the AHA Task Force on Risk Reduction. American Heart Association. Circulation 1998,97(18):1876–1887.PubMedView Article
                            20. Grundy SM, Cleeman JI, Merz CN, Brewer HB, Clark LT, Hunninghake DB, Pasternak RC, Smith SC, Stone NJ: Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III guidelines. Circulation 2004,110(2):227–239.PubMedView Article
                            21. Type 2 diabetes: national clinical guideline for management in primary and secondary care (update). [Cited 17 November 2011]; Available from:www.​nice.​org.​uk/​nicemedia/​pdf/​CG66FullGuidelin​e0509.​pdf
                            22. Cardiovascular disease—primary prevention. [Cited 17 November 2011]; Available from:www.​bcguidelines.​ca/​gpac/​pdf/​cvd.​pdf
                            23. Ryden L, Standl E, Bartnik M, Van den Berghe G, Betteridge J, de Boer MJ, Cosentino F, Jonsson B, Laakso M, Malmberg K, et al.: Guidelines on diabetes, pre-diabetes, and cardiovascular diseases: executive summary. The Task Force on Diabetes and Cardiovascular Diseases of the European Society of Cardiology (ESC) and of the European Association for the Study of Diabetes (EASD). Eur Heart J 2007,28(1):88–136.PubMedView Article
                            24. van der Heijden AA, Ortegon MM, Niessen LW, Nijpels G, Dekker JM: Prediction of coronary heart disease risk in a general, pre-diabetic, and diabetic population during 10 years of follow-up: accuracy of the Framingham, SCORE, and UKPDS risk functions: The Hoorn Study. Diabetes Care 2009,32(11):2094–2098.PubMedView Article
                            25. Davis WA, Colagiuri S, Davis TM: Comparison of the Framingham and United Kingdom Prospective Diabetes Study cardiovascular risk equations in Australian patients with type 2 diabetes from the Fremantle Diabetes Study. Med J Aust 2009,190(4):180–184.PubMed
                            26. Roger VL, Go AS, Lloyd-Jones DM, Adams RJ, Berry JD, Brown TM, Carnethon MR, Dai S, de Simone G, Ford ES, et al.: Socioeconomic position and health among persons with diabetes mellitus: a conceptual framework and review of the literature. Epidemiol Rev 2011,123(4):e18-e209.
                            27. D'Agostino RB, Russell MW, Huse DM, Ellison RC, Silbershatz H, Wilson PW, Hartz SC: Primary and subsequent coronary risk appraisal: new results from the Framingham study. Am Heart J 2000,139(2 Pt 1):272–281.PubMedView Article
                            28. The Translating Research Into Action for Diabetes (TRIAD) study; a multicenter study of diabetes in managed care Diabetes Care 2002,25(2):386–389.
                            29. Charlson ME, Pompei P, Ales KL, MacKenzie CR: A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987,40(5):373–383.PubMedView Article
                            30. de Groot V, Beckerman H, Lankhorst GJ, Bouter LM: How to measure comorbidity. a critical review of available methods. J Clin Epidemiol 2003,56(3):221–229.PubMedView Article
                            31. Harrell FE, Lee KL, Mark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996,15(4):361–387.PubMedView Article
                            32. Harrell FE: Regression modeling strategies : with applications to linear models, logistic regression, and survival analysis. Springer, New York; 2001.
                            33. Hosmer DW, Lemeshow S: Applied logistic regression. 2nd edition. Wiley, New York; 2000.View Article
                            34. van Houwelingen HC: Validation, calibration, revision and combination of prognostic survival models. Stat Med 2000,19(24):3401–3415.PubMedView Article
                            35. May S, Hosmer DW: A simplified method of calculating an overall goodness-of-fit test for the Cox proportional hazards model. Lifetime Data Anal 1998,4(2):109–120.PubMedView Article
                            36. Raghunathan T, Lepkowski J, van Hoewyk J, Solenberger P: A MultivariateTechnique for Multiply Imputing Missing Values Using a Sequence of Regression Models. Survey Methodology 2001,27(1):85–95.
                            37. Raghunathan T, Solenberger P, Van Hoewyk J: IVEware: Imputation and variance estimation software user guide. Survey Research Center, Institute for Social Research, Ann Arbor, MI; 2002.
                            38. Rubin DB: Multiple Imputation for Nonresponse in Surveys. Wiley & Sons, New York; 1987.View Article
                            39. Prospective UK: Diabetes Study (UKPDS). VIII. Study design, progress and performance. Diabetologia 1991,34(12):877–890.View Article
                            40. Ahluwalia SC, Gross CP, Chaudhry SI, Leo-Summers L, Van Ness PH, Fried TR: Change in comorbidity prevalence with advancing age among persons with heart failure. J Gen Intern Med 2011,26(10):1145–1151.PubMedView Article
                            41. Ahluwalia SC, Gross CP, Chaudhry SI, Ning YM, Leo-Summers L, Van Ness PH, Fried TR: Impact of Comorbidity on Mortality Among Older Persons with Advanced Heart Failure. J Gen Intern Med 2012,27((5)):513–519.PubMedView Article
                            42. Merikangas KR, Ames M, Cui L, Stang PE, Ustun TB, Von Korff M, Kessler RC: The impact of comorbidity of mental and physical conditions on role disability in the US adult household population. Arch Gen Psychiatry 2007,64(10):1180–1188.PubMedView Article
                            43. Grundy SM, Benjamin IJ, Burke GL, Chait A, Eckel RH, Howard BV, Mitch W, Smith SC, Sowers JR: Diabetes and cardiovascular disease: a statement for healthcare professionals from the American Heart Association. Circulation 1999,100(10):1134–1146.PubMedView Article
                            44. Buse JB, Ginsberg HN, Bakris GL, Clark NG, Costa F, Eckel R, Fonseca V, Gerstein HC, Grundy S, Nesto RW, et al.: Primary prevention of cardiovascular diseases in people with diabetes mellitus: a scientific statement from the American Heart Association and the American Diabetes Association. Circulation 2007,115(1):114–126.PubMedView Article
                            45. Williams MA, Fleg JL, Ades PA, Chaitman BR, Miller NH, Mohiuddin SM, Ockene IS, Taylor CB, Wenger NK: Secondary prevention of coronary heart disease in the elderly (with emphasis on patients > or =75 years of age): an American Heart Association scientific statement from the Council on Clinical Cardiology Subcommittee on Exercise, Cardiac Rehabilitation, and Prevention. Circulation 2002,105(14):1735–1743.PubMedView Article
                            46. Greenland P, Alpert JS, Beller GA, Benjamin EJ, Budoff MJ, Fayad ZA, Foster E, Hlatky MA, Hodgson JM, Kushner FG, et al.: ACCF/AHA guideline for assessment of cardiovascular risk in asymptomatic adults: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2010,56(25):e50-e103.PubMedView Article
                            47. Brown AF, Ettner SL, Piette J, Weinberger M, Gregg E, Shapiro MF, Karter AJ, Safford M, Waitzfelder B, Prata PA, et al.: Socioeconomic position and health among persons with diabetes mellitus: a conceptual framework and review of the literature. Epidemiol Rev 2004, 26:63–77.PubMedView Article
                            48. Weiner DE, Tighiouart H, Stark PC, Amin MG, MacLeod B, Griffith JL, Salem DN, Levey AS, Sarnak MJ: Kidney disease as a risk factor for recurrent cardiovascular disease and mortality. Am J Kidney Dis 2004,44(2):198–206.PubMedView Article
                            49. Ray KK, Seshasai SR, Erqou S, Sever P, Jukema JW, Ford I, Sattar N: Statins and all-cause mortality in high-risk primary prevention: a meta-analysis of 11 randomized controlled trials involving 65,229 participants. Arch Intern Med 2010,170(12):1024–1031.PubMedView Article
                            50. Heckbert SR, Kooperberg C, Safford MM, Psaty BM, Hsia J, McTiernan A, Gaziano JM, Frishman WH, Curb JD: Comparison of self-report, hospital discharge codes, and adjudication of cardiovascular events in the Women’s Health Initiative. Am J Epidemiol 2004,160(12):1152–1158.PubMedView Article
                            51. Centres for Disease Control and Prevention. National diabetes fact sheet: national estimates and general information on diabetes and prediabetes in the United States, 2011. Department of Health and Human Services, Centres for Disease Control and Prevention, 2011, Atlanta, GA: US; 2011. [Cited 7 June 2011]; Available from:http://​www.​cdc.​gov/​diabetes/​pubs/​pdf/​ndfs_​2011.​pdf
                            52. Pre-publication history

                              1. The pre-publication history for this paper can be accessed here:http://​www.​biomedcentral.​com/​1472-6823/​12/​12/​prepub

                            Copyright

                            © Lu et al.; licensee BioMed Central Ltd. 2012

                            This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://​creativecommons.​org/​licenses/​by/​2.​0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.