- Research article
- Open Access
- Open Peer Review
This article has Open Peer Review reports available.
Validation of ICD-9-CM coding algorithm for improved identification of hypoglycemia visits
© Ginde et al; licensee BioMed Central Ltd. 2008
Received: 04 December 2007
Accepted: 01 April 2008
Published: 01 April 2008
Accurate identification of hypoglycemia cases by International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes will help to describe epidemiology, monitor trends, and propose interventions for this important complication in patients with diabetes. Prior hypoglycemia studies utilized incomplete search strategies and may be methodologically flawed. We sought to validate a new ICD-9-CM coding algorithm for accurate identification of hypoglycemia visits.
This was a multicenter, retrospective cohort study using a structured medical record review at three academic emergency departments from July 1, 2005 to June 30, 2006. We prospectively derived a coding algorithm to identify hypoglycemia visits using ICD-9-CM codes (250.3, 250.8, 251.0, 251.1, 251.2, 270.3, 775.0, 775.6, and 962.3). We confirmed hypoglycemia cases by chart review identified by candidate ICD-9-CM codes during the study period. The case definition for hypoglycemia was documented blood glucose 3.9 mmol/l or emergency physician charted diagnosis of hypoglycemia. We evaluated individual components and calculated the positive predictive value.
We reviewed 636 charts identified by the candidate ICD-9-CM codes and confirmed 436 (64%) cases of hypoglycemia by chart review. Diabetes with other specified manifestations (250.8), often excluded in prior hypoglycemia analyses, identified 83% of hypoglycemia visits, and unspecified hypoglycemia (251.2) identified 13% of hypoglycemia visits. The absence of any predetermined co-diagnosis codes improved the positive predictive value of code 250.8 from 62% to 92%, while excluding only 10 (2%) true hypoglycemia visits. Although prior analyses included only the first-listed ICD-9 code, more than one-quarter of identified hypoglycemia visits were outside this primary diagnosis field. Overall, the proposed algorithm had 89% positive predictive value (95% confidence interval, 86–92) for detecting hypoglycemia visits.
The proposed algorithm improves on prior strategies to identify hypoglycemia visits in administrative data sets and will enhance the ability to study the epidemiology and design interventions for this important complication of diabetes care.
Hypoglycemia is an important complication in patients with diabetes that has a profound impact on quality of life and self-management. While tight glycemic control remains a hallmark to lower rates of complications [1–3], the barrier of hypoglycemia is the major limiting factor in the maintenance of euglycemia . Mild or self-treated episodes of hypoglycemia are common, especially in type 1 diabetes, with reported rates of two episodes per week . Severe hypoglycemia, or episodes requiring external assistance, may occur at least once a year, and are a significant cause of morbidity [5–7].
Although some episodes of severe hypoglycemia are treated at home by administration of oral glucose or intramuscular glucagon by family members, the most severe episodes require ambulance or emergency department (ED) visits. Although ED visits for severe hypoglycemia represent a small percentage of the total episodes of hypoglycemia in diabetes, they do serve as a good epidemiological marker of the complication and result in significant economical and psychological costs .
Most data on the epidemiology of hypoglycemia are based on highly selected patients in the setting of large clinical trials [1–3, 7], and may not generalize to the general population. Accurate identification of hypoglycemia cases by International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes may help to describe the epidemiology, monitor trends, and propose interventions to improve diabetes care through national administrative databases and local chart reviews . There are several ICD-9-CM coding options for hypoglycemia, some of which may represent other diagnoses. For example, diabetes with other specified manifestations (250.8) may represent hypoglycemia or diabetic lower extremity ulcers. While several prior studies have utilized ICD-9-CM codes to identify cases of hypoglycemia [10–14], only Johnson et al. described the accuracy of ICD-9-CM codes to identify hypoglycemia episodes, but with significant methodological limitations .
Building on this prior work, we evaluated the accuracy a new ICD-9-CM coding algorithm to identify cases of hypoglycemia presenting to the ED. We hypothesized that our proposed algorithm would identify hypoglycemia with a positive predictive value (PPV) of at least 80%.
This was a multicenter, retrospective cohort study using a structured medical record review for validation of a prospectively derived ICD-9 coding algorithm. We obtained ethics approval with a waiver of informed consent from the Partners Human Research Committee and the Beth Israel Deaconess Medical Center Committee on Clinical Investigation.
Study setting and population
This study was conducted at three urban, academic EDs, which are active participants in the Emergency Medicine Network . The EDs have a combined annual visit volume of 175,000 and are staffed by emergency medicine, internal medicine, and surgery residents, and patient care is supervised by attending emergency physicians 24 hours/day.
Using the electronic medical records system at each site, we searched the following ICD-9-CM codes to identify possible visits for hypoglycemia: 250.3 (diabetes with other coma), 250.8 (diabetes with other specified manifestations) 251.0 (hypoglycemic coma), 251.1 (other specified hypoglycemia), 251.2 (hypoglycemia, unspecified), 270.3 (leucine-induced hypoglycemia), 775.0 (hypoglycemia in an infant born to a diabetic mother), 775.6 (neonatal hypoglycemia), and 962.3 (poisoning by insulins and antidiabetic agents).
Given the diversity of potential ICD-9-CM codes, we searched this broad range of codes and in all diagnosis fields (up to ten listed) in an attempt to capture all possible ED hypoglycemia visits. For admitted patients, we examined only ED-based codes, to avoid inclusion of incident hypoglycemia that occurred during inpatient hospitalization. In cases where multiple candidate codes were present, we recorded only the first-listed code. The exception to this was for the more ambiguous codes 250.3 and 250.8, for which we preferentially recorded any of the other candidate codes if present. We based this strategy on detailed examination of the ICD-9-CM coding manual , review of the experience from previously reported approaches [10–14], and discussion with coding experts.
We identified all ED visits with candidate ICD-9-CM codes between July 1, 2005 and June 30, 2006 at each site, and obtained written ED charts. For patients with multiple ED visits during the data collection period, we requested only the first visit to avoid overrepresentation by certain patients. Trained research staff abstracted all charts using a standardized form, and the research group met weekly to discuss data collection and resolve abstraction issues. Additionally, two reviewers independently abstracted 10% of charts to evaluate inter-rater agreement in data collection. To enhance the reliability of our chart review, we abstracted only charts with complete ED triage assessment, nursing notes, and emergency physician notes and considered all other charts incomplete.
Key Outcome Measures
We considered this chart validation as the gold standard for confirmation of true hypoglycemia visits. In reviewing the ED chart, we confirmed cases of hypoglycemia based on the following criteria: 1) Any documented pre-hospital or ED glucose value (serum or capillary) 3.9 mmol/l, or 2) Charted physician discharge diagnosis of hypoglycemia. We based the glucose threshold on the consensus recommendation defined by the American Diabetes Association Workgroup on Hypoglycemia . We included physician diagnosis of hypoglycemia to capture cases in which hypoglycemia resolved prior to first glucose level, i.e. patients receiving glucose for symptoms consistent with hypoglycemia prior to blood glucose determination. Additionally, we collected patient disposition (discharge or hospital admission) to evaluate for differences in coding accuracy based on this factor.
We performed statistical analyses using Stata 9.0 (College Station, TX). We summarized data using basic descriptive statistics. We determined accuracy of specific codes and coding algorithm by calculating PPV with 95% confidence intervals (CI). Although we included a broad range of possible ICD-9-CM codes, the numbers of missed hypoglycemia visits, not captured by the candidate codes, were unknown. Under the ideal assumption that no cases were missed, we calculated estimated point estimates for sensitivity, specificity, and negative predictive values (NPV). Additionally, we performed a sensitivity analysis by increasing the presumed number of missed cases by 10%.
We determined inter-rater agreement for chart abstraction by calculating the kappa statistic for the subgroup of charts abstracted by two investigators. Additionally, we compared accuracy of the algorithm stratified by candidate ICD-9-CM code position, study site and ED disposition using chi-squared test and with two-tailed p < 0.05 considered statistically significant. Finally, we evaluated the accuracy of all ICD-9-CM codes to identify potential improvements and calculated the accuracy of the revised algorithm including the proposed modifications.
Of the 174,134 ED visits at the three institutions during the data collection period, we identified 901 patients with candidate ICD-9-CM codes. These patients accounted for 1,139 visits with possible hypoglycemia (i.e., 0.5% of all ED visits). We abstracted 679 charts (75% of visits by unique patients) and excluded the remaining 222 patients with missing or incomplete documentation. There were no significant differences in age, sex, race/ethnicity, and distribution of candidate ICD-9-CM codes comparing abstracted and excluded patients (all p > 0.05).
Predictive value of ICD-9-CM codes for identification of hypoglycemia
Confirmed Cases (%)
Accuracy of co-diagnosis codes to exclude hypoglycemia for ICD-9-CM code 250.8
Valid exclusion (%)
681.xx (cellulitis fingers/toes)
682.xx (other cellulitis)
686.90 (local skin infection)
707.xx (ulcers of lower limb)
730.27 (osteomyelitis ankle/foot)
731.8 (other osteomyelitis)
Overall, the proposed algorithm had PPV of 89% (95%CI, 86–92) for detecting hypoglycemia visits. There was no significant difference in PPV when data were stratified by site or by ED disposition (p = 0.86 and 0.22, respectively). Based on the assumption that the candidate ICD-9-CM codes captured all ED presentations with hypoglycemia, we estimate that the algorithm had 97% sensitivity, >99% specificity, and >99% NPV. Assuming the algorithm missed at most 10% of hypoglycemia visits, we estimate a minimum sensitivity of 88% with NPV and specificity remaining >99%.
Association between diagnostic code position and identification of hypoglycemia
Total confirmed hypoglycemia cases
4th or higher
Post-hoc review of the algorithm performance suggested that cases identified by 962.3 demonstrated low PPV (54%), primarily representing patients with excessive insulin or oral hypoglycemic use without development of clinical or laboratory hypoglycemia. We evaluated the full list of ICD-9-CM codes for all visits coded as 962.3 and found that 10 of 13 confirmed cases of hypoglycemia had 250.8 listed as a co-diagnosis and only 1 of 11 cases without hypoglycemia had 250.8 as a co-diagnosis. When we removed 962.3 as a candidate code and relied only on coding for 250.8 in these cases, the PPV for the overall algorithm increased from 89% to 91% (95%CI, 88–93) while the estimated sensitivity remained at 97%.
In this study, we demonstrated high predictive value for detection of ED hypoglycemia visits using the new ICD-9-CM coding algorithm. To our knowledge, this is the first chart validation of the accuracy of ICD-9-CM codes for hypoglycemia, an important complication in diabetes and common ED presentation.
Our algorithm differs in several important ways from prior reports on hypoglycemia. Two separate analyses of the relationship between anti-hypertensive agents and hypoglycemia in patients with diabetes, Herings et al. and Morris et al. used ICD-9-CM codes 251.0, 251.1, and 251.2 to identify possible episodes of hypoglycemia [12, 13]. Schorr et al. broadened candidate codes to include 250.3 and 962.3 [11, 14]. Notably, none of these studies included 250.8 in their search strategies, nor accounted these possible missed episodes of hypoglycemia. In our analysis, 250.8 comprised the vast majority (83%) of hypoglycemia cases, and exclusion of this code in these prior studies likely contributed to substantial underestimates in the incidence of hypoglycemia.
Johnson, et al. first reported the utility of 250.8 in their strategy to identify ED visits for hypoglycemia . In their analysis, this code also identified over half of hypoglycemia episodes and had a PPV of 73% for their case definition of hypoglycemia. Our chart validation differed in several important ways. First, Johnson et al. limited their search to first-listed ICD-9-CM codes and excluded all other diagnosis fields. We evaluated all diagnosis fields and while candidate codes in the first diagnosis field had a higher PPV, limitation to the primary diagnosis field would have excluded 27% of hypoglycemia cases. Additionally, candidate codes in the latter diagnosis fields maintained a high PPV (81%). Secondly, their validation of hypoglycemia cases involved searching of brief communications following the ED visit for symptoms consistent with hypoglycemia, which included confusion, loss of consciousness, and seizure. While these symptoms are consistent with hypoglycemia, they are nonspecific and have many other causes in addition to hypoglycemia . We performed detailed chart reviews to confirm hypoglycemia visits, which allowed for more precise validation. Finally, we confirmed the importance of 250.8 in identification of the majority of hypoglycemia visits, but additionally refined the search strategy by exclusion of visits with alternate co-diagnoses. Exclusion of these cases reliably eliminated 85% of false-positive cases coded as 250.8, while eliminating only a small number of true-positive cases.
Overall, 250.8 with exclusion of alternate co-diagnoses and 251.2 comprised >95% of hypoglycemia identified by the algorithm. Although there were insufficient data to evaluate 250.0 and 250.1 in this analysis, we recommend inclusion of these codes in any search for hypoglycemia. These codes are likely to have high specificity/PPV, and their utilization may be higher in different institutions and practice environments. As anticipated, codes 270.3, 775.0 and 775.6 were rare in the ED setting but may have utility for other settings, such as maternity and neonatal units.
We hypothesized that we would not be able to determine if diabetes with other coma (250.3) were caused by hypoglycemia or diabetic ketoacidosis (DKA). There were insufficient data to evaluate this hypothesis (only one case), and charts identified with this code should be interpreted with caution. Compared to prior 100% PPV in prior analysis , we were surprised by the relatively low predictive value (54%) of 962.3. Based on this finding and our secondary analysis, we recommend exclusion of 962.3 and relying only on 250.8 to identify most of these associated visits. This proposed strategy will require validation in other institutions and settings.
The ICD-9-CM classification system is imperfect for case identification, as it was created for reimbursement rather than research purposes . Given this inherent limitation, the proposed algorithm for hypoglycemia compares favorably to those suggested for a variety of diseases. ICD-9-CM coding accuracy for upper gastrointestinal disorders (93–95% PPV) and Clostridium difficile colitis (87% PPV) were similar [19, 20], but were more unreliable for stroke (61–79% PPV) , soft tissue disorders (64% PPV) , pneumococcal pneumonia (58% PPV) , traumatic brain injury (20–38% PPV) , and venous thromboembolism (31% PPV) . Based on these comparisons, the proposed ICD-9-CM coding algorithm for hypoglycemia would be expected to perform as well or better than coding strategies for other diagnoses in accurate case identification from administrative data.
The current study has some potential limitations. We did not abstract the 25% of charts that were missing or incomplete. While this lowered the number of total cases, the distribution of ICD-9-CM codes was similar in abstracted and non-abstracted charts, and the likelihood of biased accuracy estimates was small. The coding of hypoglycemia visits was based on three academic EDs and may not generalize to other areas of the hospital, outpatient providers, and other geographic areas. However, there are a finite number of options for coding hypoglycemia, and although the frequency of individual code usage will likely vary based on local practice and familiarity of the coding personnel, we anticipate that accuracy will not vary substantially. We were unable to formally evaluate false-negatives in our data set. Thus, we were only able to calculate PPV with accuracy, and our estimates for sensitivity, specificity, and NPV are significantly limited. To help address this limitation, we performed a sensitivity analysis, which demonstrated a high level of accuracy even when we increased the assumption of missed cases, but inferences based on these values should be guarded. Finally, the accuracy of case confirmation by chart review depended on a retrospective evaluation for hypoglycemia, which may overestimate or underestimate the true number of cases. Standardized definitions and training of reviewer limited the potential for such bias, and high inter-rater agreement demonstrated internal reliability of the chart review.
We derived and validated a new ICD-9-CM coding algorithm to identify hypoglycemia visits with high predictive validity. This algorithm improves on prior strategies to identify hypoglycemia cases in administrative data sets and will enhance the ability to study the epidemiology and design interventions for this important complication of diabetes care.
This study was funded by an investigator-initiated grant from Novo-Nordisk, Inc. The sponsor had no role in the study design, data collection, analysis, or manuscript. AAG was supported by the Emergency Medicine Foundation Research Fellowship Grant. Abstract presented at American College of Emergency Physicians Scientific Assembly; October 8, 2007; Seattle, WA USA
- The Diabetes Control and Complications Trial Research Group: The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993, 329: 977-986. 10.1056/NEJM199309303291401.View ArticleGoogle Scholar
- The United Kingdom Prospective Diabetes Study Research Group: Intensive blood glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes. Lancet. 1998, 352: 837-853. 10.1016/S0140-6736(05)61359-1.View ArticleGoogle Scholar
- The United Kingdom Prospective Diabetes Study Research Group: Effect of intensive blood-glucose control with metformin on complications in overweight patients with type 2 diabetes. Lancet. 1998, 352: 854-865. 10.1016/S0140-6736(05)61359-1.View ArticleGoogle Scholar
- Cryer PE: Hypoglycemia: The limiting factor in the glycaemic management of type I and type II diabetes. Diabetologia. 2002, 45: 937-948. 10.1007/s00125-002-0822-9.View ArticlePubMedGoogle Scholar
- MacLeod KM, Hepburn DA, Frier BM: Frequency and morbidity of severe hypoglycemia in insulin-treated diabetic patients. Diabet Med. 1993, 10: 238-245.View ArticlePubMedGoogle Scholar
- Laing SP, Swerdlow AJ, Slater SD, Botha JL, Burden AC, Waugh NR, Smith AW, Hill RD, Bingley PJ, Patterson CC, Qiao Z, Keen H: The British Diabetic Association Cohort Study II. Cause-specific mortality inpatients with insulin-treated diabetes mellitus. Diabet Med. 1999, 16: 466-471. 10.1046/j.1464-5491.1999.00076.x.View ArticlePubMedGoogle Scholar
- DCCT Research Group: Hypoglycemia in the diabetes control and complications trial. Diabetes. 1997, 46: 271-286. 10.2337/diabetes.46.2.271.View ArticleGoogle Scholar
- Holstein A, Plaschke A, Egberts EH: Incidence and costs of severe hypoglycemia. Diabetes Care. 2002, 25: 2109-10.2337/diacare.25.11.2109.View ArticlePubMedGoogle Scholar
- The International Classification of Diseases, 9th revision. Clinical Modification. 1997, Salt Lake City, UT, Medicode Publications, 5Google Scholar
- Johnson ES, Koepsell TD, Reiber G, Stergachis A, Platt R: Increasing incidence of serious hypoglycemia in insulin users. J Clin Epidemiol. 2002, 55: 253-259. 10.1016/S0895-4356(01)00479-6.View ArticlePubMedGoogle Scholar
- Shorr RI, Ray WA, Daugherty JR, Griffin MR: Incidence and risk factors for serious hypoglycemia in older persons using insulin or sulfonylureas. Arch Intern Med. 1997, 15: 1681-1687. 10.1001/archinte.157.15.1681.View ArticleGoogle Scholar
- Herings RMC, de Boer A, Stricker BHCh, Leufkens HGM, Porsuis A: Hypoglycemia associated with use of inhibitors of angiotensin converting enzyme. Lancet. 1995, 345: 1195-1198. 10.1016/S0140-6736(95)91988-0.View ArticlePubMedGoogle Scholar
- Morris AD, Boyle DIR, McMahon AD, Pearce H, Evans JM, Newton RW, Jung RT, MacDonald TM: ACE inhibitor use is associated with hospitalization for severe hypoglycemia in patients with diabetes. Diabetes Care. 1997, 20: 1363-1367. 10.2337/diacare.20.9.1363.View ArticlePubMedGoogle Scholar
- Shorr RI, Ray WA, Daugherty JR, Griffin MR: Antihypertensives and the risk of serious hypoglycemia in older persons using insulin or sulfonylureas. JAMA. 1997, 278: 40-43. 10.1001/jama.278.1.40.View ArticlePubMedGoogle Scholar
- The Emergency Medicine Network. [http://www.emnet-usa.org/]
- American Diabetes Association Workgroup on Hypoglycemia: Defining and reporting hypoglycemia in diabetes. Diabetes Care. 2005, 28: 1245-1249. 10.2337/diacare.28.5.1245.View ArticleGoogle Scholar
- Cryer PE, Davis SN, Shamoon H: Hypoglycemia in diabetes. Diabetes Care. 2003, 26: 1902-1912. 10.2337/diacare.26.6.1902.View ArticlePubMedGoogle Scholar
- O'Malley KJ, Cook KF, Price MD, Wildes KR, Hurdle JF, Ashton CM: Measuring diagnoses: ICD code accuracy. HSR. 2005, 40: 1620-1639.PubMedPubMed CentralGoogle Scholar
- Lopushinsky SR, Covarrubia KA, Rabeneck L, Austin PC, Urbach DR: Accuracy of administrative health data for the diagnosis of upper gastrointestinal diseases. Surg Endo. 2007, 21 (10): 1733-1737. 10.1007/s00464-006-9136-1.View ArticleGoogle Scholar
- Scheurer DB, Hicks LS, Cood EF, Schnipper JL: Accuracy of ICD-9 coding for Clostridium difficile infections: a retrospective cohort. Epidemiol Infect. 2007, 135: 1010-1013. 10.1017/S0950268806007655.View ArticlePubMedGoogle Scholar
- Goldstein JB: Accuracy of ICD-9-CM coding for the identification of patients with acute ischemic stroke: effect of modifier codes. Stroke. 1998, 29: 1602-1604.View ArticlePubMedGoogle Scholar
- Buchbinder R, Goel V, Bombardier C: Lack of concordance between the ICD-9 classification of soft tissue disorders of the neck and upper limbs and chart review diagnosis: One steel mill's experience. Am J Ind Med. 1996, 29: 171-182. 10.1002/(SICI)1097-0274(199602)29:2<171::AID-AJIM7>3.0.CO;2-V.View ArticlePubMedGoogle Scholar
- Guevara RE, Butler JC, Marston BJ, Plouffe JF, File TM, Breiman RF: Accuracy of ICD-9-CM codes in detecting community-acquired pneumococcal pneumonia for incidence and vaccine efficacy studies. Am J Epidemiol. 1999, 149: 282-289.View ArticlePubMedGoogle Scholar
- Bazarian JJ, Veazie P, Mookerjee S, Lerner EB: Accuracy of Mild Traumatic Brain Injury Case Ascertainment Using ICD-9-CM Codes. Acad Emerg Med. 2006, 13: 31-38. 10.1111/j.1553-2712.2006.tb00981.x.View ArticlePubMedGoogle Scholar
- White RH, Brickner LA, Scannell KA: ICD-9-CM codes poorly identified venous thromboembolism during pregnancy. J Clin Epidemiol. 2004, 57: 985-988. 10.1016/j.jclinepi.2004.02.003.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6823/8/4/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.