| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
ORIGINAL RESEARCH |
From the MetroHealth Medical Center, Case Western Reserve University, Cleveland, Ohio; and University of North Carolina, Chapel Hill, North Carolina.
Address reprint requests to: Jennifer Bailit, MD, MPH, Department of OB/GYN, MetroHealth Medical Center, 2500 Metro-Health Drive, Cleveland, OH 44109-1998; E-mail: jbailit{at}metrohealth.org.
 |
ABSTRACT |
|---|
TOP
 ABSTRACT MATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
METHODS: We used 2 years of Washington State Birth Events Record Data (1997 and 1998) to estimate hospitals’ risk-adjusted cesarean delivery rates using two different methods: 1) logistic regression modeling and 2) direct standardization. After exclusions, there were 123,850 births and 67 hospitals. Ranked lists of hospitals were produced by each methodology and compared using the Spearman correlation. We used 
statistics to compare the top 25% and the bottom 25% of the rankings.
RESULTS: The Spearman correlation for the ranked lists was strong (.84, P < .001). The s were .67 for the top 25% and .69 for the bottom 25%. By the logistic regression method, 19 hospitals had rates significantly higher than expected and 15 had rates significantly lower than expected. Because the direct standardization method had 57% of hospitals with no births in at least one of the risk strata, we could not determine whether these hospitals were statistical outliers.
CONCLUSION: Both methods ranked hospitals similarly. If cesarean delivery rate risk adjustment for all hospitals is desirable, the logistic regression method has the advantage of being able to determine if different rates are significantly above or below expected. However, if comparing only two large hospitals is the goal, direct standardization may be simpler to implement, provided all risk strata have at least one delivery.
Patients, providers, and third-party payers often use low rates of cesarean deliveries as a marker for quality obstetric care. This is flawed: If hospital cesarean delivery rates are compared without adjusting for the differences in patient populations, hospitals serving high-risk populations will have high cesarean delivery rates and appear to be providing poor care, even if they are providing top quality care.
Risk adjustment of cesarean delivery rates overcomes the problems of patient variation between hospitals, leaving primarily quality of care as a source of variation. The American College of Obstetricians and Gynecologists (ACOG) recognizes the need for cesarean delivery rate risk adjustment and in 2000 recommended that all cesarean delivery rates be risk adjusted (case-mix adjusted) before making comparisons.1
Having one published uniform risk-adjustment methodology for the nation would facilitate comparisons of quality among hospitals. A standard methodology would help identify hospitals with outcomes that are outside the normal experience, prompting scrutiny to determine what a center is doing right or wrong.
Two basic methodologies of risk adjustment for cesarean delivery exist in the literature. The first is multiple logistic regression. The logistic regression method predicts the number of cesarean deliveries expected in an institution based on the institution’s patient population and compares it with the hospital’s actual cesarean delivery rate. Multiple logistic regression methodology has been widely used for risk adjustment in internal medicine.2–4 The second risk-adjustment methodology, direct standardization method, applies different institution rates to a standard population.5 ACOG chose to recommend this simpler methodology.
Understanding the strengths and weaknesses of each of the methods is an important step in trying to establish a predominant method of risk adjustment. To date, this has been hard to do because the articles on the two methods of risk adjustment have used different data sources.5–8
Given the high costs involved in collecting data from patient charts and readily available birth certificate data from every birth in every hospital in the United States, we believe that birth certificate data currently are the best available data source to risk adjust populations. A recent article by DiGiuseppe et al9 compared primary and birth certificate data as sources of information for cesarean delivery rate risk adjustment. They concluded that use of reliable data elements from birth certificate data yields hospital rankings that are similar to those obtained using primary data sources.
By applying both risk-adjustment methodologies to the same data, we will be able to evaluate differences in the hospital rankings of risk-adjusted cesarean delivery rates based on each of the two methods. If both methods identify the same hospitals as quality outliers, factors such as how easy a method is to use may determine which method should be preferred.
|
MATERIALS AND METHODS |
|---|
|
TOP
ABSTRACTMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Data were obtained, with the approval of the University of North Carolina Committee on the Protection of the Rights of Human Subjects and the Washington State Department of Health Human research review board, from the Birth Events Records Data file. This data file consists of birth certificate data that is linked to maternal and neonatal discharge data. All linkages are done by the Washington State Department of Health using their algorithm.10 The study population consists of all births in Washington State in 1997 and 1998.
There were 165,261 births at 71 hospitals. Some births were excluded from the analyses for several reasons. Deliveries of nonviable infants are typically not at risk for cesarean delivery. Births at less than 24 weeks’ gestation or with weights less than 500 g were considered nonviable for 1997 and 1998 (731). Births outside of the hospital were also not at risk for cesarean delivery (7336). Hospitals having fewer than 50 deliveries in either 1997 or 1998 were excluded, as these were felt likely to be emergency room deliveries at hospitals without a labor and delivery suite and thus were not at risk for a cesarean delivery (468). We removed births with implausible gestational age and birth weight data combinations, such as greater than 3000 g at less than 28 weeks or vaginal deliveries over 7000 g (391). The mode of delivery for infants with major congenital malformations might have been influenced by their anomaly (3240). Four hospitals did not report maternal age for any patient (8701). Patients with a prior uterine scar were dropped because the decision to perform a repeat cesarean delivery is made differently from the decision to do a primary cesarean delivery (14,717). Typically, all patients with a scarred uterus are offered a cesarean delivery and the patient can choose whether or not to have a repeat cesarean delivery. However, the decision to offer a primary cesarean delivery is made by the physician, and thus primary cesarean delivery rates are a better marker for physician behavior than repeat cesarean delivery rates. Finally, patients with missing parity data were dropped (5827). After exclusions, 123,850 births and 67 hospitals remained in the data set.
We then applied the two methods of risk adjustment to the same data. The logistic regression methodology predicts the number of expected cesarean deliveries based on the institution’s patient population and compares it with the hospital’s actual cesarean delivery rate. A logistic regression model was built to predict cesarean delivery from maternal risk factors. The model, based on a 25% random sample of the data, included clinically relevant variables as determined by three perinatologists.11 These independent variables included maternal age, maternal race, receipt of public benefits, mother’s marital status, parity, gestational age, multiple pregnancy, use of prenatal care, complications (previa, breech, cord prolapse, abruption), and severity of medical conditions. Medical conditions of pregnancy were divided into severe, moderate, mild, or none. Severe maternal medical conditions included cardiac disease, acute or chronic lung disease, diabetes, eclampsia, incompetent cervix, renal disease, and Rh sensitization. Moderate medical conditions included polyhydramnios, oligohydramnios, chronic hypertension, pregnancy-associated hypertension, previous infant weighing more than 4000 g, previous preterm infant or small for gestational age infant, and uterine bleeding. Mild medical conditions included anemia, genital herpes, hemoglobinopathy, hepatitis B, syphilis, and other.
All of the variables were tested in a logistic regression model. Using likelihood ratio tests and the area under the receiver operating characteristic (ROC) curve, we removed prenatal care and marital status because they did not change the area under the curve substantially, after adjustment for the other variables. The final model was tested on the remaining 75% of the data to see how well it predicted the probability of cesarean delivery for an individual woman. The Hosmer–Lemeshow goodness-of-fit test and area under the ROC curve4,12 were used to test predictive ability.
The estimates from the model were then used to predict a probability of cesarean delivery for every woman in the data set. Although logistic regression modeling estimates the probability of cesarean delivery, to keep terminology consistent in this article we will refer to a hopital’s average expected probability of cesarean deliveries as the expected or risk-adjusted cesarean delivery rate. The risk-adjusted expected cesarean delivery rate for a hospital was calculated by taking an average of the cesarean delivery probabilities for the women delivering at that hospital. To determine if each hospital’s actual observed cesarean delivery rate was above, below, or within the risk-adjusted expected cesarean delivery rate, a 95% confidence interval was calculated around the ratio of each hospital’s observed rate over its expected rate. Based on the 95% confidence intervals around the ratio, each hospital was classified into one of three groups: having an observed cesarean delivery rate that was significantly above, within, or below expected. Finally, the hospitals were ranked from the highest adjusted cesarean delivery rate (based on z scores) to the lowest rate.4
Direct standardization shows what the cesarean delivery rate would be if the risk stratum–specific cesarean delivery rates for each hospital were standardized to a referent population. In Lieberman’s article,5 which described the use of direct standardization to calculate risk-adjusted cesarean delivery rates, one teaching hospital was standardized to one community hospital. Because we were interested in cesarean delivery rate risk adjustment of all hospitals in the state, each hospital was standardized to the rate for the entire population.
Births were divided into 12 risk strata for each hospital. Leiberman originally described 18 strata, but because we eliminated all patients with a previous cesarean delivery, for reasons stated previously, only 12 strata were applicable.
Strata categorizations are mutually exclusive. Each patient was evaluated and placed into the first stratum that she belonged. We used the same order as originally described by Lieberman.5 Risk strata used in the direct standardization method include nulliparous patients with multiple pregnancy, breech or transverse lie, pre-term delivery at less than 37 weeks, no trial of labor permitted, at least 36 weeks with medical risk, and at least 36 weeks without medical risk, and multiparous patients with multiple pregnancy, breech or transverse lie, preterm delivery at less than 37 weeks, no trial of labor permitted, at least 36 weeks with medical risk, and at least 36 weeks without medical risk. Medical risks included diabetes, polyhydramnios and oligohydramnios, chronic hypertension, pregnancy-associated hypertension, eclampsia, and abruption. These slightly differ from the medical risks described in Lieberman’s article, which also included fetal growth restriction, fetal hydrops, nonreassuring fetal condition, fever, and infant weight at least 4500 g as medical risks. Numbers of cases of fetal growth restrictions and hydrops could not be accurately obtained from birth certificate data. However, these conditions are relatively rare. Fever and nonreassuring fetal condition can be influenced by the obstetric care given, and so we did not use this as part of a case-mix adjustment. Lastly, birth weights are not known before delivery and should not be used as part of case-mix adjustment. Thus, birth weight of at least 4500 g was not used as a medical risk in the standardization method.
Using these risk strata, we calculated a cesarean delivery rate for each hospital standardized to the population rate. After standardization, any differences between the hospital rates and the population rates would be due to factors other than these risk categories. Because many of the hospitals had no births in some of the risk strata, we were unable to calculate standard errors of the standardized rates for those hospitals. So, rather than ranking the hospitals based on a z score, we ordered them from the highest to lowest standardized rate. For hospitals with adequate data, we calculated 95% confidence intervals around the standardized rate. Using these confidence intervals, we classified hospitals into one of three groups: having a standardized cesarean delivery rate that was significantly above, within, or below the population rate.
To compare the results of these two methodologies, we examined whether the hospitals were ranked in the same order using the Spearman correlation. We then compared the ranks of hospitals with the highest rates by categorizing each list of hospitals into two groups—those with the highest 25% ranks and all others. We repeated this a second time by categorizing the hospitals into two groups based on the lowest 25% ranks versus all others. Finally, we compared the hospitals from each method that were identified as being statistical outliers, using the categories of above, within, or below their expected rates. We used the Cohen statistic to see how well the two methods flag hospitals with the highest rates, the lowest rates, and the statistical outliers. A score of more than .60 was considered substantial agreement.13
|
RESULTS |
|---|
|
TOP
ABSTRACTMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
. The data quality was highest for demographic variables and lowest for complications and maternal medical conditions.
|
. The mean maternal age was 27 years. Although blacks made up only 3.5% of the sample, they had slightly more cesarean deliveries (16%) than the other race–ethnic groups. Nulliparas were almost three times more likely to have a cesarean delivery (20% versus 7%). Gestational age of 24–30 weeks, having a multiple pregnancy, and complications were all very strong predictors of having a cesarean delivery. Moderate to severe medical conditions increased the risk of cesarean delivery about two-fold.
|
. Three of the 12 risk strata had fewer than 350 births, which is less than 0.3% of all births. This may explain why some individual hospitals had no births for these strata. The cesarean delivery rate within each risk stratum varied from a high of 77% for "nulliparas with breech or transverse lie" to a low of 4% for "multiparas at at least 36 weeks with no medical risks."
|
There is substantial agreement in the rankings from both methods. A comparison of the ranked lists of hospitals for the two methods is shown in Figure 1. Each circle represents a hospital. The farther away a circle is from the diagonal line, the less the ranks agreed. Although the ranks for several of the hospitals seem far apart, the Spearman correlation coefficient is quite high (r = .84, P < .001). The statistic comparing the highest 25% ranked hospitals showed good agreement (.67), and the statistic for the bottom 25% also showed substantial agreement (.69).
|
Hospitals that did not have patients in all strata had a lower median number of deliveries (1000) than those hospitals without any missing strata (2966). However, there were some large hospitals that had empty strata. The range of volume in hospitals with empty strata was 57–5164. Eleven percent (four of 38) of the hospitals with empty strata had more than 2000 deliveries.
A test of the agreement of the statistical outliers for the 29 hospitals where the calculation could be made had a of .68 (Table 4). The standardization method recognized ten of 11 hospitals identified by the logistic regression method as performing fewer cesarean deliveries than expected. Hospitals with rates above expected did not agree as well—four of the hospitals agreed, but three of the hospitals noted by the logistic regression model for performing more than the expected number of cesarean deliveries were considered within expectations by the standardization method.
|
|
DISCUSSION |
|---|
|
TOP
ABSTRACTMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Direct standardization is a relatively easy method to comprehend and perform. It may work very well to compare two large hospitals, assuming they have births in all risk strata. However, cesarean delivery rate risk adjustment of all hospitals in a state is desirable from a quality improvement point of view. Knowing whether a hospital’s cesarean delivery rate is above what it should be for its patient population is not as important as knowing whether this difference is more than by chance alone. Being able to determine if the differences are statistically significant helps to differentiate random chance from a systematic problem in quality. Because direct standardization cannot determine if hospitals are statistical outliers if they do not have births in some of the risk strata, it will fail to produce usable information for a large number of hospitals. The problem of empty strata encountered by the direct standardization method could be overcome by adding a single, hypothetic patient to each stratum where there is no patient. Although adding a single patient might not change the results substantially, this arbitrary way of changing the data to get an answer could bias the results.
Logistic regression modeling does take some training. However, if a single standard for risk adjustment is a goal, finding personnel with the ability to perform logistic regression modeling is worth the effort. Presumably, if cesarean delivery rate risk adjustment is done by a central agency, such as ACOG or a state health department, people with this skill should be easily available.
Birth certificate data are not perfect. However, collecting information from the medical record for every delivery in the United States is not feasible. In a study comparing medical records with birth certificate and hospital discharge data, Parrish et al14 stated that "differences between the quality of the coding on the birth certificate data and hospital discharge abstract data are likely due to the incentives involved in recording and checking the data in the respective systems." This statement suggests that, if given the right incentive, hospitals are capable of collecting and reporting high-quality data. We believe that if birth certificate data were used for risk adjustment and quality assessment, the quality of birth certificate data would improve greatly.
Adoption of a single standard methodology for cesarean delivery rate risk adjustment is desirable. If cesarean delivery rate risk adjustment of all hospitals is the goal, the logistic regression method has the advantage of being able to determine if differences in risk-adjusted cesarean delivery rates are statistically significant. However, if comparing two large hospitals is the goal, the direct standardization method may be simpler.
|
Footnotes |
|---|
The abstract was presented at the annual meeting of the Society for Maternal–Fetal Medicine; February 3–8, 2003; San Francisco, California.
doi:10.1016/S0029-7844(03)00356-9
Received October 17, 2002. Received in revised form January 10, 2003. Accepted January 16, 2003.
|
REFERENCES |
|---|
|
TOP
ABSTRACTMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
2. Brewster A, Karlin B, Hyde L, Jacobs C, Bradbury R, Chae Y. MEDISGRPS: A clinically based approach to classifying hospital patients at admission. Inquiry 1985;12: 377–87.
3. Knaus W, Draper E, Wagner D, Zimmerman J. The APACHE III prognostic system: Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991;100:1619–36.
4. Iezzoni LI. Risk adjustment for measuring healthcare outcomes. Chicago: Health Administration Press, 1997:609.
5. Lieberman E, Lang J, Heffner L, Cohen A. Assessing the role of case mix in cesarean delivery rates. Obstet Gynecol 1998;92:1–7.[Abstract]
6. Keeler E, Park R, Bell R, Gifford DS, Keesey J. Adjusting cesarean delivery rates for case mix. Health Serv Res 1997;32:511–28.[Medline]
7. Aron D, Harper D, Shepardson L, Rosenthal G. Impact of risk-adjusting cesarean delivery rates when reporting hospital performance. JAMA 1998;279:1968–72.
8. Bailit J, Dooley S, Peaceman A. Risk adjustment for inter-hospital comparison of primary cesarean rates. Obstet Gynecol 1999;93:1025–30.
9. DiGiuseppe D, Aron D, Payne S, Snow R, Dierker L, Rosenthal G. Risk adjusting cesarean delivery rates: A comparison of hospital profiles based on medical record and birth certificate data. Health Serv Res 2001;36: 959–77.[Medline]
10. Hohner V, Armstrong R. Birth events records project: Developing a permanent linked database: Olympia, Washington: Washington State Department of Health, 1991.
11. Bailit J, Garrett J, Miller W, McMahon M, Cefalo R. Hospital primary cesaran delivery rates and the risk of poor neonatal outcomes. Am J Obstet Gynecol 2002;187: 721–7.[Medline]
12. Lemeshow S, Hosmer D. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982;115:92–106.
13. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.[Medline]
14. Parrish K, Holt F, Connell B, Williams B, LoGerfo J. Variations in the accuracy of obstetric procedures and diagnoses on birth records in Washington State, 1989. Am J Epidemiol 1993;138:119–27.
15. Piper J, Mitchel E, Snowden M, Hall C, Asdams M, Taylor P. Validation of 1989 Tennessee birth certificates using maternal and newborn hospital records. Am J Epidemiol 1993;137:758–68.
16. Green D, Moore J, Adams M, Berg C, Wilcox L. Are we underestimating rates of vaginal birth after previous cesarean birth? The validity of delivery methods from birth certificates. Am J Epidemiol 1998;147:581–6.
17. Buescher P, Taylor K, Davis M, Bowling J. The quality of the new birth certificate data: A validation study in North Carolina. Am J Public Health 1993;83:1163–5.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
