| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
ORIGINAL RESEARCH |
From the Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, San Francisco, California; Division of Hospital and Specialty Medicine, Portland Veterans Affairs Medical Center, Portland, Oregon; Department of Obstetrics and Gynecology, Kaiser Permanente Medical Center, San Francisco, California; Department of Obstetrics and Gynecology, University of Washington, Seattle, Washington; and Department of Obstetrics and Gynecology, Oregon Health and Sciences University, Portland, Oregon.
Address reprint requests to: Lee A. Learman, MD, PhD, San Francisco General Hospital, Department of Obstetrics and Gynecology, 1001 Potrero Street, Room 6D-9, San Francisco, CA 94110; E-mail: learmanl{at}obgyn.ucsf.edu.
 |
ABSTRACT |
|---|
TOP
 ABSTRACT MATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
METHODS: We recruited 74 residents from eight residency programs to attend the Depression Education Program, which consists of a 1-hour lecture and two 2-hour workshops combining discussion, diagnosis and treatment tools, critique of a videotape, practice with feedback, and audio-tape self-assessment. Before and after the program, participants 1) completed a questionnaire measuring knowledge, attitudes, and reported actions taken with a recent depressed patient; 2) received a standardized patient visit; and 3) kept lists of patients they suspected were depressed. Clinic patients completed a depression screening questionnaire. To assess improvement, we used paired t tests, Mc-Nemar 
2 tests, and multivariate models adjusting for training site.
RESULTS: The education program led to 3-month improvements in participants’ reported use of formal diagnostic criteria (38% before, 66% after; P = .004), clinical actions documented for suspected depression (P = .035), and perceived self-efficacy in depression care (P < .001). Perceived preparedness to diagnose depression, treat with medications, and comanage with a mental health practitioner improved (P < .05 for each). Small improvements in clinical behaviors with standardized patients and clinic-based depression detection rates were not statistically significant.
CONCLUSION: The Depression Education Program improved residents’ knowledge, confidence, and reported clinical actions with depressed patients, but did not improve most objectively assessed outcomes.
Depression is a common, under-recognized, and under-treated condition1 that is more common in women than in men.2 The prevalence of depressive disorders in obstetrics and gynecology practices ranges as high as 27%.3 Primary care physicians fail to recognize a mood disorder in a third to a half of their depressed patients4,5; consequently, obstetrician-gynecologists have the potential to improve the detection and treatment of depression in women. Unfortunately, few practicing obstetrician-gynecologists have confidence managing depressive disorders or report having been trained to manage them.6,7
Successful care of depressed patients relies heavily on communication skills.8 Although didactic resources exist for improving residents and medical students’ knowledge of depressive disorders in women,9 curricula do not exist to improve residents’ clinical and communication skills for diagnosing depression and overcoming patient resistance, discussing therapeutic options, and initiating medical treatment. The Accreditation Council for Graduate Medical Education Outcome Project is challenging our traditional approach to communication skills training and evaluation.10,11 To optimize the successful training of obstetrician-gynecologists, it will be important to determine which instructional techniques are effective for improving interviewing skills in general, and for preparing residents to diagnose and treat specific health conditions such as depression.
The purpose of our study was to evaluate a depression education program for residents in obstetrics and gynecology (Learman L, Gerrity M, Chez R, Stotland N, Bernard L, Raju M, et al. The MacArthur Foundation depression education program for residents in obstetrics and gynecology: Participant’s workbook and facilitator’s guide [unpublished training manual]) adapted from a program found to be effective for improving the clinical skills of practicing internists and family physicians.12,13 To achieve our objectives we performed a multicenter intervention testing the curriculum’s ability to improve residents’ knowledge, attitudes, confidence, and clinical skills for diagnosing and treating major depression. To fully understand the impact of the depression education program, and whether it was transitory or sustained, we employed a wide array of evaluation methods and supplemented immediate outcomes with assessments 3 months later.
|
MATERIALS AND METHODS |
|---|
|
TOP
ABSTRACTMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
The study was designed as a prospective pre- and postintervention evaluation of the Depression Education Program for residents in obstetrics and gynecology.8 The curriculum included a 1-hour lecture on the prevalence, screening, diagnosis, and treatment of depressive disorders in women accompanied by laminated pocket cards with flow diagrams and checklists summarizing key points from the lecture and listing local mental health resources. Study participants then attended a 2-hour workshop in small groups of 6–10 focused on developing communication skills and strategies for overcoming patients’ resistance to the diagnosis of depression. The instructional techniques used in the workshops included elicitation of personal objectives, group critique of a stimulus videotape, and communication skills practice with feedback and repetition. In a second workshop conducted 1 to 2 weeks later, residents reviewed audio-tapes they had made with a "resistant patient" at the end of the previous workshop, received feedback, repeated the exercise, and concluded by making a statement of how they planned to incorporate their new knowledge and skills into the care of their patients.
We created measures of depression knowledge, attitudes, and self-reported clinical actions based on curricular content and established measures.13 These were administered by questionnaire within 1 week before and 1 week after the education program and 3 months later. The two postintervention questionnaires also assessed the perceived value of the education program and detection of standardized patients. During the 2–4 months before and after the intervention, each participant’s continuity clinic patients completed a written two-question depression screening tool14 in English, Spanish, or Chinese and an accompanying informed consent form, where required. The screening and consent forms were separated from the patients’ medical records, and the residents did not know their content. At the end of each continuity clinic session, participating residents completed cards indicating which patients they suspected were depressed. The postworkshop questionnaire included scales asking participants to rate the value of each curriculum component (lecture, pocket cards, videotape, skills practice, audiotape review) in helping to achieve three curricular objectives: 1) screening and diagnosing depression, 2) presenting the diagnosis and overcoming resistance, and 3) understanding the functions of the patient interview. Residents rated each item using a five-point Likert scale ranging from "not at all valuable" (1) to "extremely valuable" (5).
During the 4 weeks before and after the intervention, an unannounced standardized patient visited each participant. Two standardized patient cases were developed as modifications of cases developed previously.13 The cases were developed to have the same level of difficulty or complexity and to be interchangeable as an assessment tool. Both cases involved a reproductive-aged woman who had recently moved to the area and had arranged a drop-in appointment with a problem (headache or weight gain) she was concerned might be caused by her birth control method. If asked, each patient would express five of nine symptoms of major depressive disorder. Each patient was instructed to use a prompt, or clue, if the resident focused exclusively on their presenting symptoms and did not address other signs or symptoms of depression. We performed block randomization by site using a computer-generated random numbers list to assign half the residents to scenario A at baseline and scenario B at follow-up and the other half at each site to the opposite sequence. All actresses spent 10–20 hours in training for their roles by observing each other, role-playing, and comparing responses to checklists. The principal actresses had a final role-play with the principal investigator that was videotaped and reviewed as a group.
Immediately after their visits, the standardized patients completed checklists to document the residents’ depression screening, diagnosis, treatment, and follow-up behaviors. The standardized patients also completed global ratings of the residents’ communication skills, and indicated whether they thought the residents had attended the depression workshop. Screening and diagnosis behaviors included the number of depression criteria assessed, assessment of suicidal ideation, stresses at work and at home, screening for a history of physical or sexual abuse, and discussing the possibility of depression. Treatment and follow-up behaviors included discussion of treatment options, recommending or prescribing medications, recommended consultation with a mental health practitioner, and scheduling follow-up within 2 weeks. Global ratings of communication skill included a nine-item version of the American Board of Internal Medicine’s Patient Satisfaction Scale,15 a single-item assessment of rapport, and an adapted version of Kaplan’s three-item Participatory Decision-Making Scale.16 All global ratings were made on five-point Likert scales, and composite scale scores were the sums of the individual item scores. The questionnaires completed by residents after the workshop (preintervention standardized patient visit) and at the 3-month follow-up (postintervention standardized patient visit) included an assessment of standardized patient detection. Residents were asked to indicate whether they were aware they had been visited (forced choice) and, if so, whether detection occurred before, during, or after the visit (forced choice), and what specifically led to visit detection (open ended).
During the 2–4 months before the intervention and the 2–4 months after the intervention, we collected two sets of data for calculation of the depression detection rate. The median baseline data collection period was 17 weeks, and the follow-up data collection period was 16 weeks. The depression detection rate was calculated as the fraction (number of patients suspected to be depressed by the residents)/(number of patients answering "yes" to one or both depression screening questions). The numerator was measured by distributing and collecting pocket cards for residents to keep during each continuity clinic. The cards listed the patients on their schedule, with space available to add names, and instructions to draw a line through names of scheduled patients not seen. To the right of each name was a box the resident was instructed to mark if he or she thought the patient was likely to have the diagnosis of major depressive disorder. The denominator was measured by distributing and collecting screening questionnaires from patients scheduled to be seen by each participating resident while they were waiting to be called into the examination room. Questionnaires in English, Spanish, and Chinese asked patients to complete a two-item screening tool.14 When institutional review boards required consent forms to be completed by patients (four sites), these were placed on the reverse side of the screening tool. Screening forms were distributed and collected before patients were assessed by the residents. If the patient answered "yes" to one or both questions (depressed mood, anhedonia), she was considered screen positive and potentially detectable by the resident.
Medical records were requested for each patient visit at which the resident thought the patient was likely to be depressed (as evidenced by checking that box on the pocket card). All patient and resident identifiers were redacted and replaced with study codes. With the exception of one site, whose institutional review board insisted that the redacted medical records not leave the institution, the records were distributed at random to five faculty members for coding. Local faculty coded the records from the site that could not release them. All coders used a checklist measuring the presence or absence of documented behaviors related to depression screening, diagnosis, and treatment. Each faculty member predominantly received records from training programs other than his or her own, and all were blinded to whether the visit occurred before or after the education program. Interjudge reliability was assessed using ten records that were coded by all judges.
For questionnaire items scored on ordinal (Likert) scales, we calculated medians and interquartile ranges. We subjected scales included in the baseline questionnaire to factor analyses and scale reliability analyses.17 For published scales, we aimed to confirm the reported factor structures and reliability of the established scale. For incompletely developed or new scales, we sought to group individual items into fewer interpretable scales that had reasonable reliability (Cronbach 
.60 or more). For medical record review, we calculated interjudge reliability using the intraclass correlation coefficient. We used paired t tests and the McNemar 2 tests to assess improvement over time in continuous and categoric variables. All tests were two tailed, with an of .05 and a statistical trend defined as .05 
P .10. We used mixed linear models to adjust for cluster sampling (multiple residents from the same training program) using statistical software designed for that purpose.18 We examined the distributions and plots of ordinal data and composite scales to assure normality before treating the variables as continuous in parametric multivariate analyses.
Sample size calculations indicated that data from 50 residents would provide 80% statistical power, at a two-tailed of .05, to detect a minimum increase in the reported or observed use of a specific clinical behavior from 20% at baseline to 46% postintervention. The sample size calculations assumed that improvements were not associated with the site at which the residents trained (ie, the cluster in which they were sampled). If significant cluster effects were present, we would need data from 20 residents per training site to retain 80% power to detect a minimum improvement of the same magnitude as indicated above.
|
RESULTS |
|---|
|
TOP
ABSTRACTMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Sixty-three (85%) of the enrolled participants were female, and nearly all were in postgraduate years 2 through 4 of training. The mean age was 30.4 years (standard deviation 2.8). The continuity clinics were located principally within a faculty practice (38%), a public hospital clinic (42%), and a health maintenance organization clinic (10%). The clinic patient mix included both pregnant and nonpregnant women for 90% of the participants. Two thirds of the clinic sites had at least 20% of patients who did not speak English, and 70% had bilingual clinic staff or hospital interpreters available. Fifty-five residents (74%) reported that a mental health practitioner was available at their clinic site and accessible by phone or written request.
Residents reported high value for all program components, but were able to discriminate which components seemed more helpful for achieving specific educational objectives. Most of the participants considered the lecture and pocket cards to be very or extremely valuable for the objectives of screening and diagnosing depression (69% and 79%, respectively). Most considered the skill practice and audiotape feedback exercises very or extremely valuable for the objectives of presenting the diagnosis and overcoming resistance (73% and 74%, respectively) and understanding the functions of the patient interview (63% and 69%, respectively). Watching and critiquing the videotape vignette was considered by half of the participants (49–51%) to be very or extremely valuable for achieving each of the objectives.
Substantial improvements occurred in most of the domains assessed in the study questionnaire immediately and 3 months after the Depression Education Program (Table 1
). The table depicts the site-adjusted P values for 3-month improvements by the 42 residents with data at all three time points. In multivariate analyses, the site at which the resident trained was not a statistically significant predictor of improvement. The overall knowledge test score improved modestly, and the percentage of residents considering themselves very knowledgeable in the use of antidepressant medications doubled. Improvements were also observed in clinical behaviors that the residents reported using with their most recent depressed patient. The largest of these was in the use of formal diagnostic criteria. There were also statistical trends toward declining use of a mental health practitioner and improvement in appropriate scheduling of a follow-up visit within 2 weeks. Two thirds of the residents reported using direct questioning for suicide assessment before the curriculum, and the rate did not show a statistically significant improvement 3 months later.
|
Fifty participants completed both baseline and follow-up standardized patient assessments. Differences in screening, diagnosis, treatment, and follow-up behaviors were small in magnitude and not statistically significant (Table 2). The table depicts the P values for 3-month improvements, adjusted for site of training and sequence of pre- and poststandardized patient characters. In most cases, changes in observed behavior occurred in a positive direction, although none were statistically significant. Patient satisfaction was judged as high by the standardized patients at baseline (41 of 45 points) and declined modestly, but significantly, after the program. Rapport and participatory decision-making scores were also high at baseline and follow-up assessment. The standardized patients reported that they had been detected in 10% of the interactions preintervention and 18% postintervention; however, the residents reported much higher detection rates. Most of the preintervention visits (56%) were not detected by the residents, whereas most of the postintervention visits (82%) were detected. The detections occurred before or during the standardized patient interaction, and were thus potentially able to influence the residents’ behavior, in 20% of preintervention visits and 42% of postintervention visits.
|
Blinded review of medical records revealed a 50% increase in documented clinical actions (education, counseling, referral, medication, etc) for continuity clinic patients suspected to be depressed after the Depression Education Program (0.39 actions per patient preintervention versus 0.59 actions per patient postintervention, P = .035). Interjudge reliability was high (.70–.80) among the five faculty completing checklists of depression documentation in the continuity clinic records of patients in whom the participating residents considered depression to be likely.
|
DISCUSSION |
|---|
|
TOP
ABSTRACTMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Our study has several strengths. The recruitment of a cross-section of residents from a wide variety of training programs and working with diverse groups of continuity clinic patients enhances the generalizability of our findings. The elaborate measures conducted before and after the curriculum, and 3 months later, provide a comprehensive understanding of what a 5-hour curriculum can and cannot accomplish, and the challenges of using standardized patients in continuity clinics.
There are several limitations worthy of mention. Our Depression Education Program bundled a diverse set of instructional techniques that, together, improved residents’ performance. Therefore, we cannot be certain which components of the program conferred the most benefit and which, if changed, would have improved its impact on objectively assessed competencies. Given evidence from the continuing education literature, we expect that the interactive workshops played a more important role than the lecture alone in effecting changes that endured 3 months later.20 However, our program did not include the ongoing feedback and reinforcement of the knowledge and skill that may be essential to sustaining improvements in behavior. We posit that the program’s impact would also be enhanced by nesting it within a systematic communication skill curriculum for residents and by pairing the module with clinic-based systems interventions21,22 including ongoing feedback on performance.
Second, it was not possible to design a feasible randomized trial comparing the Depression Education Program to usual training in depressive disorders. Randomization within each training program would not work because of frequent interaction and information sharing between residents who would have been randomized to different groups. Randomization of programs would not work because a much larger number of programs would have had to have been involved, which was beyond the scope of the study. Because of the nonrandomized design, it is possible that the improvements we observed would have occurred over time without the intervention. We find this explanation unlikely given the lack of formal education on communication skills and managing major depressive disorders for residents in obstetrics and gynecology at the programs participating in the study. Third, although residents who did not participate had clear logistical reasons, selection bias is possible at the programmatic level. Participating programs and program directors might not be representative of all programs in terms of receptivity and support for a 5-hour curriculum fit into an already tight didactic schedule. Fourth, institutional review board requirements of patient consent limited our ability to screen enough patients to reliably estimate depression detection rates for residents at most sites. The two-question screening tool is commonly used in primary care office questionnaires and is easy to complete, yet the requirement of informed consent in some institutions made it more difficult and time consuming for some patients to answer the questions.
Finally, detection of the standardized patients was greater than expected, despite our diligent efforts to schedule them using the same procedures as for any clinic patient. As the residents gained more experience with detection, their success improved. Greater detection and potentially a backlash evident in the residents’ communication skills with the actresses postintervention made the validity of the resulting data questionable at best. Standardized patients have been the gold standard for measuring physician competency, but this study underscores the difficulty of using unannounced actresses in busy outpatient training sites. Recent evidence supports the validity of using less reactive measures, such as clinical vignettes,23 and we recommend that other educational investigators assess communication skills using a combination of vignettes completed by residents and rating scales completed by patients, colleagues, and staff. A recent national study identified that 74% of family practice residents, 36% of internal medicine residents, and 22% of obstetrics and gynecology residents felt "very prepared" to diagnose and treat depression, which was listed among other common problems they would encounter in future practice.24 In our study, the Depression Education Program doubled residents’ perceived preparation to diagnose depression (24% to 50%) and to comanage depression with a mental health practitioner (32% to 61%), and quadrupled their preparedness to treat depression with medications (6% to 28%). We conclude that the Depression Education Program is a promising intervention for improving obstetrics and gynecology residents’ knowledge, attitudes, confidence, and reported clinical actions in caring for depressed patients. The program’s impact on residents’ objectively assessed skills and patient outcomes cannot be established from the available data.
|
Footnotes |
|---|
Pam Edwards, MD, is gratefully acknowledged for skillfully cofacilitating all of the workshops with local faculty, including Drs. Ann Eastman (Kaiser-Permanente Medical Center, Oakland), Michelle Young (Kaiser-Permanente Medical Center, Santa Clara), Kathy Blenko (Santa Clara Valley Medical Center), and Hal Holbrook (Stanford University).
Received April 16, 2002. Received in revised form June 20, 2002. Accepted July 25, 2002.
|
REFERENCES |
|---|
|
TOP
ABSTRACTMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
2. Williams JB, Spitzer RL, Linzer M, Kroenke K, Hahn SR, deGruy FV, et al. Gender differences in depression in primary care. Am J Obstet Gynecol 1995;73:654–9.
3. Bixo M, Sundstrom-Poromas I, Bjorn I, Astrom M. Patients with psychiatric disorders in gynecologic practice. Am J Obstet Gynecol 2001;185:396–402.[Medline]
4. Simon GE, Von Korff M. Recognition, management, and outcomes of depression in primary care. Arch Fam Med 1995;4:99–105.[Abstract]
5. Ormel J, Koeter MWJ, van den Brink W, van de Willige G. Recognition, management and course of anxiety and depression in general practice. Arch Gen Psychiatry 1991; 48:700–6.[Abstract]
6. Gerrity MS, Williams JW, Dietrich AJ, Olson AL. Identifying physicians likely to benefit from depression education. A challenge for health care organizations. Med Care 2001;39:856–66.[Medline]
7. Schmidt L, Greenberg BD, Holzman GB, Schulkin J. Treatment of depression by obstetrician-gynecologists: A survey study. Obstet Gynecol 1997;90:296–300.[Abstract]
8. Badger LW, deGuy FV, Hartman J, Plant MA, Leeper J, Ficken R, et al. Psychosocial interest, medical interview, and the recognition of depression. Arch Fam Med 1994;3: 899–907.[Abstract]
9. Association of Professors of Gynecology and Obstetrics. Depressive disorders in women: Diagnosis, treatment and monitoring. Washington: APGO, 1997.
10. ACGME Outcome Project. Chicago: Accreditation Council for Graduate Medical Education. Available at http://www.acgme.org/outcome. Accessed 2002 Oct 11.
11. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002;287:226–35.
12. Cole S, Raju M, Barrett J, Gerrity M, Dietrich A. The MacArthur Foundation depression education program for primary care physicians: Background and rationale. Gen Hosp Psychiatry 2000;22:299–358.[Medline]
13. Gerrity MS, Cole SA, Dietrich AJ, Barrett JE. Improving the recognition and management of depression. Is there a role for physician education? J Fam Pract 1999;48:949–57.[Medline]
14. Whooley MA, Avins AL, Miranda J, Browner WS. Case-finding instruments for depression: Two questions are as good as many. J Gen Intern Med 1997;12:439–45.[Medline]
15. Stump TE, Dexter PR, Tierney WM, Wolinsky FD. Measuring patient satisfaction with physicians among older and diseased adults in a primary care municipal outpatient setting: An examination of three instruments. Med Care 1995;33:958–72.[Medline]
16. Kaplan SH, Gandek B, Greenfield S, Rogers W, Ware JE. Patient and visit characteristics related to physicians’ participatory decision-making style: Results from the Medical Outcomes Study. Med Care 1995;33:1176–87.[Medline]
17. SAS Institute Inc. Measuring the internal consistency of a test, using PROC CORR to compute Cronbach’s alpha. SAS Communications 1994;20(4):TT2–5.
18. SAS Institute Inc. SAS/STAT user’s guide, version 8. Cary, North Carolina: SAS Institute Inc., 1999.
19. Williams JW, Rost K, Dietrich AJ, Ciotti MC, Zyzanski SJ, Cornell J. Primary care physicians’ approach to depressive disorders: Effects of physician specialty and practice structure. Arch Fam Med 1999;8:58–67.
20. O’Brien T, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education meetings and workshops: Effects on professional practice and health care outcomes (Cochrane Review). Cochrane Database Syst Rev 2001;2: CD003030.
21. Scholle SH, Agatisa PK, Krohn MA, Johnson J, McLaughlin MK. Locating a health advocate in a private obstetrics/gynecology office increases patient’s receipt of preventive recommendations. J Womens Health Gend Based Med 2000;9:161–5.[Medline]
22. Shoenbaum M, Unutzer J, Sherbourne C, Duan N, Rubenstein L, Miranda J, et al. Cost-effectiveness of practice-initiated quality improvement for depression. Results of a randomized controlled trial. JAMA 2001;286: 1325–30.
23. Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction. A prospective validation study of 3 methods of measuring quality. JAMA 2000;283:1715–22.
24. Blumenthal D, Gokhale M, Campbell EG, Weissman JS. Preparedness for clinical practice. Reports of graduating residents at academic health centers. JAMA 2001;286: 1027–34.
This article has been cited by other articles:
|
|
 |
|
  A. J. Dietrich, T. E. Oxman, J. W. Williams Jr, K. Kroenke, H. C. Schulberg, M. Bruce, and S. L. Barry Going to Scale: Re-Engineering Systems for Primary Care Treatment of Depression Ann. Fam. Med, July 1, 2004; 2(4): 301 - 304. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. B. Berenson, C. R. Breitkopf, and Z. H. Wu Reproductive Correlates of Depressive Symptoms Among Low-Income Minority Women Obstet. Gynecol., December 1, 2003; 102(6): 1310 - 1317. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
