NAGPS Survey Team
National Association of Graduate-Professional Students
survey@nagps.org      http://survey.nagps.org/

The National Doctoral Program Survey, an online assessment of educational practices in doctoral programs in the U.S. and Canada, was conducted by the National Association of Graduate-Professional Students (NAGPS) between March 30 and August 15, 2000. The survey, funded by a grant from the Alfred P. Sloan Foundation, was a follow-up to a pilot effort, the PhDs.org Graduate School Survey, conducted in 1999 by Geoff Davis and Peter Fiske. Current and recent doctoral students were asked to report on their experience in graduate school and to assess their programs' implementation of educational practices recommended by the National Academies, the Association of American Universities, and others. Over 32,000 students completed surveys, representing nearly 5,000 doctoral programs at almost 400 graduate institutions across the United States and Canada.

Survey results are available online at http://survey.nagps.org/. The results include both discipline-level results disaggregated by a variety of participant characteristics as well as program-level reports for the 1300 programs from which 10 or more responses were received. An online tool ranks doctoral programs based on their implementations of various recommended educational practices. Each doctoral program has an associated discussion board to facilitate conversations between students, faculty, and administrators. Program-level results for programs receiving between 5 and 9 responses are available to participants, their chairs, and their deans, but not to the general public.

An overwhelming majority of survey participants reported positive educational experiences:

• 81% of respondents say they are satisfied with their doctoral programs.
• 86% of respondents say they are satisfied with their advisors.
• 80% would recommend their programs to prospective students.

Students reported the greatest satisfaction in programs that have curricula that prepare them for a variety of careers, that provide them with comprehensive information before enrollment, and that treat them with respect. The common thread is that satisfaction is strongly linked to choice: students want curricula broad enough to give them a choice of careers, they want information to ensure that their choices are informed, and they want the choices they make to be respected.

Despite their overall satisfaction, survey participants express important concerns about preparation for non-academic careers, curricular breadth, training for teaching, and program diversity. These student concerns are in areas that institutions and the academic community have themselves identified as important components of graduate education through recent reports from the National Science Board (NSB 1998), the National Academies of Science (NAS 1995; NRC 1998), the Association of American Universities (AAU 1998), and others.

Highlights of the results are as follows:

• Students report insufficient guidance to prepare them both for non-academic careers (62%) and academic careers (30%). Placement services are also lacking, with 64% citing inadequate placement services for non-academic careers and 36% for academic careers.
• Most doctoral students engage in teaching as part of their graduate training, and many undergraduates are taught by graduate teaching assistants. Respondents report not receiving appropriate preparation and training before they enter the classroom (45%), and lack appropriate supervision to help improve their teaching skills (49%). Further, 39% do not feel that their needs and interests are giving appropriate consideration in determining which courses they are assigned to teach.
• While 62% of respondents say that their program actively recruits students from underrepresented groups, the program environment is not always as supportive for these students once they arrive. Women's satisfaction with program environment is 11% lower than their male counterparts (69% vs. 80%), and while 74% of Caucasians and Asian-Americans feel comfortable and supported, only 60% of African-Americans, Latinos, and Native Americans report the same. Female and minority students' responses to all questions were consistently more negative than those of their counterparts.
• Most programs provide accurate information about program expectations and costs. However, few academic programs provide information to prospective students about the career placement outcomes (35%) and completion rates (30%) of recent program students.
• Many students feel that those who express an interest in pursuing non-research careers lose the favor of the department. This includes those interested in non-academic positions as well as academic positions at teaching colleges.
• Despite recommendations that students have the opportunity to broaden their educational experiences outside their own departments, only about half of respondents say that they are encouraged to gain additional skills through internships and coursework outside their program.
• While 65%-75% of respondents in all fields report satisfaction with preparation for academic careers, preparation for non-academic careers ranged widely, from 65% satisfied in Engineering but only 21% in the Humanities.
• Despite respondents' reporting that they receive appropriate training for academic careers, only 43% of students in the Life Sciences believe that the teaching experience available to students in their program is adequate preparation for an academic/teaching career. The Humanities fare much better in this area, with 72% reporting adequate training.
• Over 80% of students in all fields report a supportive student community in their program, and 69-76% indicate that they are involved in decisions relevant to their education.
• More than 80% of students in all fields report positive mentoring experiences, including continuous and constructive feedback on their progress toward their degree.
• There are relatively few differences based on gender or ethnicity, but there a number of differences between U.S. citizens and those from other countries. Some of this difference, however, may be related to the field distribution of international doctoral students, which is more highly weighted to the Physical and Life Sciences and Engineering.
• Despite an increasing attention to ethics within academic fields, few students report training in this area. The disciplines with the largest fraction of students reporting such training are Education (71%), Life Sciences (70%) and Social Sciences (62%), while Engineering (44%), the Humanities (43%), and Physical Sciences (35%) have the smallest fraction.
• Students in the Physical Sciences (80%), Engineering (79%), and Life Sciences (72%) are generally satisfied with their funding, but there is a belief that sufficient financial support is not as available to students in the Humanities (51% satisfied), Education (57%), and Social Sciences (59%).
• Recommendations for best practices have stressed the need for students to receive frequent and thoughtful assessments of their progress, and about two-thirds of respondents satisfied with the availability of annual assessments. Engineering respondents, however, were less likely to receive such progress reports with only 47% responding positively.
• There is reason to be encouraged about the future. Those who have not yet received their degree report a higher satisfaction with nearly all of the best practices than students who received their degree in or before the year 2000.

The survey consisted of 48 questions in nine areas relevant to doctoral education: information for prospective students, curricular breadth and flexibility, teaching, professional development, career guidance and placement services, time to degree, faculty mentoring, program climate, and overall satisfaction. Each section contained a set of multiple choice questions with the following possible answers: Strongly Agree, Agree, Disagree, Strongly Disagree, Don't Know, and Not Applicable. Each section also contained a free response box to allow respondents to expand upon their responses to any questions in that section.

The survey was administered online from March 30 - August 15, 2000. NAGPS and the Survey Team publicized the survey widely with the assistance of other stakeholders in doctoral education, who were invited to become supporters of the survey by publicizing it to students. Members of the Survey Team asked the major disciplinary societies and professional associations in each academic field to help spread the word among their membership. Survey volunteers also contacted graduate deans, graduate student associations, and program chairs to encourage them to invite participation from their students.

As the survey progressed, deans, program chairs, graduate student leaders, and professional associations were contacted again and provided with statistics on their students' participation together with a request to publicize the availability of the survey again. Program chairs and graduate deans were automatically informed about the survey as soon as someone from their program or university completed the survey and provided the administrator's e-mail address. At least 60 universities, 115 doctoral programs, 60 graduate student associations, 80 professional societies, and 19 additional associations told the Survey Team about their efforts to publicize the survey through newsletters, e-mail lists, flyers, and Web sites. These organizations are listed as "Supporters" at <http://survey.nagps.org/about/>. Many professional societies' publicized the survey via their newsletters and e-mail mailing lists, and the Chronicle of Higher Education's Career Network, Science's Next Wave, and Physics Today all ran articles on the survey.

After completing the survey, respondents were directed to a confirmation screen that included an invitation to send an e-mail about the survey to other graduate students by completing an online form. This grass-roots, viral publicity mechanism was one of the most effective means of survey promotion.

Several measures were employed to promote the validity of responses. First, participants were asked to make an explicit affirmation that their responses were truthful. Responses without such an affirmation were discarded. Second, participants were prompted for their e-mail address, and a code was sent to that address. Participants were prompted for the code at the end of the survey to verify that the participant had provided a correct e-mail address. Free text comments have been reported only for participants who verified their addresses in this fashion.

The National Doctoral Program Survey is an observational study, not a controlled experiment. Participants were self-selected, and as a result their responses may not fully reflect the opinions of the entire doctoral student population. Despite the self-selection, there is important evidence that the concerns expressed in the survey responses represent widely-held student opinions rather than a small but outspoken set of negative voices.

1. As noted above, the vast majority of students expressed satisfaction with their programs and with their advisors overall.
2. Our survey reached a broad cross section of the graduate student population. The demographics of the survey participants, after controlling for discipline, are similar to the demographics of graduate students and recent cohorts of Ph.D.'s. Selection bias appears to have had little effect on participant demographics. See http://survey.phds.org/about/sample.php for details.
3. When compared to a survey conducted using more traditional methods, a pilot survey using methods similar to ours showed very similar findings. The 1999 PhDs.org Graduate School Survey, on which the National Doctoral Program Survey is based, shared a number of questions with a recent national study conducted at the University of Wisconsin-Madison and funded by the Pew Charitable Trusts (Golde and Dore 2001, http://www.phd-survey.org/). The Survey on Doctoral Education and Career Preparation obtained responses from 42.3% percent of all students surveyed: doctoral students in their third year or beyond in 11 disciplines at 27 leading research universities. Student assessments of their educational experiences as measured in the Madison study were similar to those in the PhDs.org study. In fact, the comparison of the two suggests that our results, if anything, may be positively biased, as the responses in the Golde and Dore study were nearly uniformly more negative than those in the PhDs.org study. See http://survey.phds.org/about/sample.php for details.

The National Association of Graduate-Professional Students (NAGPS) was founded in 1986 as a 501(c)(3) nonprofit organization dedicated to improving the quality of graduate and professional student life in the United States. To this end, NAGPS works to actively promote the interests and welfare of graduate and professional-degree-seeking students in public and private universities, as well as in the public and private agencies at the local, state, and national levels. In addition, through its national office and regional networks, NAGPS acts as a clearinghouse for information on graduate and professional student groups at all stages of development.

NAGPS members include over 130 graduate student associations at universities throughout the United States, together representing nearly 900,000 graduate and professional students. Affiliate memberships are also held by over 65 institutions. NAGPS maintains a close working relationship with other organizations dedicated to post-graduate education, including the Council of Graduate Schools, the Association of American Colleges and Universities, the National Black Graduate Students Association, the Graduate Student Caucus of the Modern Language Association, the Coalition of Graduate Employee Unions, the American Medical Student Association.

Malaina L. Brown was chair of the NAGPS Research Committee and Principal Investigator for the survey from its inception through Summer 2000. She focused on foundation relations, financial and project oversight, publicity, and funding.

Dr. Geoff Davis is author of the 1999 PhDs.org Grad School Survey on which the National Doctoral Program Survey is based and creator of the Sloan-funded PhDs.org Web site. In addition, he served as chief technical consultant for the NAGPS Survey and is the author of most of the software for the survey and its presentation and analysis.

Adam P. Fagen is chair of the NAGPS National Doctoral Program Survey Committee and a doctoral student at Harvard University. He has been involved with the project from its inception, focusing on public and organizational relations, publicity, survey design, and technology coordination.

Dr. Susan M. Niebur was co-chair of the NAGPS National Doctoral Program Survey Committee and recently received her Ph.D. from Washington University in St. Louis. She has been involved with the project from its inception, focusing on institution and association relations, publicity, survey design and coordination of the survey advisory board.

Kimberly Suedkamp Wells is president of NAGPS and a doctoral student at the University of Missouri-Columbia. She has served as project coordinator since October 2000 and has focused on project and financial oversight, legal issues and advisory board relations.

American Astronomical Society. 1997. An Examination of Graduate Education in Astronomy. http://www.aas.org/publications/baas/v29n5/edrpt.html

Association of American Universities. 1998. Report and Recommendations, Committee on Graduate Education. http://www.aau.edu/reports/GradEdRpt.html

Council of Graduate Schools and Graduate Record Examinations. 1998. Survey of Graduate Enrollment.

Golde, C.M., and T.M. Dore. 2001. At Cross Purposes: What the experiences of doctoral students reveal about doctoral education. Philadelphia, PA: A report prepared for The Pew Charitable Trusts. http://www.phd-survey.org/

Modern Language Association. 1997. Final Report of the MLA Committee on Professional Employment. http://www.mla.org/reports/profemp/profemp_frame.htm

National Academy of Sciences Committee on Science, Engineering, and Public Policy. 1995. Reshaping the Graduate Education of Scientists and Engineers. http://www.nap.edu/readingroom/books/grad/summary.html

National Research Council. 1998. Trends in the Early Careers of Life Scientists. Office of Scientific and Engineering Personnel, National Academy Press. http://www.nap.edu/readingroom/books/trends/

National Science Board. 1998. The Federal Role in Science and Engineering Graduate and Professional Education, NSB 97-235. http://www.nsf.gov/cgi-bin/getpub?nsb97235