September 29, 2010

NRC Releases Long Awaited Doctoral Program Rankings

After five years, $4 million and a lot of effort across many institutions, the National Research Council has released an update to their 1995 assessment of doctoral programs. A Data-Based Assessment of Research-Doctorate Program in the United States analyzes 2005-06 academic year data collected from over 5,000 doctoral programs at over 200 universities. The NRC collected data directly from faculty, students, graduate programs, and institutions. The Graduate School coordinated UW participation in the assessment, which you can learn more about on their website.

Programs are ranked on the same 21 key variables by two different methodologies, the results of which are reported separately. These methodologies were very complex, but, essentially, the “S” (Survey-Based) rankings weight the relative value of the 21 key variables by program, based on faculty ratings of the relative value of each variables in a given discipline. For example, in the physical sciences, the number of external grants won is weighted more heavily than it would be for an English program. Alternatively, the “R” (Regression-Based) ratings are more similar to the traditional ‘reputation ranking’ where faculty were asked to rank a set of random programs, and then the key variables most associated with the highest ranked programs were assigned the most weight in the overall analysis of programs. Both sets of rankings are reported as ranges (e.g. a program might be ranked as somewhere between 3rd and 11th, at a 90% level of confidence).

While many UW programs do well in these rankings, criticisms of both the data and methodology are important to consider. Inside Higher Ed weighs in with an assessment of the ambivalence surrounding the veracity of the rankings, and the UW’s own Dean of Engineering, Matt O’Donnell, released a statement about possible shortcomings. UW Computer Science & Engineering also issued a strong critique, on which the Chronicle of Higher Education reported. The Chronicle also compiled these data in an easy to use format and offered its own analysis of the report’s delay and overall worth.

How meaningful these rankings are will be debated in the days ahead, but there is at least one important and indisputable conclusion included in the report, which is that public universities play an outsized role in educating our nation’s graduate students:

“Seventy-one percent of the programs ranked in the NRC study are in public universities. The proportion of programs in the universities with the largest programs is similar (70 percent). Among the 37 universities that produced 50 percent of Ph.D.’s from 2002 to 2006, 70 percent were public. Although public universities rely increasingly on nonpublic sources of funding, cutbacks in public funding for universities has a powerful effect on doctoral education simply because of how many large Ph.D. programs exist in public universities.”