THES has published their second ranking of leading universities around the world here (you can register for a 2-week free trial period to see the rankings). In format it looks like a U.S. News excercise, but (and this is key!) the focus is (largely) on the research and graduate quality of institutions. 60% of the overall rank is, happily, based on research/faculty quality: 40% based on an international reputation survey of academics; 20% on citations as measured by Thomson ISI's database (which is skewed towards medicine and the hard sciences). That's the good news!
Unfortunately, being journalists, the THES folks apparently have no real idea what they're doing, so mixed in with these research-oriented criteria are incommensurable and unrelated factors: an evaluation of graduates by employers (10%); the percentage of international faculty and students (5% each); and faculty/student ratio (20%). These factors aren't uninteresting, but how they should be weighted relative to the research strengths of an institution is utterly mysterious.
THES is clearly still figuring out what it's doing, and some of the mistakes are rather dramatic. In 2004, for example, the University of Texas at Austin was ranked 15th in the world by THES, and in the top ten in the US, a result that was, shall we say, a bit surprising. Scrutinizing the data, it became clear to me what had happened: Texas was ranked in the top ten in citations per faculty member, an astonishing result for a university without a medical school (research in medicine drives citation measures), but one that would make good sense if the University of Texas Southwestern Medical School at Dallas (one of the real medical powerhouses in the world, with more Nobel Laureates on its faculty than any other medical school in the US) were counted as part of the Austin results, which is obviously what had happened. This year, THES apparently figured out that Austin and Dallas are two different campuses, and UT Austin's citation per faculty member dropped dramatically--as did its overall rank (from 15th in the world, to 26th, and to 12th in the US--more realistic results).
Although THES claims to include only schools with undergraduates, they rank the University of California, San Francisco 17th in the world, though UCSF is exclusively devoted to graduate education in the health fields (that it fares so well is one indication of how skewed the results are to universities with strengths in medicine and biological sciences). Duke jumped from outside the top 50 last year into the top 15, a change that can only mean there was some kind of mistake last year or this (probably last year--my guess is THES neglected to count the medical school).
All this being said, the core THES data on research quality of institutions is interesting, and the effort is far superior to the absolutely bizarre "world university rankings" being put out by a Chinese university, which rewards institutions for its dead Nobel Laureates (if you used to be really good, that counts in your favor here!) and--because it has no peer review component and relies heavily on citation measures--is hugely skewed to schools strong in medicine, engineering, and the hard sciences (a first-rate "arts and humanities" school would barely register in the Chinese rankings). On the other hand, THES is not sufficiently forthcoming about its methods and underlying data to permit a thorough evaluation of its results.
According to the 2005 Times Higher Education Supplement, here are the top 20 American research universities (excluding the medical school at UC San Francisco, for the reasons noted above); given the big impact of medicine on these rankings, I note which schools also have medical schools, and note, very roughly, its quality. Bear in mind that this is the overall ranking based on the odd stew of criteria noted above:
1. Harvard University (excellent medical school)
2. Massachussetts Institute of Technology (no medical school)
3. Stanford University (excellent medical school)
4. University of California, Berkeley (no medical school)
5. Yale University (strong medical school)
6. California Institute of Technology (no medical school)
7. Princeton University (no medical school)
8. Duke University (excellent medical school)
9. Cornell University (strong medical school)
10. University of Chicago (strong medical school)
11. Columbia University (strong medical school)
12. University of Texas, Austin (no medical school)
13. Johns Hopkins University (excellent medical school)
14. University of Pennsylvania (excellent medical school)
15. University of Michigan, Ann Arbor (strong medical school)
16. University of California, Los Angeles (strong medical school)
17. University ofCalifornia, San Diego (strong medical school)
18. Carnegie-Mellon University (no medical school)
19. Northwestern University (average medical school)
20. Boston University (average medical school)
Apart from the presence of BU and the absence of Wisconsin and Illinois (and putting aside quibbles about relative placements) this is not a surprising top 20 list. (This isn't gratuitous BU-bashing, or Wisconsin/Illinois-praising: I'm going on other data about research and faculty quality, such as the National Research Council rankings, membership in learned societies, and the like. By these indicators, the BU performance is completely bizarre.)
More interesting, perhaps, are the results when we look just at the "peer review" scores, i.e., the evaluations by other academics around the world. Because the surveys were international in character, and although THES claims to have balanced the response pools by geographic region and fields, one still suspects that fields which are truly international (like the sciences and medicine) fare better than those fields which are more geographically bounded (like the humanities and many of the social sciences). Still, the results are a lot closer to the mark in depicting research strength than the amalgmated results, above, based on the stew of incommensurable criteria. (The one exception, again, is Boston University: I suspect this will turn out to be this year's "big mistake," probably attributable to some error in data compilation. The U Mass result is also somewhat surprising.) Here are the overall peer evaluation scores for the U.S.
1. Harvard University (100)
2. University of California, Berkeley (95)
3. Massachussetts Institute of Technology (84)
4. Stanford University (78)
5. Yale University (71)
6. Princeton University (69)
7. Columbia University (56)
7. Cornell University (56)
9. University of California, Los Angeles (52)
9. University of Chicago (52)
11. Johns Hopkins University (50)
12. California Institute of Technology (48)
13. University of Texas, Austin (47)
14. University of Michigan, Ann Arbor (46)
15. University of California, San Diego (43)
16. University of Pennsylvania (42)
17. Boston University (41)
18. University of Illinois, Urbana-Champaign (39)
19. University of Massachussetts, Amherst (38)
20. Duke University (36)
20. New York University (36)
20. Purdue University (36)
20. University of Wisconsin, Madison (36)
All well and good, but where the peer review results get a bit more c urious is when we start looking at the specialty areas, broken down as "Arts & Humanities," "Social Science," "Science, "Technology," and "Biomedicine." Unfortunately, there is no indication of what disciplines fall within the scope of each of these categories.
We get some idea of the importance of strength in medicine and the biological sciences when we look at the U.S. top 20 based on peer evaluation in "Biomedicine"; note that only Berkeley, MIT, Princeton, and Cal Tech on this list lack medical schools; UC San Francisco and Baylor are devoted exclusively to medicine and the life sciences. Schools like Hopkins and Duke get their highest scores in this category, indicating the extent to which their overall rank is driven by their outstanding medical schools.
1. Harvard University (100)
2. Stanford University (81)
3. Johns Hopkins University (77.7)
4. University of California, Berkeley (69.2)
5. Yale University (63.9)
6. Massachussetts Institute of Technology (60.6)
7. University of California, San Diego (59.1)
8. University of California, San Francisco (54.9)
9. Duke University (52.1)
10. Columbia University (50.6)
11. Cornell University (50.5)
12. University of California, Los Angeles (44.2)
13. Princeton University (42.7)
14. California Institute of Technology (41.5)
15. Baylor College of Medicine (39.6)
16. Boston University (38.7)
17. University of Michigan, Ann Arbor (37.7)
18. University of Washington, Seattle (36.5)
19. University of Chicago (35.9)
20. University of Pennsylvania (33.8)
The U.S. top 20 in "Science" (which presumably means physics, chemistry, mathematics, and astronomy, as well as those biological sciences unrelated to medicine) is a fairly plausible list:
1. University of California, Berkeley (92.7)
2. Harvard University (89.9)
3. Massachussetts Institute of Technology (87.3)
4. Princeton University (80.4)
5. Stanford University (79.1)
6. California Institute of Technology (72.4)
7. Cornell University (64.3)
8. Yale University (60.1)
9. University of Chicago (58.3)
10. University of California, Los Angeles (49.3)
11. University of California, Santa Barbara (46.8)
11. University of Texas, Austin (46.8)
13. University of Illinois, Urbana-Champaign (45.9)
14. Columbia University (45.8)
15. University of Michigan, Ann Arbor (39.3)
16. Johns Hopkins University (39)
17. University of California, San Diego (36.7)
18. University of Pennsylvania (33.8)
19. University of Wisconsin, Madison (31)
20. Boston University (30.9)
The top 20 U.S. schools in the "Technology" areas (presumably meaning Engineering, but perhaps not only that) show the importance of this category for schools like Carnegie-Mellon, Purdue and Texas A&M, which get their highest scores here:
1. Massachussetts Institute of Technology (100)
2. University of California, Berkeley (98.7)
3. Stanford University (84.9)
4. California Institute of Technology (78)
5. Carnegie-Mellon University (65.8)
6. Georgia Institute of Technology (58.7)
7. Harvard University (58.3)
8. University of Illinois, Urbana-Champaign (54)
9. University of Texas, Austin (53.4)
10. Cornell University (51.5)
11. Purdue University (51.2)
12. University of California, Los Angeles (50.6)
13. Princeton University (49.8)
14. University of Massachussetts, Amherst (46.2)
15. University of Michigan, Ann Arbor (42.5)
16. University of California, San Diego (40.9)
17. Yale University (40.5)
18. University of Wisconsin, Madison (39.2)
19. Texas A&M University (38.8)
20. Boston University (37)
Here are the top 20 U.S. universities in "Arts & Humanities" based on peer evaluation according to THES:
1. Harvard University (100)
2. University of California, Berkeley (77.8)
3. Yale University (77.4)
4. Princeton University (69.2)
5. Columbia University (56.5)
6. Massachussetts Institute of Technology (53.5)
7. University of Texas, Austin (50.2)
8. Georgetown University (45.4)
9. University of Chicago (44.1)
10. University of Michigan, Ann Arbor (43.9)
11. University of California, Los Angeles (41.5)
11. University of Pennsylvania (41.5)
13. Brown University (40.9)
14. Johns Hopkins University (38.3)
15. University of Virginia (36.7)
16. New York University (36.5)
16. University of Utah (36.5)
18. Duke University (34.3)
19. Purdue University (32.5)
19. University of Illinois, Urbana-Champaign (32.5)
19. University of New Mexico (32.5)
19. University of Wisconsin, Madison (32.5)
The top five make perfect sense, but then the list becomes, shall we say, surprising. One wonders what disciplines are being counted? Linguistics and Philosophy at MIT are very strong, but is Linguistics being counted here, and should two fields be sufficient? Classics, Philosophy, Latin American History are strong at UT Austin, but is that enough to make Texas better than Chicago in "Arts & Humanities"? Georgetown is a complete surprise. Perhaps the emphasis is being laid on "Arts," which would give an advantage to a school like Texas with reputable programs across the arts (from Art History to Film to Music) in contrast to schools like Chicago which don't offer programs in all these fields. In any case, it is a surprising top 20 list. Note, too, that Cornell is nowhere to be found on this list, even though it has strong programs in most major Humanities fields.
Now take a look at the top twenty in "Social Science" according to THES's peer evaluations:
1. Harvard University (100)
2. University of California, Berkeley (85.3)
3. Stanford University (80.1)
4. Yale University (77.3)
5. Massachussetts Institute of Technology (73.9)
6. University of Chicago (73)
7. Princeton University (68.9)
8. Columbia University (63.3)
9. Cornell University (57.1)
10. New York University (50.3)
11. University of Pennsylvania (48.6)
12. University of California, Los Angeles (47)
13. Boston University (46.1)
14. Carnegie-Mellon University (43)
15. University of Michigan, Ann Arbor (41.5)
16. Northwestern University (39.5)
17. University of Massachussetts, Amherst (39.4)
18. University of Texas, Austin (36.2)
19. Duke University (36)
19. Pennsylvania State University (36)
How can Michigan be 15th in the "Social Science" category, given its top five (by every other measure) departments in Political Science, Sociology, Psychology, and Anthropology? Is its relative weakness in Economics dragging it down? (The strong showing of MIT here suggests Economics is very important to the category.) It's all a bit mysterious. The results change a good bit if you then look at THES's citation per paper measure in Social Science; here are the top ten:
1. University of Chicago (9.9)
2. Carnegie-Mellon University (9.4)
2. Harvard University (9.4)
4. Massachussetts Institute of Technology (8.9)
5. Stanford University (8.5)
5. University of California, San Diego (8.5)
7. Northwestern University (8.1)
7. University of Rochester (8.1)
9. Duke University (8)
9. Princeton University (8)
Michigan's position doesn't change much (it has 7.6 cites per paper), while Berkeley, with a "mere" 5.3 cites per paper (the same as UT Austin) doesn't even make the top twenty.
Only seven schools rank in the top 20 in all five areas (Harvard, Berkeley, Yale, MIT, UCLA, Princeton, and Michigan); seven additional schools rank in the top 20 in at least four areas (Stanford, Columbia, Cornell, Chicago, Penn, Texas, and, surprisingly again, BU). Five schools are in the top 20 in at least three areas (Johns Hopkins, UC San Diego, Duke, Cal Tech, and Illinois). Here is how these schools are evaluated in the specialty areas (they are listed alphabetically):
BOSTON UNIVERSITY
16 in Biomedicine
20 in Science
20 in Technology
Not in top 20 in Arts & Humanities
13 in Social Science
CALIFORNIA INSTITUTE OF TECHNOLOGY
14 in Biomedicine (despite not having a medical school)
6 in Science
4 in Technology
Not in top 20 in Arts & Humanities
Not in top 20 in Social Science
COLUMBIA UNIVERSITY
10 in Biomedicine
14 in Science
Not in top 20 in Technology
5 in Arts & Humanities
8 in Social Science
CORNELL UNIVERSITY
11 in Biomedicine
7 in Science
10 in Technology
Not in top 20 in Arts & Humanities (very suspicious result!)
9 in Social Science
DUKE UNIVERSITY
9 in Biomedicine
Not in top 20 in Science
Not in top 20 in Technology
18 in Arts & Humanities
19 in Social Science
HARVARD UNIVERSITY
1 in Biomedicine
2 in Science
7 in Technology
1 in Arts & Humanities
1 in Social Science
JOHNS HOPKINS UNIVERSITY
3 in Biomedicine
16 in Science
Not in top 20 in Technology
14 in Arts & Humanities
Not in top 20 in Social Science
MASSACHUSSETTS INSTITUTE OF TECHNOLOGY
6 in Biomedicine (despite not having a medical school)
3 in Science
1 in Technology
6 in Arts & Humanities
5 in Social Science
PRINCETON UNIVERSITY
13 in Biomedicine (despite not having a medical school)
4 in Science
13 in Technology
4 in Arts & Humanities
7 in Social Science
STANFORD UNIVERSITY
2 in Biomedicine
5 in Science
3 in Technology
Not in top 20 in Arts & Humanities
3 in Social Science
UNIVERSITY OF CALIFORNIA, BERKELEY
4 in Biomedicine (despite not having a medical school)
1 in Science
2 in Technology
2 in Arts & Humanities
2 in Social Science
UNIVERSITY OF CALIFORNIA, LOS ANGELES
12 in Biomedicine
10 in Science
12 in Technology
11 in Arts & Humanities
12 in Social Science
UNIVERSITY OF CALIFORNIA, SAN DIEGO
7 in Biomedicine
17 in Science
16 in Technology
Not in top 20 in Arts & Humanities
Not in top 20 in Social Science
UNIVERSITY OF CHICAGO
19 in Biomedicine
9 in Science
Not in top 20 in Technology
9 in Arts & Humanities
6 in Social Science
UNIVERSITY OF ILLINOIS, URBANA-CHAMPAIGN
Not in top 20 in Biomedicine
13 in Science
8 in Technology
19 in Arts & Humanities
Not in top 20 in Social Science
UNIVERSITY OF MICHIGAN, ANN ARBOR
17 in Biomedicine
15 in Science
15 in Technology
10 in Arts & Humanities
15 in Social Science
UNIVERSITY OF PENNSYLVANIA
20 in Biomedicine
18 in Science
Not in top 20 in Technology
11 in Arts & Humanities
11 in Social Science
UNIVERSITY OF TEXAS, AUSTIN
Not in top 20 in Biomedicine (no medical school at Austin campus)
11 in Science
9 in Technology
7 in Arts & Humanities
18 in Social Science
YALE UNIVERSITY
5 in Biomedicine
8 in Science
17 in Technology
3 in Arts & Humanities
4 in Social Science
Just wanted to note that the University of Chicago does not have an engineering school, so if that's what they mean by technology, it makes sense that we're not in the top 20.
Posted by: Marsha | November 09, 2005 at 07:43 AM