11 Canadian Universities Decline to Participate in Canadian Version of U.S. News Rankings

MOVING TO FRONT FROM AUG. 14, SINCE THERE IS NOW A LIVELY COMMENTS SECTION.

Story here; an excerpt:

Eleven Canadian universities advised Maclean's magazine on Monday that they will not participate in this year's survey that assigns rankings to each institution because of concerns about the methodology and the validity of some of the measures.

In a letter to Tony Keller, the magazine's managing editor of special projects, the universities said they have expressed their "considerable reservations" to Maclean's for some years, but to little avail.

"Thus far, these serious concerns have gone largely unaddressed, and there is still no evidence that Maclean's intends to respond to them," they said.

The universities said they already publish a lot of data online about themselves and intend to add more to allow people to make valid comparisons.

"However, it is truly hard for us to justify the investment of public funds required to generate customized data for your survey when those data are compiled in ways that we regard as oversimplified and arbitrary," they said.

The letter was signed by the presidents of:

• University of Toronto
• McMaster University
• University of Ottawa
• University of British Columbia
• Simon Fraser University
• University of Alberta
• University of Calgary
• University of Lethbridge
• University of Manitoba
• Université de Montréal
• Dalhousie University

The universities said they found it inappropriate that the survey collects data on a wide range of things — such as class size, faculty, finances, library and reputation — and then arbitrarily assigns weightings to generate a single ranking number....

It will be interesting to see what effect this has on the credibility of the Maclean's rankings.  (Canadian readers:  do they have any credibility?  Comments are open for non-anonymous postings.)  Certainly the fact that the two preeminent research universities in Canada--Toronto and British Columbia--are participating in the boycott should help.  One wonders, though, why McGill was not a signatory (do they fare especially well in Maclean's?). 

Of course, the other interesting question is why leading American research universities haven't followed suit?  I suppose the worry is that it would be very hard to get schools to stick to an agreement not to participate.  Schools like Harvard and MIT and Stanford can weather whatever abuse U.S. News would dole out to them if they didn't complete the surveys, but other schools could ill afford it, especially those (like Duke or Penn, among many others) that tend to be systematically overrated in U.S. News relative to other measures of academic quality.

It will be interesting to see what happens in Canada.

UPDATE:  My colleague Les Green's observations, from the comments section, deserve to be read:

The systematically overrated Canadian universities are indeed over-represented among the non-signatories. 

But the Macleans rankings have nothing like the influence in Canada that US News rankings have in the US. There are a number of reasons for this. One is that the quality gap between the top Canadian universities and the bottom ones is nothing like the gap between the top and bottom American universites, so the value of rankings to the prospective student is much less. This is in turn partly explained by the fact that there is no equivalent to the spectacularly rich private US schools in Canada; but neither is there any equivalent to the starved public degree-mills or the bizarre little religious and ideological enterprises that pretend to teach at a university level. Regulation puts a quality floor under Canadian tertiary education (and perhaps, controversially, also a ceiling above it). Then there is a much more pronounced regional culture up here. UBC is indeed a very good research university; but it isn't drawing many top students from Ontario or Quebec (the largest provinces). Most Ontario students just aren't all that interested in the differences, if any, between UBC and Simon Fraser. Finally, remember that the US-Canada border is, for the monied classes, porous. Occasionally one or the other of the better Canadian universities pretends to be the "Harvard of the North." But well-to-do parents aren't fooled. The Harvard of the North is *Harvard*. Finally, it seems to me that, bad as it is, US News is actually *better* at ranking than is Macleans, whose staff obviously lack the competence and contacts to do even a mediocre job of assessing the quality of Canadian universities.

ANOTHER UPDATE:  Canadian philosopher Thomas Hurka at the University of Toronto writes:

Interesting that you posted about the Canadian universities and the Maclean's ranking. One issue that was brought up years ago by the Alberta universties but got no response from the magazine is the following. Perhaps the most important factor in the overall Maclean's ranking is average grades of incoming students. But, as Les Green noted on your site, Canadian universities much more than US ones draw the bulk of their students locally, from their own provinces. And high school grading practices in different provinces are very different, e.g. Alberta has province-wide exams, which lowers grades (because you're not being graded by your own teacher, who has a stake in your success), while Ontario has a cash scholarship for averages above 80%, which
inflates grades (since teachers want their students to get the cash).  This systematically favours Ontario over Alberta universities, and in particular favours Queen's U, which has the highest incoming average in the country. It wouldn't be too hard to adjust for these differences -- just control by the percentage of a university's incoming class that's out-of-province and normalize by average grade 12 results per province.  But Maclean's refused to do anything like that. No doubt that partly
explains why all three Alberta universities are among those pulling out of the exercise, while Queen's is not.

Comments

[I. Glenn Cohen]

It is particularly interesting because the University of Toronto has consistently ranked first in its category (the rankings, as I recall, differentiate between smaller liberal arts schools and larger research universities) for the better part of the decade.

[Les Green]

The systematically overrated Canadian universities are indeed over-represented among the non-signatories.

But the Macleans rankings have nothing like the influence in Canada that US News rankings have in the US. There are a number of reasons for this. One is that the quality gap between the top Canadian universities and the bottom ones is nothing like the gap between the top and bottom American universites, so the value of rankings to the prospective student is much less. This is in turn partly explained by the fact that there is no equivalent to the spectacularly rich private US schools in Canada; but neither is there any equivalent to the starved public degree-mills or the bizarre little religious and ideological enterprises that pretend to teach at a university level. Regulation puts a quality floor under Canadian tertiary education (and perhaps, controversially, also a ceiling above it). Then there is a much more pronounced regional culture up here. UBC is indeed a very good research university; but it isn't drawing many top students from Ontario or Quebec (the largest provinces). Most Ontario students just aren't all that interested in the differences, if any, between UBC and Simon Fraser. Finally, remember that the US-Canada border is, for the monied classes, porous. Occasionally one or the other of the better Canadian universities pretends to be the "Harvard of the North." But well-to-do parents aren't fooled. The Harvard of the North is *Harvard*. Finally, it seems to me that, bad as it is, US News is actually *better* at ranking than is Macleans, whose staff obviously lack the competence and contacts to do even a mediocre job of assessing the quality of Canadian universities.

[Suman Chakraborty]

I agree with the previous poster - the University of Toronto's rankings are always at the top and its decision not to participate is particularly notable (Maclean's breaks their list up into three categories - Primarily Undergrad, Comprehensive and Medical Doctoral). The real effect though is on numbers given the comparatively small number of institutions - 11 out of 67 schools profiled (according to the latest guide) is a big blow.

[Ravi Malhotra]

I agree with Prof. Green. I wonder how much longer MacLeans' survey can last now. Hopefully its days are numbered. I think a useful approach for Americans to think about MacLeans is to regard it akin to Brennan's (bizarre) rankings of law schools rather than US News. To provide context, Brennan has ranked Minnesota above Harvard.

[Ed]

One reason why this type of boycott is easier in Canada than it would be in the U.S. is the sheer number of American colleges and universities. Getting a majority of the top American institutions to agree to boycott U.S. News and to live up to their agreements would be both riskier and more labor-intensive. As you say, universities like Duke (especially) and Penn have a lot to lose if the U.S. News rankings go away -- or even correct some of their shortcomings. These schools may do even better if they are the only ones providing accurate information, so other universities have a strong disincentive to join such a boycott unless pretty much everyone is on board.

[Matt]

Just as a point of interest, the Wharton School at Penn _has_ recently boycotted a number of business school rankings, as have, I believe, several other top business schools.

[DJ]

Some of the comments above illustrate why Canadian universities don't want to play the ratings game like US universities. The US News rankings, and the generally overly competitive attitude of many US universities, are a turn-off to many other university traditions - especially ones that don't expect their students to pay insane tuition and remember that they are indeed universities and not businesses. I lecture law in the UK, where rankings systems are almost universally derided and even Oxbridge doesn't reach the heights of arrogance of Harvard, Yale, etc. Penn, Duke, and other institutions are excellent schools (as are Toronto, McGill, and so on), no matter they aren't ranked as high as some others. And as for postgraduate studies - one follows the expert in one's chosen area, not necessarily the highest ranked school.

[Brendan]

Here are some thoughts from a recent canadian undergrad. I fear they are a little chaotic, but they might be of interest.

Having done my undergraduate at the lowest ranked large research university in canada (according to Maclean's) I can safely say that our university (I still work there in various ways) is not impressed. For instance, we have an incredibly low acceptance grade (like, a C, or low 60s) and I think the school sees itself as offering a service to the province. this means our average entrance grades, exit grades, class sizes, and (thankfully) tuition are quite low. It also means that the bottom feeder classes are quite large with people of little intellectual ability.
Of course, that means nothing for those who are serious students, for instance, we also apparently have some of the most rhode scholars of any canadian university (at minimum, I know we are nowhere near the bottom). We also have certain areas of research in which we are quite strong, or certain programs that are: our law school, for instance, is usually about 6th, when it is ranked.
And yet, we keep breaking our records for yearly enrolment, which means bigger classes, lower grade averages etc.--cementing our position at the bottom of maclean's list.

On anote note, the small university in my city was ranked 4th in canada one year. then it dropped severely, and, as far as I know, the school has not changed very. Of course, the schools that are now ranked ahead might have improved, but the point is that this particular school did not get worse. that, however, will not be the perception for those looking at the rankings.

No one I have ever talked to has taken Maclean's seriously. for my own part, if I had gone somewhere else, and shunned my local school because of its low ranking, then, with the exceptions of top ranked canadian schools on the leiter report I would have probably gotten a worse philosophical education.

[porlock junior]

I'm sure Steven Koblik, formerly president of Reed College, was delighted to hear that the Canadian universities had decided to follow the example the college set ten years ago when it dropped out of the US News rankings game. As will Colin Diver, the new president, whose essay on not cooperating with the rankings is at
http://www.theatlantic.com/doc/200511/shunning-college-rankings

Now why can't the big-name big schools in the US get together to subvert the system? They of all schools could afford the damage they'd supposedly suffer. And check Diver's article for how much damage it really has done to small-fry (prestigious, but small fry) Reed.

[T. Bailey]

I am glad to see some action on this question -- a sizeable number of my classmates at the University of Victoria school of law enrolled there because it had garnered first place in the Macleans law school rankings the previous year. The problem with this was that these same students had not considered the political character of the school, nor it's well-known focus on environmental and aboriginal law. They grumbled constantly about the lack of business and corporate-commercial upper year course offerings, and whined openly about the school's "political correctness." Without the Macleans ranking, perhaps they might have researched the school in detail before applying.

There is another Canadian magazine which also ranks law schools -- I believe it's Canadian Lawyer magazine. I heard (from a former dean of UVic Law) that although the methodology is supposedly more sound, it is primarily a ranking by graduates of their particular schools. It seems to me that, unless the interviewees graduated at least five years previous to the survey, the results would be questionable indeed (since law graduates from some schools are viewed more favourably than others, as a matter of expediency on the part of firms). Thus, the more reliable responses would also be the less current ones, and subsequently the value of the entire endeavour seems suspect. Assuming my information is correct.

[Thom Brooks]

I'd like to flesh out a bit more the line of thought in DJ's post.

In the UK, "league tables" of institutions and their subjects are universally derided, indeed. In fact, the only ones that take them seriously seem to be the newspapers that publish them...and university heads (who like to manipulate the "results" in marketing material, perhaps understandably so). Perhaps one of the primary reasons that these rankings are widely condemned seems to be one of the main reasons why Canadian law schools are boycotting MacLean's survey: the newspapers (such as the Guardian and the Times) collects data on average entry scores, student:faculty ratios, £'s per student spent on the library, etc. and then produce a metric where these things are given a curious weight (often equal). Thus, a top-rated department in a complex, burdensome RAE (that already accounts for a number of factors) counts as one measure, alongside class sizes which is an equally weighted second measure. And so on. You end up with a positively ridiculous (and scandalous) ranking of departments and institutions, often with some departments (such as in philosophy) making the top ten....even though they don't exist! The newspapers claim they welcome feedback, but complaints are treated as sour grapes from folks who've underperformed. It's incredible. (And I have thought of taking your lead and doing for Politics in the UK what you've done for Law and Philosophy, Brian.)

Of course, increasingly insane tuition fees seem on the horizon in the UK---Imperial has made known its desire to charge as much as £15,000 per year and the top up fee of £3,000 per year for EU students (about £7,800 for non-EU) per year looks set to increase ahead of schedule, perhaps doubling. Competition has spread to the UK....

[PaulRaymont]

It's good to see this happen, partly because Maclean's is becoming a really trashy magazine. It's being replaced by the Walrus as the respectable Canadian news magazine.

The universities listed above are of very high quality. Toronto and UBC are great, but so too are the less well known Dalhousie, Alberta, Calgary, McMaster and the francophone U de Montreal. They're all important regional research centres, many of which have good medical schools. They're too good and prominent to be hurt by anything Maclean's might do in retaliation -- the rankings would lose all credibility with the broader public if these schools were suddenly ranked lower.

Maclean's has hyped lots of small, undeserving schools. For example, there are several small universities in Nova Scotia and New Brunswick that have scored highly but that aren't very good.

The best example is Mt. Allison University. My uncle taught there for about thirty-five years. For most of that time, there was no pretense that Mt. A was part of an elite. But then Maclean's put them at the top of its category for smaller universities for several years running. Suddenly, this mediocre school with relatively undistinguished faculty was esteemed as some sort of Smith College North.

Maclean's fostered the myth that the smaller Maritime universities were like the best New England liberal arts colleges, where bright students enjoy individual attention from excellent profs. However, there's nothing like that good old American liberal arts school tradition in Canada. The best schools for undergrads and grads in Canada have always been the larger research centres, where bright students get some attention from generally excellent profs and lots of good TAs.

It's notable that none of those smaller Maritime schools are on the above list of eleven dissenters.

I've taught in contractually limited positions at several Canadian schools (UBC, Calgary, Carleton, Toronto, York, Trent, etc.), and the Maclean's rankings have always struck me as totally unhelpful. They encourage selecting a school based on some cool marketing image that impressed the rankers (the liberal arts image, or the Oxbridge aura that emanates from McGill and Queen's) rather than on the basis of truly relevant criteria.

[Tad Ekam]

When I worked at my school's undergrad newspaper, and my girlfriend worked at her school's undergrad newspaper, we would get a yearly call from Macleans asking what the new campus trends were. We had a competition to see what the silliest things we could convince them to print were. I will report that Macleans has standards enough not to report that self-trepanation was a massive trend amongst undergrads.