11 universities opt out of Maclean's survey

[winner2000]

Funny how a good number of those schools rank pretty poorly on the list...


11 universities opt out of Maclean's survey

CAROLINE ALPHONSO

Globe and Mail Update

Eleven of Canada's universities are refusing to participate in the influential Maclean's university rankings, saying they find the magazine's survey methodology to be "oversimplified" and "arbitrary."

In a letter sent Monday to Maclean's, the coalition said they will not provide any data for the magazine's popular, yet controversial, annual survey of universities, coming out this fall.

The letter was signed by the presidents of the University of British Columbia, Simon Fraser University, University of Alberta, University of Calgary, University of Lethbridge, University of Manitoba, University of Toronto, McMaster University, University of Ottawa, University of Montreal and Dalhousie University.

"In various ways and for some years, many institutional spokespersons have expressed considerable reservations about the methodology used in the Maclean's university survey and the validity of some of the measures used," the presidents wrote.

"Thus far, these serious concerns have gone largely unaddressed, and there is still no evidence that Maclean's intends to respond to them."

The Maclean's university ranking, a competitor to The Globe and Mail's University Report Card, has been the subject of much debate. Universities have objected to being scored and ranked. The letter will no doubt lead to other schools pulling out of the rankings, sources said.

The letter comes on the heels of universities saying they didn't want to participate in Maclean's recent graduate survey. Despite objections from various universities, the issue, which looked at alumni satisfaction with their university experience, was still published using data from various sources.

Tony Keller, managing editor of special projects at Maclean's, said in an interview Monday that the 16th annual rankings issue will continue in its current form regardless of whether institutions willingly participate.

"This data is all available and we will be publishing it," he said.

Among their objections, the university presidents say that they are concerned that the Maclean's rankings aggregate data from a range of variables. The magazine, they say, arbitrarily assigns weights to variables that are of questionable validity. This, they say, does a disservice to students.

"The variables selected by Maclean's also fail to capture the breadth of experiences students say are important in their university education," they write.

Indira Samarasekera, president of the University of Alberta, said in an interview: "We felt that if they were not using the data in a way that we considered rigorous, given that we are academic institutions that are committed to accuracy, transparency and quality ... we really couldn't justify public institutions spending taxpayer dollars to support the exercise."

Mr. Keller, however, said the methodology used by Maclean's is fair.

"Based on 16 years of experience of doing this at Maclean's, based on extensive consultation with the universities and with experts in the field, we came up with a ranking of all the elements that make up quality in a university," he said.

[civ@uw]

Hasn't UofT done fairly well in these rankings?

[divx]

Hasn't UofT done fairly well in these rankings?
yes, ranked 2nd for eng, that's pretty good

[soulflare]

They're just pissed that Western won... and now, between the Saugeen Stripper and half the nation quitting, we're assured to defend our crown. :lol:

[jljdaigl]

Ya but alot of those schools rank pretty low, haha, i guess they dont like to see it in writting

[Chief]

Ya but alot of those schools rank pretty low, haha, i guess they dont like to see it in writting
Well looking at independant world rankings, UofT, UBC, and Mac are 3 of the 4 Canadian universities that rank in the top 100 (24th, 36th, and 88th respectively.) So if 3 of the best ranked by an independant world authority have called MacLeans rankings "oversimplified" and "arbitrary," I believe them.
O, back to school thread, so I guess I should source this information. (http://www.royaledu.net/PDF/2004_academic_ranking_top500.pdf)

[Sylvestre]

The letter was signed by the presidents of the University of British Columbia, Simon Fraser University, University of Alberta, University of Calgary, University of Lethbridge, University of Manitoba, University of Toronto, McMaster University, University of Ottawa, University of Montreal and Dalhousie University.

low? UBC, UofA, UofC, UofT, Mac, OttawaU and SFU are all top schools (and that's more than 50% of the list). If anything, what's interesting is most aren't in GTA. Most aren't even in Ontario.
I agree with them. I've always found the results are biased for Ontario schools, and for schools with top post-graduate programs (which are rarely indicative of good undergrad educations)

[JimG]

Macleans gets their results by sending out surveys. How can you compare things about student life when most people have only gone to one or two universities. You can't say whether UofT or York or Mac or Waterloo is better if you haven't been to all of them.

[ynchu]

SFU wants out because they constantly got lower rankings in various areas. Remember back in the days SFU used to say it won the "most comprehensive" award? Now what? Because SFU ain't winning, it wants out?

The fact is simple and obvious. Start spending more per student and SFU will do better! Don't try to hide that fact that SFU spends almost the least per student.

[MizTEcK]

what i'd like to see is a separate ranking for undergraduate and graduate schools. UT for example is only prominent for its graduate studies and research, undergrade is horrible compared to other universities.

[nkwu]

I thought Western was rated highest for their mid year thing, part of their regular circulation, but they never fared well in the Maclean's big "Guide to Canadian Universities" that is seperate from circulation.

[Eyies]

Interesting news, thanks OP.

[elemenopee]

Doesn't sound like it's going to make any difference



from http://www.herald.ns.ca/Front/522173.html

Dalhousie president, Tom Traves — along with the leaders of universities in Ontario, British Columbia, Quebec, Manitoba and Alberta — addressed a letter Monday to the magazine’s special projects editor, Tony Keller, outlining the reason for dropping out of the 2006 survey.

....

Mr. Keller told The Canadian Press that even though some universities have refused to participate in the survey, they will still be assessed for the annual University Ranking Issue, which hits newsstands Nov. 2.

"Our (upcoming) issue . . . will contain all the data it’s always contained and it will rank every school that it’s always ranked, including every one of the universities mentioned in the letter," he said. "From a reader’s perspective, this isn’t going to change anything."

[longo]

saw this in the paper today! I'm with the universities. Down with Macleans rankings!

[Rocky1231]

Here's the actual letter they sent:


August 14, 2006


Mr. Tony Keller
Managing Editor, Special Projects
Maclean’s 11th floor
One Mount Pleasant Road
Toronto, ON M4Y 2Y5


Dear Mr. Keller:

We regret to advise you that our universities will not be participating in the 2006 Maclean’s questionnaire.

We share Maclean’s goal of providing good information for students and their families who are researching post-secondary education. We also compliment you on your editorial coverage of the post-secondary sector. Many of the articles in Maclean's have contributed to the national discussion about post-secondary education, and have helped to frame students’ choices. Our concern relates specifically to Maclean's attempts to generate a global ranking of Canadian universities.

In various ways and for some years, many institutional spokespersons have expressed considerable reservations about the methodology used in the Maclean’s university survey and the validity of some of the measures used. Thus far, these serious concerns have gone largely unaddressed, and there is still no evidence that Maclean’s intends to respond to them.

We welcome public assessment of our work, and all our institutions devote significant resources to that end. We already publish a great deal of data about our own institutions on-line and intend to publish more in future, ideally in the form of standardized datasets that will facilitate valid temporal and institutional comparisons. However, it is truly hard for us to justify the investment of public funds required to generate customized data for your survey when those data are compiled in ways that we regard as over-simplified and
arbitrary.

Our concerns about Maclean’s misuse of data in its rankings issue can be briefly recapitulated here.

To begin with, the Maclean’s rankings aggregate data from a range of variables related to the student body, class sizes, faculty, finances, library and reputation. It is inappropriate to aggregate information across a range of programs at a large and multi-dimensional research university into a single ranking number. Consider how such an approach might pervert one’s understanding of a general hospital that is ranked #1 in obstetrics and #10 in cancer care. Averaging these rankings would result in this hospital being ranked “#5 overall”. For the patient seeking care in one of these areas, such a measure would be useless at best and misleading at worst. This is, effectively, the method that Maclean’s applies to Canadian universities by its calculation of “league tables” based on the arbitrary assignment of weights to variables which, by themselves, are of questionable validity. The variables selected by Maclean’s also fail to capture the breadth of experiences students say are important in their university education such as, for example, extra-curricular activities or the opportunity for rich and diverse interactions
with peers and faculty outside the classroom.

We are also concerned by Maclean’s recent attempt to draw comparisons of student experience across incomparable surveys of student engagement, and Maclean's reliance on survey data with low response rates and all the associated response biases that arise from skewed profiles of respondents. The responsible compilation and comparison of data is a core tenet of academic research. Several universities already show student survey data, in context, on their own web sites and question Maclean's decision to pull different kinds of data out of context and compare “apples and oranges”.
Maclean’s treatment of these survey data, in our view, fails to give appropriate notice to these methodological limitations.

It is not just the Maclean’s student survey that has suffered from low response rates. Equally troubling is the fact that a clear majority of individuals who receive the Maclean's reputational survey do not respond. This is a particular concern as the results of the reputational survey not only affect rankings in a significant way, but are given prominence separately by your magazine.

This is only a partial accounting of the methodological flaws in the Maclean’s rankings. In short, the ranking methodology used by Maclean’s is oversimplified and arbitrary. We do find it ironic that universities are being asked to subsidize and legitimize this flawed methodology, when many faculty, staff and students at our institutions are dedicated in their research to ensuring that data are collected rigorously and analyzed meticulously.

We remain open to the possibility of collaborating with Maclean’s at some future date, particularly if we can agree on means to ensure that the data will be valid and the analyses truly informative. Meanwhile, we will continue to publish data on our websites to facilitate informed student and family choice.

Yours truly,

[konfusion666]

Doesn't sound like it's going to make any difference

I wonder if the Universities have the grounds to make this into a lawsuit?

[winner2000]

low? UBC, UofA, UofC, UofT, Mac, OttawaU and SFU are all top schools (and that's more than 50% of the list). If anything, what's interesting is most aren't in GTA. Most aren't even in Ontario.
I agree with them. I've always found the results are biased for Ontario schools, and for schools with top post-graduate programs (which are rarely indicative of good undergrad educations)

I don't know what constitutes a "top" school in your books, but many of these schools DID rank low on the Maclean's rankings. I found this comment in the discussion page of the article:

Francis Amelia from Victoria, Canada writes

....The Macleans rankings have three categories, Primarily Undergraduate, Comprehensive, and Medical Doctoral. Here are the objecting schools and their 2005 rankings: ~Primarily Undergraduate Ranking (21 listed schools) University of Lethbridge (13th; 12th in 2004) ~Comprehensive Ranking (11 listed schools) Simon Fraser University (tied for 3rd; 4th in 2004) ~Medical Doctoral Ranking (15 listed schools) University of Toronto (tied for 1st; 1st in 2004) University of British Columbia (4th; 4th in 2004) University of Alberta (6th; 6th in 2004) University of Montreal (7th; 7th in 2004) McMaster University (11th; tied for 8th in 2004) University of Ottawa (12th; 12th in 2004) Dalhousie University (13th, 13th in 2004) University of Calgary (14th; 14th in 2004) University of Manitoba (15th; 15th in 2004) Interesting that most of the complaintants here are in the Medical Doctoral Category. Not too surprising at all that most of these schools have done quite poorly in these rankings. Way to go U of T, though. For more info, http://www.macleans.ca/universities/article.jsp?content=20060613_163951_3996 But really, don't read macleans, it's kind of like the Fox News of Canada.

[Sylvestre]

I don't know what constitutes a "top" school in your books, but many of these schools DID rank low on the Maclean's rankings. I found this comment in the discussion page of the article:
.

Top as in Alberta has one of the best engineering physics programs (actually better than Queens, Mac or UBC. Also a very strong EE program.
MAC is excellent in the life sciences, and for grad studies, it's pretty much THE place to go for nuclear studies.
Ottawa is pretty much the only french speaking school in this country (other than UAQM/ect I suppose) and happens to have the tops language dept.
SFU has a top poli-sci program, with co-op, with a large number of contacts within the federal gov. (even more than Ottawa surprisingly)
Manitoba has pretty much the only farm/agricultural program in the province.

So really, they are top schools in their respective niches (or in general in some cases).
I agree with what the letter said. Would you pass over a hospital that's #1 in pediatrics but 5th in the overall list if you had a sick child?

Western's not all that high on the list but trust me, if you want to do medical imaging, the buck stops at Robarts.
Dalhousie isn't high on the list, but if you want to do acoustics (grad or undergrad) it's the school with the most active research, with University of Windsor a close second.

[36469]

i heard Queens dropped out too, but i guess not.
Queens has nothing to worry about.

[konfusion666]

But really, don't read macleans, it's kind of like the Fox News of Canada.

I wasn't aware as to how bad it had gotten... until I read one today for the first time in years while waiting in line at the haircutter's.

Holy crap! It makes the National Post look like a commie rag! :lol:

Disappointing that a magazine which used to be pretty neutral has taken on such an overtly neo-conservative perspective.

[ziptron]

I'm sure it was perhaps their rankings that made them act on it. Though I fully support the universities decision to opt out. There are far too many people out there who can't tell good data apart from bad data and will take McLeans to heart because they won't go research anything on their own.

Plus, rankings by a private magazine can be sketch, who knows how much "support" McLeans got from any of those universities. Most people won't think about that, so therefore, most people will listen to McLeans, go to those "top" universities and then those Universities will get more money and people won't know any different because they usually stay at one university for their undergrad.

After you're finished, it barely matters what university you came from. It's what you know, and who you know, not how you came about that knowledge.

So who cares about half of the stuff they "rank". Half the time lectures are usless, and half the people don't go. So then why are lecture/class sizes such a BIG issue?

I also agree with one of the previous posts, how in the hell can they judge Universities against eachother by data from students when most of them have only gone to only one university anyway?

[jd82]

I hope that the public at large finds out about these letters. I think too many students & parents are taking Maclean's rankings too literally and basing their decisions on institution in large part to where the school ranks in the magazine.

[DrLee]

Macleans gets their results by sending out surveys. How can you compare things about student life when most people have only gone to one or two universities. You can't say whether UofT or York or Mac or Waterloo is better if you haven't been to all of them.


Because they dont ask: Did you like your university's campus better than UofWaterloo (for example)?

They ask about parts of the campus they like and didnt like and then it's compared as a whole. People dont have to know which has better things in what area, they just have to know their own program/uni, Macleans puts it together... that's kind of the purpose of the whole thing.

Nah, you can't trust the universities. Seriously, are you telling me that an indepedent, most likely nearly unbiased, reviewing agency is a bad thing? Like, us univ. students should know better than anyone how much absolute crap spews from the mouths of our university officials, especially when enticing new students.

Macleans is a great resource for students and secondary students ranking universities, whether in certain fields or overall. I also congratulate Macleans for sticking it to the universities opting out by continuing. Perhaps their argument of over simplicity would be more credible if they werent trying to block the review or not participate by giving information, now they are guaranteeing a compounding of their issues this year.

I was a UofWaterloo student (finished b.a.), and now a UofToronto student and I'll still say it, my school is wrong on this.

[DrLee]

Top as in Alberta has one of the best engineering physics programs (actually better than Queens, Mac or UBC. Also a very strong EE program.
MAC is excellent in the life sciences, and for grad studies, it's pretty much THE place to go for nuclear studies.
Ottawa is pretty much the only french speaking school in this country (other than UAQM/ect I suppose) and happens to have the tops language dept.
SFU has a top poli-sci program, with co-op, with a large number of contacts within the federal gov. (even more than Ottawa surprisingly)
Manitoba has pretty much the only farm/agricultural program in the province.

So really, they are top schools in their respective niches (or in general in some cases).
I agree with what the letter said. Would you pass over a hospital that's #1 in pediatrics but 5th in the overall list if you had a sick child?

Western's not all that high on the list but trust me, if you want to do medical imaging, the buck stops at Robarts.
Dalhousie isn't high on the list, but if you want to do acoustics (grad or undergrad) it's the school with the most active research, with University of Windsor a close second.

Yup, also correct. People need to do some of their own research on some level because Macleans doesnt have categories for every field. It's good for picking a strong overall university, for example Western or Waterloo that have both done very well now and in the past. These are both large campuses that are strong UG schools in all areas...

[JimG]

Because they dont ask: Did you like your university's campus better than UofWaterloo (for example)?

They ask about parts of the campus they like and didnt like and then it's compared as a whole. People dont have to know which has better things in what area, they just have to know their own program/uni, Macleans puts it together... that's kind of the purpose of the whole thing.

Nah, you can't trust the universities. Seriously, are you telling me that an indepedent, most likely nearly unbiased, reviewing agency is a bad thing? Like, us univ. students should know better than anyone how much absolute crap spews from the mouths of our university officials, especially when enticing new students.

Macleans is a great resource for students and secondary students ranking universities, whether in certain fields or overall. I also congratulate Macleans for sticking it to the universities opting out by continuing. Perhaps their argument of over simplicity would be more credible if they werent trying to block the review or not participate by giving information, now they are guaranteeing a compounding of their issues this year.

I was a UofWaterloo student (finished b.a.), and now a UofToronto student and I'll still say it, my school is wrong on this.

Without a standard, people's opinions of whether they like their university are worthless. People read the rankings because they want to compare universities, you can't do that if the results aren't comparable to some standard or each other. There's other biasing factors as well, the demographics of say a school in the middle Manitoba are probably different from those at U of T campuses. The costs are different across universities as well as individuals at a university. Someone who pays more may expect more and vice versa. I certainly wouldn't call the results unbiased.

[anom]

I don't think individual schools do enough to promote their "best" programs.
Maclean's is just giving a general overview, and if potential university students are dumb enough to take it as gospel then we can easily see why the Canadian education is what it is today. And if they are so concerned with the rankings, they are universities, the place where research is done...why not take it into their own hands?

Top as in Alberta has one of the best engineering physics programs (actually better than Queens, Mac or UBC. Also a very strong EE program.
MAC is excellent in the life sciences, and for grad studies, it's pretty much THE place to go for nuclear studies.
Ottawa is pretty much the only french speaking school in this country (other than UAQM/ect I suppose) and happens to have the tops language dept.
SFU has a top poli-sci program, with co-op, with a large number of contacts within the federal gov. (even more than Ottawa surprisingly)
Manitoba has pretty much the only farm/agricultural program in the province.

So really, they are top schools in their respective niches (or in general in some cases).
I agree with what the letter said. Would you pass over a hospital that's #1 in pediatrics but 5th in the overall list if you had a sick child?

Western's not all that high on the list but trust me, if you want to do medical imaging, the buck stops at Robarts.
Dalhousie isn't high on the list, but if you want to do acoustics (grad or undergrad) it's the school with the most active research, with University of Windsor a close second.

[teknoluv]

Has anyone read Maclean's response to this in the Sep 4 issue?

[konfusion666]

Has anyone read Maclean's response to this in the Sep 4 issue?

was that posted online?

[JoKeRr]

i heard Queens dropped out too, but i guess not.
Queens has nothing to worry about.

Ya Queen's dropped out of Maclean's as well, http://qnc.queensu.ca/story_loader.php?id=44ec9c53c348a

[MizuRyuu]

i am happy that so many universities chose to opt out of the Maclean's survey.... for the past 2-3 years where i been reading their university review, i really do not find that it give a comprehensive review..... often, all the programs in a university is lumped together to calculate the ranking, which isn't actually that useful to prospective students..... ie: if i want to pick an university for their engineering program, i don't care that UofT rank high because their science program pulled their ranking high..... the fact that Maclean refuse to have separate catagories for the different faculty make the ranking fairly useless....

[SSP]

I don't think individual schools do enough to promote their "best" programs.
Maclean's is just giving a general overview, and if potential university students are dumb enough to take it as gospel then we can easily see why the Canadian education is what it is today. And if they are so concerned with the rankings, they are universities, the place where research is done...why not take it into their own hands?

That's certainly not how they advertise the issue. By their wording, any parent/new students would assume its THE guide to picking a good university. Like someone said, each University has its strong and weak points and I rarely see mcleans showcasing any of that info, IMHO.

[Mgz]

saw this in the paper today! I'm with the universities. Down with Macleans rankings!

me too, U of A 4tw.. Viva la resistance!

we have THE best Chemical Engineering program.. Everyone in Alberta knows that U of A, UofC and U of L are good univerisity, but the ranking is pathetic :( . Lot of international student from my country rely on the MacLean survey to pick their school therefore not many student want to come here :|

also it costs the University money and resourse to do the survey for MacLean :/

[DaVibe]

You can add Ryerson to that list. The president sent out an E-mail just recently addressing the issue and why.

Sounds legitimate about their grading system and how Ryerson staff wanted them to look at other topics and gradings but Maclean's will continue to judge the way they do.

Schmeh.