The Boston Globe checks in with Forbes to find out how the magazine’s annual rankings could penalize Northeastern University for having a low four-year graduation rate given that it is a five-year school. And a magazine editor actually defends the methodology, saying, well, gee, we’ve got to base our rankings on something.
We also learn that 27.5 percent of the rankings are based on student satisfaction, “determined primarily by feedback on RateMyProfessors.com.” Wow. Without getting into the pros and cons of RateMyProfessors, let me just observe that the sample size for most instructors is so low that the data are entirely anecdotal.
Discover more from Media Nation
Subscribe to get the latest posts sent to your email.
“Sample size.” 🙂
@Michael: Hmmm … do many unreliable surveys become statistically significant when they’re all mashed together? Maybe they do. I know that’s how a lot of meta-studies are conducted.
And for what it’s worth, I don’t think RateMyProfessors is garbage. Students can probably learn a lot by reading the comments, no matter how few they are. You can tell whether someone has made a sincere attempt to offer an honest assessment or is just whining. If I were a student, I’d definitely look at it. But trying to extract data from those few comments seems futile.
Your points are well-taken, but one nit-pick: if Forbes is using data from RateMyProfessors.com as part of their analysis, what matters is the sample size for the entire university, not the sample size for an individual professor.
The university-wide sample size might still be lousy — not only statistically small, but systematically-biased by being dominated by angry D students or suck-up shillers…
Forbes tries to justify their stupidity.
Wow. Forbes just out-did the credit rating agencies. That took some doing.
As this country continues to accumulate 2.6 million of debt each minute, 160 million an hour and 3.8 billion a day, I think the downgrade of this nation’s credit rating is spot on.
Hopefully our meatheaded, self-serving, partisan politicians in Washington will finally get their act together and make a concerted effort to solve the seemingly endless list of problems now threatening this country.
Oops. I now realize that Mr. Read is referencing the “other” credit rating agencies.
I still think a majority of this country’s politicians are meatheads.
Dan, when I was in graduate school at a major university, my appointment was as an administrative intern in the institutional research office. For 3 years I responded to these idiotic surveys. That was the main point of the job. There was so much work, unlike 99% of other grad students, I had a year round job, not just when school was in session.
Forbes college rating is fairly new, when compared to US News, Princeton, Newsweek, Barron’s, and others most people have never heard of.
Two things tell me that Forbes operation is amateur hour: they reveled that they use ratemyprofssor.com and the weight of its data. NO survey reveals their data points exactly, and NEVER makes the weight public. Why? Because, schools are already trying to game the surveys, if you make the data points public, then some schools are going to try and influence the data. I would eat my hat if some small college or university, trying to make a name for themselves, isn’t on ratemyprofessor.com pumping up the numbers. A lot worse happens.
The people that did the Forbes work might not even work for Forbes. More and more publications contract out this work to the lowest bidder. Big players like US News would never farm out the work, because it’s part of their brand and the money is too good, but the marginal players look at the college ratings game as a quick buck.
@Bill: Very interesting. By the way, as far as I can tell, the college ratings are the only thing that’s left of U.S. News. There’s no longer a print magazine, and its website is a well-kept secret. But its ratings are taking very seriously, not least at Northeastern.
Excellent editorial in Tuesday’s Globe.