I have been watching these rankings come out since the early 1990s and the response is always the same. If a city, province, university, whatever ranks well – it gets wide citation and makes numerous Powerpoint slidedecks. if not, the methodology is studied with a fine tooth come to bring some cathartic relief.
Such is the case with the Moneysense top cities to live list out yesterday and with the advent of Twitter and Facebook we can get real time angst and jubilation.
Look, folks, sometimes a kiss is just a kiss, a smile is just a smile – the fundamental things apply as time goes by.
These lists always have methodological problems and too much complaining just looks a little too defensive. I remember with a number of universities boycotting the Maclean’s rankings a few years ago – who got the worse of that in the media – Macleans or the universities?
Way back in the 1990s I had a look at the methodology used to calculate the KPMG competitive alternatives report and it wasn’t pretty. It’s kind of like sausage – good to eat but you don’t want to think too much about how they were made.
Moncton has probably been one of the most aggrieved in at least two of these studies. In the crime rankings, MacLean’s didn’t even get the city name right talking about ‘Codiac’ as if it was a place (it’s the name of the RCMP coverage area). What simpleton researcher at MacLeans doesn’t even know that Codiac is not a city in Canada? Maybe MacLeans outsourced to Red Deer or Mumbai.
Then there was the Learning Index which comes out each year and weeks after Moncton made the Top 7 Most intelligent cities list, the CLI reported that Moncton was among the most stupid cities in Canada.
Which is it? Stupid or smart? Depends on the methodology – I preferred (ahem) the Intelligent Cities methodology because it was based primarily on doing smart things while the CLI was more interested in education levels in the population and access to museums.
Bob Manning from Saint John got it just about right last year when he said we welcome all of these studies and we look at them to find out ways we can improve our city. I think that’s the right mindset around these studies because if we reject out of hand all studies that don’t show us in a good light we become resistent to any criticism and therefore lose our chance to think about improvement.
I have encouraged communities to benchmark themselves against peer communities across Canada – that way we get an apples to apples comparison. It’s difficult to compare Moncton to Toronto or Campbellton to Calgary. But Monton to Kingston, Regina, Kelowna, etc. makes better sense. Then you can truly track your progress (decline) against peers – which is an important intellectual discipline.
> that way we get an apples to apples comparison
Yes, but it depends very much on just what data is collected. As we see with the Learning Index, very different things can be used to differentiate between ‘intelligent’ and ‘stupid’. It’s important to identify the *right* things.
Data is not automatically useful, and benchmarking is not automatically informative. You can draw some very incorrect conclusions and actually point your city in the wrong direction through a misuse of benchmarking.
Take, for example, ‘new cars’, which in the MoneySense report is “2007-2009 model year vehicles as a percent of total vehicles.” Does that reflect prosperity? Or does it reflect terrible road conditions? Or does it reflect a dysfunctional mass transit system? Attempting to increase the ‘new cars’ rating could be helpful to a city, or harmful.
My own view is that these benchmark comparisons often do more harm than good. Moncton is not Calgary or Vancouver, and we shouldn’t try to be. What works well, and reflects success, in those cities may be something very different from what works in Moncton. What we value here – bilingualism, say – might be thought of as unnecessary or worse in the other centres.
Evaluations of cities should be conducted according to their own criteria. What goals has the city set for itself? Is it approaching those goals? Vancouver decided it did not want freeways – it would rank very low compared to Edmonton, which has energetically constructed freeways. But the choice represents differing values, not the success of one and the failure of the other.
I find these studies are like the hecklers in the front row during a hockey game. Those players who are not focused and determined hear them (and take them to heart/or are offended). Those who don’t, score and win. Plain and simple.
I find these studies are like the hecklers in the front row during a hockey game. Those players who are not focused and determined hear them (and take them to heart/or are offended). Those who don’t, score and win. Plain and simple.