It’s happened before – a top brands list with results that are difficult to fathom. This time we have research conducted by Nielsen for the 2011 issue of Superbrands placing Maggi, Dulux and Mortein ahead of Apple, and with a top ten heavily favouring FMCG names.

For a list that claims to be “Australia’s most popular brands” the results are surprising. Conspicuous in their absence are Apple, McDonald’s and Coca-Cola, who consistently rank at the top of most popular and most valuable brand lists, both in Australia and globally. And unless Maggi flew under my radar this year, I doubt it’s one of Australia’s top favourite consumer brands. I certainly don’t recall seeing people queued around the block for the latest two-minute noodle product release.

According to the study’s press release: “Conducted between May and August 2011, Nielsen’s intensive research covered more than 90 consumer products and services categories, identifying around 500 brands that passed Superbrands’ exhaustive qualification tests.”

However, when asked about the methodology, Nielsen revealed that each participant was asked the same two open ended (unprompted) questions 50 times – 1) what is your favourite brand and 2) what is your second favourite brand – cycling through the different product categories. The responses were then added together and ranked to form a list of the top ‘Superbrands’.

Here is the list of the nation’s ten “favourite brands” (no particular order was provided) that this methodology generated:

These types of top brand surveys often use generalisations such as ‘most popular’ or ‘best brand’, which is problematic. The semantics around how the top brand list is described needs to match the methodology, otherwise you get a list which masquerades as something it’s not. Such is the case with this list, which is not derived from “intensive research” or “exhaustive qualification tests”.

The use of unprompted brand recall questions is one of the causes of these results. With an open ended question (as opposed to a prompted list), the names that rise to the top are polluted by brands that the consumer has seen or thought about most recently, giving you a list of brands that are top of mind. Nuances in the way the questions are asked, how the sample is selected or the weighting of the data may also have impacted on the results.

Rival top brand lists, Interbrand and Millward Brown’s BrandZ, use a mixture of market data and consumer survey attributes to put together their brand lists. The results from their lists this year vary slightly but contain roughly the same names at the top.

While these are global lists designed to measure brand more holistically, most of the names appear because they’re consumer favourites.

Commenting on the Superbrands process, Peter Richardson from Superbrands Australia, said: “Superbrands pays tribute to the strongest and most valuable brands in Australia today. Sheer size doesn’t cut it… to be a Superbrand requires the consistent management of the company’s values, beliefs and product quality.”

Thankfully, additional grading criteria including market dominance, longevity, goodwill, customer loyalty, and overall market acceptance are used to put the Superbrand publication together.

The best way to describe the findings from this PR exercise is somewhere between ‘best brand that comes to mind’ and ‘favourite of the household’s main grocery buyer’. The best way to take them is with a grain of salt, and not as an indication of the quality of research being produced by our local agencies.