Marketers should be suspicious of anyone claiming to solve a complex problem with a simple solution, write Dr Dave Stewart and Prof Steve Worthington, and the ever-popular net promoter score sinks of simplicity.

Dear CMOs,

As chief marketing officer (CMO), you should know, perhaps better than your C-suite colleagues, to be suspicious of any consultant claiming to have solved a complex problem with a simple solution.

Simple solutions are naturally attractive because they are easily applied to what in reality are often very complex and intractable management situations. But what is often overlooked is that a simple solution to a complex problem is usually the result of a number of simplifying assumptions which are hidden from view. Consequently, any manager who relies on their intuition and the herd instinct when assessing a consultant’s simple solution to a complex problem is unlikely to be aware of its underlying assumptions, let alone question their veracity.

However once a new management approach is in place, it can be difficult to dislodge, no matter how perverse the consequences. By the time any disparity between a consultant’s claims and reality becomes fully apparent, management has usually invested significant time and resources embedding the approach in the organisational culture. Moreover, reputations of senior managers are often bound up with its success, making its removal problematic.

So it is with the net promoter score (NPS), introduced to the business world by Bain and Company’s, Fred Reichheld in a 2003 article in the Harvard Business Review, which was soon followed by his 2006 book, The Ultimate Question. Like a modern day Don Quixote, Reichheld is a crusader on a quest to cure CEOs of their addiction to “bad profits”, which he describes as “profits earned at the expense of customer relationships”. He warns that bad profits create detractors that “strangle a company’s growth”, but by delighting customers we turn them into promoters – “not only increasing their own purchases but also providing enthusiastic referrals” generating “good profits” and growth. But Reichheld’s quest does not end there. The real treasure is what he describes as the “Missing link between the Golden Rule, loyalty and true growth.” Thankfully, after a “long and arduous quest” our hero discovers the missing link, christening it the net promoter score.

This rhetorical style pervades his writing; one could be cynical and view it merely as a device for disguising the shallowness in his claims, none of which are supported by any credible evidence.

So what then are Reichheld’s claims?

Claim #1: There are only three types of advocacy behaviours exhibited by customers – that of a promoter, passive and detractor.

Reichheld intones that this is a “fundamental perspective”, but whose perspective this is can only be guessed at. If this is so fundamental, what happens to the customer who gives a ‘mixed review’? They are not the ecstatic promoter, the whingeing detractor “trapped in a bad relationship”, nor the mute passive. There may be many such customers. Perhaps you yourself are one of them?

Claim #2: Customers reveal their behaviour when they respond to one question on a zero to 10 scale. To the question, “How likely is it that you would recommend this company to a friend or colleague?” Promoters give a nine to 10 rating, passives a rating of seven to eight, and a response below seven unmasks a detractor.

Reichheld does not provide any evidence in support of this claim. Instead he pronounces that customer responses “link so closely to their behaviors that it is a practical surrogate for what they will do”. But if this were true, why then would he choose to withhold the supporting evidence?

Fortunately, a 2007 article in an academic journal, Managing Service Quality, does provide objective evidence for us to examine in part the veracity of this claim. The article gives the results of a market research study of over 8000 US customers across retail banking, mass-merchant retailing and internet service providers (ISPs). Of particular interest are customer responses to the recommend question and whether they actually recommended the organisation over the following year.

The table below shows the results for these three industries.

For example, in retail banking over a half of ‘promoters’ do not behave like promoters, and over a quarter of ‘passives’ behave like promoters! On this basis alone, the evidence is clear: Reichheld’s “fundamental perspective” is fundamentally flawed.

While this study does not explore negative comments being made by an organisation’s customers, there is no reason to expect that the results for ‘detractors’ would be any better. Why? Because any customer who does not intend to recommend an organisation could just as well be a ‘passive’.

The next two claims concern NPS which Reichheld defines as the difference between the percentages of promoters and detractors.

Claim #3: Increasing NPS drives organisational growth.

Reichheld’s ‘evidence’ in support of this claim is given in appendix A of his book, comprising charts of NPS by growth for selected organisations across half a dozen industries. While the charts show positive relationships between growth and NPS, they do not demonstrate that NPS drives growth. In one chart, for example, it shows growth over 1999-2002 versus NPS for 2001-2002. The difference in the two time periods is important. Essentially this suggests that growth drives NPS, which is the opposite of Reichheld’s claim, making his argument entirely specious. This point was documented in the academic literature in 2007 (Journal of Marketing).

Implicit claims #4: The value of a promoter to the organisation is equal and opposite to the value of a detractor, and a passive customer has zero value to the organisation.

We have discussed this at length in a recent article on the problems with the net promoter score.

READ: The problems with net promoter scores: how to better measure customer advocacy »

While Reichheld does not explicitly make these claims in his book, nonetheless they are implicit in the NPS calculation. We don’t intend to repeat the full text of the earlier article here. Suffice to say that there is no evidence to support these claims. Quite the contrary, as a worked example in his book contradicts both claims.

NPS – a case of collective delusion

Many in the corporate world like NPS because it is so simple. Everyone ‘gets it’, from the board all the way down to frontline staff. But it is the simplicity of NPS that hides a dark side which we have sought to reveal.

Despite the fact that we have exposed NPS to be a failed metric based on unsupported claims and false assumptions, we do not expect business leaders to suddenly ‘see the light’. As explained at the outset, there is too much at stake here to suddenly reverse earlier decisions. But we do think that after more than a decade since its release, it is time for business leaders to fully investigate whether NPS is providing the strategic direction they expected. We believe that you can play an important role in bringing this to the attention of your C-suite colleagues and in so doing, begin the search for a more credible alternative to NPS. For example, perhaps instead of looking in the rear view mirror at how our customers have perceived us, we should instead look ahead through the windscreen at how far our customers think we can take them with us as an organisation. What is our ‘growth’ score? How many customers are prepared to accompany us as an organisation on a ‘growth’ trajectory?

So to conclude, in a recent documentary, famed magician and sceptic, James Randi remarks: “No matter how smart or how well-educated you are, you can be deceived.” Almost paradoxically, with NPS the deception is hidden in its simplicity.

Yours sincerely,

Dr Dave Stewart, Marketing Decision Analysis, Melbourne

Professor Steve Worthington, Swinburne University, Melbourne