I read with interest Bruce Temkin’s article proposing to retire the NPS and replace it with a new True Loyalty Measure (TLM) metric. The article also begins with an invitation to be involved in a discussion, so here I am.

To Bruce’s first question, what kind of NPS segment do I see myself in toward the NPS? I’ll answer honestly: I fall into the passive category. I recognize that the NPS has strength in orienting the organization’s culture toward customer-centricity, which is no small thing. It is a wrong choice from a purely statistical and mathematical point of view. Perhaps the worst choice you could make, and I have demonstrated this mathematically in several articles.

To imitate Bruce’s article, here I report my views over the years on the NPS:

The first time I used the Net Promoter Score in a project was in 2006. I always treated NPS as a Customer Feedback Management methodology and gave less importance to the value of measurement. Over time, mainly because of the big volatility problem, I had to focus a lot on the statistical validity of the metric, and that’s where the problems started. I won’t list them here because that’s not the article’s purpose, but it prompts me to consider myself a Net Promoter Score passive. I do not have an entirely negative view, but I am not enthusiastic about it either.

I agree with Bruce that “…measurement is much, much less important than the system you create around…” however, it must honestly be said that the system is a perfect copy of Customer Feedback management methodologies. I wrote a paper on this argument in 2008 that separates method from measurement and is agnostic to NPS. You can read it here.

I do, however, smile at Bruce’s proposal to retire the NPS and replace it with a new single KPI that uniquely measures the customer experience. Let me elaborate on the reason for my smile:

  1. The NPS is still the result of academic research where the centrality of the syllogism is inescapable. To propose a methodology without severe academic research is at least risky. I searched for the existence of a paper along these lines but could not find it.
  2. I reread the proposed TLM question — and read it again. To me, it is not clear: “If you could choose, how likely would you be to do business with <<ORGANIZATION>>?” At first, I wondered why I should not choose; I thought I was living in a free world. Then I am asked with what probability would you do business – perhaps I am stupid, but the question is not at all apparent and straightforward. It is called simple, but it is not.
  3. Four possible answers are offered, ranging from “Very Likely” to “Unlikely.” It would be interesting to understand the rationality behind this choice. It is a Likert, and in particular, a “forced choice” because, since the scale of values is even, it lacks the middle point- the typical “Neither agree nor disagree” option. We force the choice of respondents toward the extremes, but then in the total calculation, in addition to eliminating the center point, we eliminate the two low box categories. From a statistical point of view, without a mathematical explanation, it remains somewhat vague.
  4. So we segment the respondents into four categories, eliminate two types (2 low boxes), then take the respondents’ percentage in the first top box and subtract half of the respondents in the box sounder. The result is calculated this way: the percentage of “Very likely” + (the percentage “Somewhat likely”)/2. It would be interesting to understand why we apply a weight of 0.5 to the second full box. What are the rationales behind it?
  5. I am not sure, as the author claims, that we are dealing with a behavioral question. Yes, the question is direct, but in my opinion, it still measures attitude toward whether you want to do business, not ascertained behavior.

However, it is to Bruce’s credit to admit candidly that “I haven’t tested or implemented this metric anywhere yet, so it’s a good time for some joint development.” so we are talking about pure theory, certainly not supported by research. So we are invited to participate in an empirical experiment. I am a little surprised. We’re still talking about a person and a firm that has access to large amounts of historical customer experience measurement data from firms around the world. That reality of data is available to you, it seems to me somewhat reductive to suggest an empirical approach of a theoretical metric having large amounts of data available to then be able to present something robust. Can’t you do something better?

I also smile at the last part of the article, where you try not to antagonize those who have made a big business out of the NPS, and I’m talking about Bain & Co. Let’s say it out of the teeth: if you criticize a metric and preach about replacing it with something new and better, then you can’t say, “yeah, but they at Bain & Co. know how to use it and so they can do it.” The only thing I understand, in that precise message, is the desire not to open controversy and to wash out appearances.

I am not convinced that this metric can replace the NPS. I am convinced, and I have also written it, that in times of Customer Journey Phigital, the NPS is quite obsolete. The measurement framework I have proposed, especially for digitization, is based on more straightforward premises. However, there is no single value for measuring Customer Experience. One has to work on composite and interrelated metrics. Unfortunately, there is not one single metric that includes everything. The reality is much more complicated — and interesting.

Is True Loyalty Measure a valid NPS replacement?
Author:
Federico Cesconi

Read the article on LinkedIn

Start growing with sandsiv+ today
Request a demo