Customer Effort Score and the Link (or Not) to Customer Loyalty

By Dr. Fred Van Bennekom, GreatBrook Consulting

[one_half]As someone who had read the 2010 Harvard Business Review article, “Stop Delighting Your Customers,” I was intrigued that the authors now had a book, The Effortless Experience: Conquering the New Battleground for Customer Loyalty. I was looking forward to reading it to find out what new things they have found, as well as to find out more about the research methodology behind their findings.

The research methodology description in the Harvard Business Review (HBR) article was so sparse that I simply could not accept their findings even though others, including companies paying good money to the Corporate Executive Board (CEB), accepted the findings with little critical thought to their validity. Alas, the research methodology description in the book is about as sparse as in the HBR article. They devote less than a page to describing “Our Methodology In Brief,” which is mostly geared to making us feel impressed that they got 97,000 responses. Having read the book and read “between the lines” to infer the research process, I am yet more skeptical about the findings.

In an ironic twist, the authors in writing this book violate the very prescriptions they give their readers. They try to “Wow!” their readers will all kinds of “delightful” statistics, yet for the intelligent reader trying to
understand the research behind the findings, this is truly a “high effort experience.” There is nothing seamless about the transition from research description to statistical analysis to findings and prescriptions.

Lessons From the Book

First, the idea that you should reduce customers’ effort in doing business with you is simply a common sense no-brainer. (Whether it’s a predictor of loyalty… Well, that’s a different story. Keep reading.) A customer won’t be satisfied, let alone loyal, if a company creates high customer effort. We’ve all been there.

The message basically is to “pick the low hanging fruit.” If you’ve had any exposure to quality initiatives, you’re probably familiar with that metaphor. If you were going to pick apples, which ones would you pick first? Logically, you’d pick the best apples that were within arm’s reach — the low hanging fruit. In a quality improvement initiative, the same logic holds. Go for the easy wins with a good payback and little risk. Reducing customer effort is a juicy apple in easy reach.

Second, their research indicates that customers do more “channel switching” than most companies believe. Most customers will attempt a resolution of their issue first by the web and then call if necessary. In fact, while on the phone, the customer may well be on the website as well. I do that. Effortless proposes that companies are mistaken to segment customers into chat, web, or phone customers. In fact, most use more than one, and the movement to web first is happening faster than companies believe.

Third, Effortless also argues that this switching corrupts First Contact Resolution (FCR) statistics. If a customer starts on the web and then switches to a phone resolution, a check mark should not be placed in the FCR column. True, and this shows even more the flawed data model in customer management systems that don’t recognize multiple contact points under one incident number.

Fourth, Effortless suggests that contact center agents should guide the customer through the resolution process to leave them more knowledgeable about how to resolve issues on their own in the future. This makes  eminent sense, especially if the customer is contemporaneously on the web. A Fidelity agent did this with me recently. This is part of “next issue avoidance,” which is an area near and dear to my heart as a long-time researcher on the impact of product design upon customer experience.

Indeed, the book has many positive points, but let’s turn to the big issue of the research methodology and analytics to support the customer effort conclusions. Specifically:

  • Have they proven that customer effort drives or predicts customer loyalty?
  • Should you be using the Customer Effort Score (CES) in your transactional surveys (in place of the Net Promoter Score®) to identify customers who will or will not be loyal?
  • Have they truly identified a new metric to which we should pay homage as we have done with NPS?

I can certainly see using CES on a transactional survey to identify an at-risk customer, but claims for CES as a predictor of customer loyalty are simply unfounded. (In fact, they even say that!) Those companies that use CES as a loyalty predictor may be mislead.

Customer Effort as a Loyalty Predictor

Why do I say CES is not a proven loyalty predictor? There are several reasons:

  1. The research model is weak. Measures of actual loyalty are not part of the model. If you don’t measure actual loyal behaviors, you cannot have a good predictor of loyalty.
  2. The research design, especially the questionnaire design, and execution are flawed. Even the authors agree that they lack proficiency in writing valid survey questions! But more issues exist.
  3. The application of statistics is flawed. The authors simply do not provide enough information about statistical processes or the correct interpretation of the statistics they produce. If the statistical  interpretations in this book were the final exam in a college statistics class, the student would get a B-. Perhaps.

For those who don’t wish to read further and apply the CES “just because,” Caveat Emptor. But for the inquisitive, let’s look behind the curtain to better understand the numbers behind CES and its link (or lack of it) to customer loyalty.

The Research Model

The research design behind the book’s findings has a few shortcomings. First and foremost, for a book about predicting loyalty, they never actually measure loyalty.

[/one_half]
[one_half_last]

If you read the book, The Effortless Experience, you will constantly read the authors’ claim for the Customer Effort Score as a “predictor of loyalty.” Occasionally, the authors will use the phrasing “intended future loyalty.”

For example, on Page 17, they write, “In our global survey, we found virtually no statistical relationship between how customer rates a company on a satisfaction survey and their future customer loyalty.” And on Page 157, “And that’s where the Customer Effort Score really shines — It helps us understand the actual impact of the service experience (and only the service experience) on customer loyalty.”

A world of difference lies between the intended and actual loyalty behaviors. When the authors omit “intended future” in the book and in blog posts, they are misleading and overstating the value of their research findings.

Here’s the research design, as I have inferred it. They posed a survey to a great number of people. This one survey had questions that (attempted to) measure the type and level of effort customers experienced in a  service transaction, among other things. It also had questions at the end of the survey that measured what surveyors call “attitudinal outcomes.” Those are the summary questions — e.g., overall satisfaction, likelihood to recommend, likelihood to buy again in the future, and likelihood to buy other products from the company. These summary questions they say describe customer loyalty. Higher scores means higher
loyalty; lower scores mean the inverse.

Their statistical analysis compared the scores from the CES question to the scores given for the attitudinal outcome questions on the same survey. However, they never compare scores from service transactions today to actual future customer loyalty behaviors.

How well do the intended loyalty measures predict actual loyalty behavior? We don’t know. And they never raise that distinction as a critical assumption to their findings. The assumption here is that what people  say matches what people do in the future. Professional researchers make clear their assumptions that affect interpretation and application of the findings. When you read their book, every time they claim predictive abilities for customer loyalty, remember the text should read “intended future customer loyalty.”

Compare this research model to the research behind Net Promoter Score (NPS). Reichheld et al. compared scores on the recommendation question to actual future industry profitability. Shortcomings exist in that model too, but its longitudinal nature is far, far superior to the point-in-time research done in Effortless.

Service and Only Service in the Model

Notice in that above quotation from page 157 and the one below from page 12, the authors make explicit that the model also treats only data about service interactions.

“We intentionally limited the study to service center transactions and their impact on customer loyalty. Obviously, customer loyalty is a product of all the interactions a company has with the customer — its brand reputation, their friends’ and family’s perceptions of the company, the value and the quality of the products, and of course customer service, among other things.“

This approach games the result. If you only ask questions about service interactions, that’s what’s going to be shown to be key predictors of the customer loyalty outcome in the regression models (I believe) they ran.  If they had included those other causal factors, the statistical significance and level of impact of the service causal factors may have plummeted.

Does Effort Drive Loyalty or Disloyalty?

Okay, let’s accept the research model as valid for the moment. The book is still not about its subtitle: Conquering The New Battleground For Customer Loyalty. More accurately, the book is about Conquering The New Battleground For Minimizing Customer Disloyalty. As the authors write on page 130, “The argument of this book… is to mitigate disloyalty by reducing customer effort.”

If the prescriptions in the book are accurate and you implement them, you will reduce customer disloyalty, not generate loyalty. In a blog post about the research, the practice manager writes:

“We recommend focusing on moving customers to at least a `5’ on the 7-point CES 2.0 scale (or `somewhat agree’). Moving a customer from a `1’ to a `5’ boosts their loyalty by 22%, but there are diminishing  returns after that — moving a customer from a `5’ to a `7’ only boosts their loyalty by 2%.”

So, reducing effort moves a customer from a disloyal to a neutral position, but reducing the effort further leads to no loyalty improvement. When you’ve reduced customer effort, have you increased customer loyalty? Not if a competitor matches your effort reduction — and then works on the drivers of positive loyalty. The lesson here — never mentioned in the book — is: Once you’ve reduced effort, then delight your customers. That is exactly the argument behind the Kano Model, which has been around for decades. The authors claim to have broken new ground in theory development. They have not.

Or to put it in my own phrasing, “separate the bow-wow from the wow.” First, eliminate the bow-wow moments, then focus on the wow moments. (If you use that quote, you’d better give me attribution.)

In another article, we’ll take a look at other issues in this study and book, including survey questionnaire design, survey execution, and application of statistics for study conclusions. Stay tuned…

Dr. Fred Van Bennekom is the Founder and Principal of GreatBrook Consulting, an organization specializing in customer surveying. Fred is a popular speaker at industry conferences and is the author of Customer Surveying. He can be reached at (978) 779-6312 or fvb@greatbrook.com.

[/one_half_last]