Beware Of The Secret Shopper Syndrome

Lots of firms — from banks to retailers — use “secret” or “mystery” shoppers to evaluate if individual stores/branches are delivering good (or the right) service. With this type of an approach, consumers are sent to different locations to complete a specific task (or tasks). Then they are asked to fill out a survey based on their experience. The shoppers usually receive some small payment for their efforts.

This is a great way to get a meaningful sample with structured data about customer experiences.  But unfortunately, sometimes this type of an approach can go bad — as it did for my wife during a recent trip to the post office to mail a package.

United States Postal ServiceWhen my wife got to the front of the line, the mailman behind the counter told her (this is accurate, but a bit paraphrased):

I’m can’t take credit cards; our machine has been broken for 3 hours…. My manager wouldn’t let me hang a sign out front where people would see it. We can’t mess with the decor, because a mystery shopper may come by and we’d lose points.

There’s something definitely broken here. In this branch of the post office, they are optimizing around the experience of a mystery shopper, not the actual customer.

Unfortunately, this problem is not confined to the USPS. And it’s indicative of a bigger problem: misaligned metrics. You can see this problem at a car dealerships whenever a salesperson tells a new car owner “you’ll be getting a survey and make sure to give me a perfect score” or when a salesman makes sure that only “friendly clients” get to fill out satisfaction surveys.

Whenever your metrics aren’t quite aligned right, it puts people in a position where they can — and often will — game the system. Who loses? Customers. Because instead of focusing on the customer and his/her experience, employees focus on optimizing some arbitrary metric — like consistent decor.

Don’t get me wrong, I think that data from sources like mystery shoppers can be very valuable. It’s just a matter of how the overall program (including employee incentives) is structured.

The bottom line: Focus on the needs of real, not mysterious, customers.

Written by 

I'm an experience (XM) management catalyst; helping organizations improve results by engaging the hearts and minds of their employees, customers, and partners. I enjoy researching and speaking about leading-edge XM topics. I lead the Qualtrics XM Institute, which is the world's best job. We're igniting a global community of XM Professionals who are inspired and empowered to radically improve the human experience. To achieve this goal, my team focuses on thought leadership, training, and community building. My work is driven by a set of fundamental beliefs: 1) Everything starts and ends with human beings, so you need to understand how people think, feel, and behave; 2) XM is a discipline that needs to be woven throughout an organization's entire operating fabric; and 3) Building the XM discipline requires a combination of culture, competency, and technology.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.