Three changes to your annual customer satisfaction survey can better focus your improvement initiatives
Many organizations conduct comprehensive customer satisfaction surveys this time of year in preparation for their annual planning exercises. Surveys aid decision making for process improvement initiatives and can also be used to test new product and service ideas. But most companies miss easy opportunities to gather valuable information along the way because they fail to do three things:
1. Ask segment identification questions.
Surveys typically ask customers for demographic information (age, gender, income, location) or company-specific information (products or services purchased, account size, salesperson) to categorize and compare responses. But these questions tend to be driven by whatever is easy to measure, not by what’s most essential to know. Asking respondents to identify themselves by market segments allows companies to align their products, services, and process improvements with the benefit of greater context.
For example, let’s say a travel company identified three distinct segments in their customer base: families vacationing on school holidays, seniors enjoying their retirements, and corporate group travelers celebrating successful sales years. While all may purchase travel services, each customer set values what the company does differently: holiday availability may be more important to families, while low travel cost may be more important to retirees. The company may also determine that some customer segments are more profitable than others or represent opportunities for growth. In this example, the company could ask, “Which statement best describes you?” and offer descriptions of the three segments.
When data are analyzed according to cleanly defined segments, suddenly results have far greater strategic impact. Prioritization becomes much easier when improvements can be traced to potential top-line revenue or bottom-line profit by customer segment.
2. Ask what’s important.
Many organizations use the same survey questions they’ve always used, which allows them to track customer satisfaction changes over time. This certainly makes sense, but doing so blindly eliminates the chance to verify what’s relevant. The Importance-Satisfaction (I-S) survey technique adds the question “How important is this to you?” along with a rating scale (e.g. 1= not at all important to 5 = extremely important) to track customers’ current and changing tastes. Collecting both importance and satisfaction provides a handy measurement of the gap between the two.
High-performing organizations periodically challenge their understanding of customer preferences. After receiving the prestigious Malcolm Baldrige National Quality Award a second time in 1999, Ritz-Carlton CEO Horst Schulze remarked:
“During our implementation of the Baldrige Criteria, I’ve learned that although the sparkle of the chandeliers is important, it is not the only priority of our guests. It doesn’t matter that the petunias are perfect if the valet dents your car or your bill is wrong. We’ve learned to make our customers’ priorities our priorities… We’ve shifted from looking ‘out’ from our own perspective to looking ‘in’ from the customer’s.” 1
Besides asking customers the right questions to determine areas of most concern, eliminating questions deemed unimportant allows organizations to make surveys shorter, which in turn increases survey response rates.
3. Analyze correctly.
Most companies use very basic calculations which can lead to the wrong conclusions. Managers typically compare average satisfaction scores from one period to the next or between one group and others. For example, if average satisfaction was 4.3 in 2012 and is 4.4 in 2013, managers often conclude satisfaction has improved 0.1. This may not be true—the determination has not accounted for experimental error, and the difference may actually be due to randomness. Student’s t or Analysis of Variance (ANOVA) is the correct approach when comparing sample means between two or more groups. Technically, if data are not normally distributed (most satisfaction data are skewed, not bell-curve shaped) or respondents select from a set of discrete ratings (e.g. categories 1-5 on a Likert Scale or 1-10 for Net Promoter Scores®), contingency tables with chi-square analysis then becomes the correct approach. To properly separate the “signal” from the “noise,” the analyst must calculate p-values to determine if differences between sample means are statistically significant.
Most executives want to understand cause-and-effect relationships and make predictions, not just make comparisons. For example, call center managers may want to know, “Which will have a greater impact on overall customer satisfaction: reducing call hold times or resolving issues on the first call?” In this case, analysts use multi-way cross-tabulation tables, but when evaluating four or more factors, the analyses are more elaborate (correspondence, classification trees, log-linear). The right methods can uncover a wide range of new insights, but managers should ask qualified statisticians for help.
Collecting customer feedback is a healthy first step for annual business planning. Adding segment identification, importance-satisfaction questions, and proper statistical analysis can then make a substantial impact on what is decided during business planning.
Source:
- James Hunt, Elaine Landry, and Jay Rao, 2000. “Case Study: The Ritz-Carlton Hotel Company, 1992 and 1999 Malcolm Baldrige Quality Award Winner,” Journal of Innovative Management, Fall 2000. GOAL/QPC.