Measuring Customer Satisfaction and Loyalty: Survey Design, Use, and Statistical Analysis Methods

By Bob E. Hayes

The third edition of this best-seller updates its detailed information about how to construct, evaluate, and use questionnaires, and adds an entirely new chapter on customer loyalty. Included are two different methods of sampling and determining an appropriate sample size for reliable results; the reliability and validity of results; real examples of customer satisfaction measures and how they can be used; guidelines for developing questionnaires; scale development; the concept of quality; frequencies; sampling error; two methods of determining important service or product characteristics as perceived by the customer; discussion on the measurement and meaning of customer loyalty, and methods for loyalty-based management. -Readers will gain a sound grasp of the scientific methodology used to construct and use questionnaires utilizing the author's systematic approach. They will be able to pinpoint and focus on the most relevant topics, and study both the qualitative and quantitative aspects of questionnaire design and evaluation. These and many more important scientific principles are presented in simple, understandable terms.

• Language: English
• Publisher: ASQ Quality Press
• Release date: Apr 22, 2008
• ISBN: 9781636941974

Book preview

Measuring Customer

Satisfaction and Loyalty

Survey Design, Use, and

Statistical Analysis Methods

Also available from ASQ Quality Press:

Managing the Customer Experience: A Measurement-Based Approach

Morris Wilburn

Customer Satisfaction Research Management

Derek Allen

Competing for Customers and Winning with Value: Breakthrough Strategies for Market Dominance

R. Eric Reidenbach and Reginald W. Goeke

Analysis of Customer Satisfaction Data

Derek Allen and Tanniru R. Rao

Measuring Customer Satisfaction: Survey Design, Use, and Statistical Analysis Methods, Second Edition

Bob E. Hayes

Measuring and Managing Customer Satisfaction: Going for the Gold

Sheila Kessler

Customer Satisfaction Measurement Simplified: A Step-by-Step Guide for ISO 9001:2000 Certification

Terry G. Vavra

ANSI/ISO/ASQ Q10002-2004: Quality management — Customer satisfaction — Guidelines for complaints handling in organizations

ANSI/ISO/ASQ

The Certified Manager of Quality/Organizational Excellence Handbook, Third Edition

Russell T. Westcott, editor

The Quality Toolbox, Second Edition

Nancy R. Tague

Making Change Work: Practical Tools for Overcoming Human Resistance to Change

Brien Palmer

Innovation Generation: Creating an Innovation Process and an Innovative Culture

Peter Merrill

To request a complimentary catalog of ASQ Quality Press publications, call 800-248-1946, or visit our Web site at http://qualitypress.asq.org.

Measuring Customer

Satisfaction and Loyalty

Survey Design, Use, and

Statistical Analysis Methods

Third Edition

Bob E. Hayes

ASQ-Print

ASQ Quality Press

Milwaukee, Wisconsin

American Society for Quality, Quality Press, Milwaukee, WI 53203

© 2008 by ASQ

All rights reserved. Published 2008.

Library of Congress Cataloging-in-Publication Data

Hayes, Bob E., 1963-

Measuring customer satisfaction and loyalty : survey design, use, and statistical analysis methods / Bob E. Hayes.—3rd ed.

p. cm.

Includes bibliographical references and index.

ISBN 978-0-87389-743-3 (hardcover)

ISBN 978-0-87389-111-0 (pdf)

ISBN 978-1-63694-197-4 (epub)

1. Consumer satisfaction—Statistical methods. 2. Questionnaires. I. Title.

HF5415.335.H39 2008

658.8'12—dc22

2008020729

No part of this book may be reproduced in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher.

ASQ Mission: The American Society for Quality advances individual, organizational, and community excellence worldwide through learning, quality improvement, and knowledge exchange.

Attention Bookstores, Wholesalers, Schools, and Corporations: ASQ Quality Press books, videotapes, audiotapes, and software are available at quantity discounts with bulk purchases for business, educational, or instructional use. For information, please contact ASQ Quality Press at 800-248-1946, or write to ASQ Quality Press, P.O. Box 3005, Milwaukee, WI 53201-3005.

To place orders or to request a free copy of the ASQ Quality Press Publications Catalog, including ASQ membership information, call 800-248-1946. Visit our Web site at www.asq.org or http://www.asq.org/quality-press.

asq-logo-quality-press-address-black

For Marissa

This book is dedicated to you. You have grown into a beautiful person. You are a kind, loving, and forgiving person, qualities I admire. You make the world more joyful, and remind me that there is nothing more important than family and friends. Thank you for being there for me in my greatest time of need. Your support, love and acceptance have helped me immeasurably. I love you.

Table of Contents

Cover

Title page

Copyright

Contents

List of Figures and Tables

Preface

1 Introduction

MODEL OF CUSTOMER SATISFACTION QUESTIONNAIRE DEVELOPMENT AND USE

2 Determining Customer Requirements

QUALITY DIMENSION DEVELOPMENT

Identification of Quality Dimensions

Establishing Specific Examples of Quality Dimensions

Software Industry

Summary

CRITICAL INCIDENT APPROACH

Critical Incidents

GENERATING CRITICAL INCIDENTS

Interview

Categorization of Critical Incidents

QUALITY OF THE CATEGORIZATION PROCESS

Inter-judge Agreement

COMPREHENSIVENESS OF THE CUSTOMER REQUIREMENTS

BANKING INDUSTRY

AUTOMOBILE INDUSTRY

STATISTICAL SUPPORT

CHAPTER SUMMARY

3 Reliability and Validity

THE MEANING OF CUSTOMER PERCEPTION AND ATTITUDE

RELIABILITY

Classical Measurement Theory

Standard Error of Measurement

SOURCES OF ERROR AND RELIABILITY ESTIMATES

Measures of Stability and Test-Retest Reliability

Measures of Equivalence and Parallel Forms Reliability

Internal Consistency

Split-Half Reliability Estimate

Cronbach's Alpha Estimate

TYPES OF ERROR AND RELIABILITY ESTIMATES

BENEFITS OF SCALES WITH HIGH RELIABILITY

Detection of Differences in Satisfaction Levels

Detection of Significant Relationships Between Variables

FACTORS AFFECTING RELIABILITY

Number of Items in the Scale

Sample of People

THE NEED FOR MULTIPLE ITEMS IN MEASUREMENT

Summary

VALIDITY

Content-Related Strategy

Criterion-Related Strategy

Construct-Related Strategy

Summary

CHAPTER SUMMARY

4 Customer Satisfaction Questionnaire Construction

DETERMINING QUESTIONS OR ITEMS

Satisfaction Items Revisited

CHARACTERISTICS OF GOOD ITEMS

RESPONSE FORMATS

Checklist Format

Likert-Type Format

Advantage of Likert-Type Format

INTRODUCTIONS TO CUSTOMER SATISFACTION QUESTIONNAIRES

ITEM SELECTION

Judgmental Item Selection

Mathematical Item Selection

SUMMARY OF ITEM SELECTION

WEB-BASED CUSTOMER SURVEYS

Benefits of Web-Based Surveys

Principles for Designing Web-based Surveys

Tips for Conducting Web-Based Surveys

CUSTOMER LOYALTY MEASUREMENT

MEASUREMENT OF CUSTOMER LOYALTY

CUSTOMER SURVEYS AND CUSTOMER LOYALTY

5 Sampling Methods

STATISTICAL SAMPLING

TYPES OF SAMPLING: CENSUS, JUDGMENT, AND STATISTICAL

Census Sampling

Judgmental Sampling

Statistical Sampling

Summary

SAMPLE SIZE AND SAMPLING ERROR

Sampling Error

Standard Error of the Mean

Calculating Sample Size

Summary

RANDOM SELECTION PROCEDURES

Systematic Selection with a Random Start

Random Number Sampling

Summary

RESPONSE RATES

Establishing Response Rates

Methods of Increasing Response Rates of Mail Surveys

Summary

6 Customer Loyalty 2.0: Beyond the Ultimate Question

NPS METHODOLOGY

NPS CLAIMS

Recent Scientific Challenges to NPS Claims

Problems with NPS Research

MEASUREMENT AND MEANING OF CUSTOMER LOYALTY

Specificity vs. Generality of Measurement

Potential Loss of Revenue when using the NPS: An Example

Summary and Conclusions

Objective vs. Subjective Measures of Loyalty and Measurement Error

What Loyalty Questions are Measuring: A Factor Analytic View

Advocacy, Purchasing, and Retention Loyalty

Reliability of Loyalty Indices

Net Promoter Score vs. Satisfaction vs. Purchase Same

Single-Item Measures or Aggregated Metrics

Ranking Companies on Loyalty

ALI, PLI, and RLI Predict Business Growth

Hayes Loyalty Grid

Summary

7 Using Customer Satisfaction Questionnaires

EXAMPLES OF CUSTOMER SATISFACTION QUESTIONNAIRES

Software Quality

Facilities Department

Banking Industry

Automobile Industry

Statistical Support Department

USES OF CUSTOMER SATISFACTION QUESTIONNAIRES

SUMMARY INDICES

Presentation of Data

Summary

IDENTIFY IMPORTANT CUSTOMER REQUIREMENTS

Summary

CONTROL CHARTS

Interpretation of the Control Chart

Types of Data

CONTROL CHARTS FOR ATTRIBUTE DATA

The p Chart

Example of the p Chart

The c Chart

Example of the c Chart

Summary

CONTROL CHARTS FOR VARIABLE DATA

The x̅ and s Control Charts

Example of x̅ and s Charts

ORGANIZATIONAL COMPARISONS

DETERMINE EFFECTIVENESS OF INTERVENTION PROGRAMS

Training Programs

Benchmarking

ADDITIONAL QUESTIONS

CUSTOMER LOYALTY MANAGEMENT

Customer Lifetime Value (CLV)

Increasing the Lifetime Value of the Customers

Business Model and Customer Lifetime Value

CUSTOMER LOYALTY MANAGEMENT

Micro (Individual) Approach to Loyalty Management

Macro (Organizational) Approach to Loyalty Management

LOYALTY DRIVER ANALYSIS

Derived Importance

Performance

Loyalty Matrix

SUMMARY OF MICRO VS. MACRO APPROACH

MAKING IMPROVEMENTS AND ROOT CAUSE ANALYSIS

SUMMARY

8 Examples of Customer Satisfaction Questionnaires

DENTAL PATIENT SATISFACTION

Questionnaire Development

Sampling Plan

Survey Administration

Survey Results

Factor Analysis of the Client Satisfaction Survey

Summary

READER SURVEY

Questionnaire Development

Administration of the Survey

Results

Overall Newspaper Satisfaction Score

Summary

COFFEE SHOP

Questionnaire Development

Administration of the Survey

Results

Summary

CONCLUSIONS

Introduction to Appendices

Appendix A Critical Incidents Interview Forms

Appendix B Satisfaction Items and Customer Requirement Forms

Appendix C Measurement Scales

Appendix D Frequencies, Percentages, Probabilities, Histograms, and Distributions

Appendix E Descriptive Statistics

Appendix F Statistics, Parameters, and Sampling Distributions

Appendix G Decision Making and Hypothesis Testing

Appendix H T-Tests

Appendix I Analysis of Variance

Appendix J Regression Analysis

Appendix K Factor Analysis

Appendix L Table of Random Numbers

Bibliography

List of Figures and Tables

Preface

The first edition of this book was published in 1992. At that time, there were no books that covered the topic of customer satisfaction questionnaire development and use. Since then, the field of customer satisfaction measurement has flourished. In 1997, the second edition was published with more information about real-life examples of customer satisfaction questionnaires. It is hard to believe that 11 years has elapsed since the second edition. The growth of the field of customer satisfaction measurement has proliferated greatly, with articles and books being written by both researchers and practitioners. I have learned much about the field of customer satisfaction measurement through my professional experience. I have been fortunate enough to have conducted survey research for top-level enterprise companies including Siebel Systems, Agilent Technologies, Oracle, and Cisco Systems, and have managed customer satisfaction research as an employee to Fortune 500 firms and as an independent consultant. Much of the new content in this edition is based on that experience.

This edition, like the first two editions, includes topics such as customer requirement determination, uses of customer satisfaction questionnaires, reliability and validity of questionnaires, sampling techniques, item generation, and real-life examples of customer satisfaction questionnaires. The third edition contains information about customer loyalty measurement. There is no shortage of articles and books on the concept of customer loyalty, but some of the books, I feel, have given a simplistic look at the measurement of customer loyalty that detracts from the scientific underpinnings of psychological measurement. I have dedicated a chapter to the measurement of customer loyalty that examines both the meaning and measurement of customer loyalty. Specifically, this chapter includes a critique of the research behind the Net Promoter Score (NPS) and presents research that extends the field of customer loyalty measurement.

I have included discussions on Web-based surveys and have included information on customer loyalty management.

I had a desire to write this third edition for some time now. I put off writing the book for too long. Due to certain life-changing events, I now have the time and, more importantly, the motivation to complete this edition. Although writing is sometimes difficult for me, I do enjoy the process of trying to convey important scientific principles in simple, understandable terms. All too often, important scientific principles in textbooks are not presented in a way that allows for easy comprehension. I hope I have accomplished my goal of sharing what I know in a meaningful way.

I present these scientific principles in relation to the development of customer satisfaction questionnaires, but these issues are equally applicable in an attempt to develop any attitude measure (for example, employee attitude surveys). Although I do discuss several topics in scale development, my intent is not to cover the entire content of this area, but to present information directly related to the development of customer satisfaction questionnaires.

The book contains eight chapters. One chapter discusses the importance of customer satisfaction in the American economy, and the other chapters cover specific aspects of customer satisfaction questionnaire development and use. The chapters are designed to be complementary and to develop a coherent picture of customer satisfaction questionnaire development and use. The book contains several appendices that present important statistical concepts to aid readers who lack a statistical background. An understanding of the information in the appendices is helpful but not essential to developing customer satisfaction questionnaires.

Chapter 1 introduces the concept of quality and emphasizes the importance of assessing customer satisfaction and customer loyalty. Awareness of the importance of customer satisfaction/loyalty stems from two sources: (1) a national quality award given to outstanding American companies that demonstrate outstanding product and service quality, and (2) the desire to achieve financial security of companies that are able to satisfy their customers and subsequently able to increase customer loyalty. In addition, this chapter presents a model of customer satisfaction questionnaire development and use designed as an overview for the remaining chapters. This model will help readers see the big picture of customer satisfaction questionnaire development and use before reading about the specific steps in subsequent chapters.

Chapter 2 covers two methods of determining important service or product characteristics as perceived by customers. Identification of these characteristics is necessary in generating questions for the customer survey. These methods are called the quality dimension development process and the critical incident technique. The quality dimensions include those general aspects of the service or product that customers say define the quality of the service or product. The two methods also identify specific examples of these important characteristics that will help determine the types of questions that can be included in a customer satisfaction questionnaire.

Chapter 3 covers measurement issues related to customer satisfaction questionnaires, discusses reliability and validity, and contains formulae to estimate reliability. It is important to evaluate the quality of any measurement system. When we measure parts that come from an assembly line, we want to be sure that the measurement process accurately reflects the true size of the parts we are measuring. Similarly, we want to ensure that the measurement of our customers' attitudes accurately reflects their attitudes. Imprecise measurement can lead to poor business decisions. This chapter shows you how to evaluate the measurement properties of your questionnaires.

Chapter 4 presents guidelines for developing questionnaires, including the types of questions to be included, characteristics of good questions, response formats, and information to be included in the introduction. Additionally, this chapter discusses the benefits of conducting Web-based customer