People Aren’t Robots: A Practical Guide to the Psychology and Technique of Questionnaire Design

By F. Annie Pettit PhD FMRIA

The 4th edition is newly revised in 2023 and includes a forward by Pravin Shekar, The Outlier Marketer and Former Interim CEO of ESOMAR!

This book will help marketers, brand managers, and advertising executives who may have less experience in the research industry create great questionnaires and collect high quality data. It will also help academic and experienced researchers write questionnaires that are better suited for the general population, particularly when using research panels and customer lists.This book was conceived by experienced researcher with more than twenty years of practical experience who realized that many questionnaire guides continue to treat the people who answer questionnaires as robots rather than as fallible, imperfect people.Topics include general considerations related to the process, how to write screener questions, how to write data quality questions, and how to tackle specific types of questions from single-selects, grids, scales, and more. And, it does it from an empathetic and respectful point of view which points people first and researchers second.

• Language: English
• Publisher: F. Annie Pettit PhD FMRIA
• Release date: Oct 27, 2016
• ISBN: 9781370693108

F. Annie Pettit PhD FMRIA

Annie Pettit, PhD, is a Fellow of the Marketing Research and Intelligence Association. She has more than 15 years of experience as an online market researcher and specializes in market research methodologies, survey research, and data quality. Her expertise in research methods and data quality has been highlighted through numerous conference presentations, including ARF, CASRO, MRA ,MRIA, NetGain, NewMR, and IIR. She has also published numerous articles in both professional and refereed magazines and journals. Annie tweets at @LoveStats and maintains the LoveStats marketing research blog where she occasionally showcases her attempts at being a better baker and gardener

Book preview

People Aren’t Robots:

A practical guide to the psychology and technique of questionnaire design

F. Annie Pettit, PhD, FCRIC

Copyright © 2023 by F. Annie Pettit

Cover art by Dustin Holmes

All rights reserved. This book or any portion thereof may not be reproduced or used in any manner whatsoever without the express written permission of the publisher except for the use of brief quotations in a book review or scholarly journal.

All examples conceived by the author.

Fourth Printing: 2023

ISBN 978-1539730644

Dedicated to the millions of people

who kindly share their time and opinions to

help make our world a better place

People Aren’t Robots

Table of Contents

Preface 4

Foreword 6

Chapter 1: Get in the frame of mind 8

People aren’t robots 8

People make mistaktes 8

People are basically good 9

People are busy 9

People have needs 10

People are biased 12

People first, researchers second 13

Chapter 2: General Techniques 14

General to specific 14

Use humanistic language 14

Shorten everything 14

Choose common 17

Be lenient with grammar 18

Use Your Manners 19

Have fun 20

Assume mobile only 21

Power to the people 22

Chapter 3: Screen for the right people 23

Category screener 23

Branded screener 25

Region screener 27

Market research screener 28

Screen out on more than extremes 29

How late is too late 30

Chapter 4: Screen for quality people 31

People having a bad day 31

People wanting the incentives 31

Data quality process 32

Data quality measures to use 33

Red herrings 33

Straightlining 34

Ranking 35

Sum question 35

Following instructions 35

Over click 36

Under click 36

Low incidence 36

High incidence 36

Open end length 37

Profanity 37

Gibberish 38

Open end satisficing 38

Speeding 39

Avoid: Contradictions 39

Do not use: Select Strongly Agree here 40

Do not use: Regional vocabulary 40

Chapter 5: Generic techniques 42

Know what a question is 42

Operationalize 42

Provide an opt-out 43

Start with the qualifier 45

Respect all people 45

Use respectful words 45

Use people first language 46

Use accessible technology 47

Avoid leading and loaded questions 48

Avoid double barreled questions 48

Avoid referring to earlier answers 49

Order of answer options 49

Alphabetical 49

Chronological 50

Randomized 50

Reverse ordered 51

Style of answer options 51

Chapter 6: Question-specific techniques 54

Numeric questions 54

Start with a number 54

Include a zero 54

Include the largest possible number 55

Don’t duplicate answers 56

Don’t skip answers 56

Use intuitive breaks 57

Use common breaks 57

Play the misconstrue game 58

‘Or’ does not equal ‘to’ 59

Etc does not equal eg does not equal ie 60

Use your words 61

Request estimates 62

Single-selects/Multi-selects 63

Scales 64

Scales don’t reveal truth 64

Number of points 64

Balanced scales 64

Unbalanced scales 65

Midpoints 65

Move adverbs to the scale 66

Scale labels 67

Grid/matrix questions 67

Deconstruct grids 67

Choose few grid items 68

Create reverse keyed items 69

Avoid words that negate 69

Instructions 69

Rank questions 70

Sum questions 70

Other (Please specify) 70

Short text 71

Long text 71

Chapter 7: Sensitive Questions 73

Don’t project negativity 73

Socially undesirable behaviours 74

Illegal activities 75

Chapter 8: Demographics 77

Gender/Sex 77

Age 78

Matching words 79

People don’t want to feel old 79

Race/Ethnicity 80

Education 80

Employment 81

Children 82

Chapter 9: Tidbits 83

Branding a questionnaire 83

Questions per page 83

Pretesting 83

Email subject lines 84

Reminders 84

Reveal the results 85

About the Author 87

Preface

Questionnaire design is easy until you’re faced with horrid data quality. The problem, as with most things, is that there is an art and science to designing an effective questionnaire. This book will guide you in a short, easy to read, and easy to follow format.

How is it different from all the other questionnaire design books out there?

It gives practical advice from someone who has witnessed more than twenty years of good and poor choices that experienced and inexperienced questionnaire writers make. Yes, even academic, professional researchers make plenty of poor questionnaire design choices. Yes, even I make mistakes. I’ve seen every mistake in this book in live questionnaires.

It outlines how to design questions while keeping in mind that people are fallible, subjective, and emotional human beings. Not robots. It’s about time someone did this, don’t you think?

I hope that once you understand and apply these techniques, you too will hear people say this is the best questionnaire I’ve ever answered!

Foreword

When you don’t know where to go

No map can help

If you think you know

but don’t know where you are

No mapmaker can help you

When you know what you know

and what you don’t

yet have the courage the seek the right help

Then you have week and truly arrived

on the path forward

one step at a time

one learning, implemented

This book is a map

Pick it

Read it

Use it

Re-read

It is never cold

It is always the attire

As there is never a wrong answer

Only a desire to ask the right questions

'coz people aren’t robots!

***

Pravin Shekar

The Outlier Marketer

Former Interim CEO of ESOMAR

Chapter 1: Get in the frame of mind

Great questionnaires can only be created if you, the author, are in the right frame of mind. Yes, you must have a specific set of research goals written out and ready to be operationalized into specific questions but that’s not what

Reviews for People Aren’t Robots

[amalelagder237 · Rating: 4 out of 5 stars]

I learned a lot of practical steps from this book as a newbie researcher. I'd recommend this short read for everyone. You'll get the most of it if you wrote a survey's checklist according to this book so you'd double check everything before sending every survey over. Didn't like the formatting though so 4 starts.