Using Scanner Data for Food Policy Research

By Mary K. Muth, Abigail Okrent, Chen Zhen and Shawn Karns

Using Scanner Data for Food Policy Research is a practitioners’ guide to using and interpreting scanner data obtained from stores and households in policy research. It provides practical advice for using the data and interpreting their results. It helps the reader address key methodological issues such as aggregation, constructing price indices, and matching the data to nutrient values. It demonstrates some of the key econometric and statistical applications of the data, including estimating demand systems for policy simulation, analyzing effects of food access on food choices, and conducting cost-benefit analysis of food policies.

This guide is intended for early-career researchers, particularly those working with scanner data in agricultural and food economics, nutrition, and public health contexts.

• Describe different types of scanner data, the types of information available in the data, and the vendors that offer these data
• Describe food-label data that can be appended to scanner data
• Identify key questions that researchers should consider when acquiring scanner and label data for food policy research
• Demonstrate how to use scanner data using tools from econometric and statistical analyses, including the limitations in interpreting results using the data
• Describe and resolve key methodological issues related to using the data to facilitate more rapid analyses
• Provide an overview of published literature as background for designing new studies
• Demonstrate key applications of the data for food policy research

• Language: English
• Publisher: Academic Press
• Release date: Oct 12, 2019
• ISBN: 9780128145470

Mary K. Muth

Mary K. Muth, PhD, is director of RTI International's Food, Nutrition, and Obesity Policy Research Program. Muth conducts research studies for government agencies and other organizations to analyse the impacts of policies, regulations, and other initiatives affecting food and agriculture. She specializes in the areas of nutrition, food security, food waste, food pricing, food labelling, food reformulation, and food safety. She has extensive experience analysing food availability, purchase, and consumption data and developing economic models of the impacts of food policy. Dr. Muth is also an adjunct associate professor in the Department of Agricultural and Resource Economics at North Carolina State University.

Book preview

Using Scanner Data for Food Policy Research

First Edition

Mary K. Muth

Abigail M. Okrent

Chen Zhen

Shawn A. Karns

Table of Contents

Cover image

Title page

Copyright

About the authors

Preface and acknowledgments

1: What is scanner data and why is it useful for food policy research?

Abstract

1.1 Understanding barcodes—A necessary condition for working with scanner data

1.2 Overview of types of scanner data

1.3 Assessing the extent of coverage of scanner data across the marketplace

1.4 Types of food policy and research questions relevant for scanner data

1.5 Barriers and considerations in using scanner data

1.6 Overview of this book

2: Sources of scanner data across the globe

Abstract

2.1 IRI

2.2 Kantar Worldpanel (household scanner data in multiple countries outside of the United States)

2.3 Nielsen

2.4 SPINS (US data only)

2.5 Store scanner data obtained directly from stores

2.6 Questions to consider when acquiring or purchasing scanner data

3: Label and nutrition data at the barcode level

Abstract

3.1 Overview of contents of food product label datasets

3.2 Vendors offering food product label data (multiple countries)

3.3 Food product label data available at no cost (United States only)

3.4 Questions to consider when acquiring or purchasing label and nutrition data

4: Methodological approaches for using scanner data

Abstract

4.1 Overview of data structure, fields, and linking

4.2 Adjusting quantities to common units

4.3 Adjusting quantities for underreporting

4.4 Aggregating across relevant dimensions for an analysis

4.5 Working with scanner data for private-label products

4.6 Constructing missing prices and weights for random-weight products

4.7 Accounting for discounts and coupons

4.8 Constructing price indices

4.9 To weight or not to weight the data

5: Insights from past food research using scanner data

Abstract

5.1 Data comparisons

5.2 Food prices, taxes, and subsidies

5.3 Promotions and advertising

5.4 Food safety

5.5 Nutrition, health, and food production labels

5.6 Tracking the nutritional composition of foods and its effect on diet quality

5.7 Market competition

5.8 US food assistance programs

5.9 Food access

6: Estimating food demand systems using scanner data

Abstract

6.1 General approach to estimating demand systems

6.2 The censored EASI demand system and estimation of nutrient elasticities

6.3 Data and variable construction

6.4 Empirical results

6.5 Concluding remarks

7: Measuring the food environment using scanner data

Abstract

7.1 General approach to measuring the US food environment

7.2 Approximate Healthy Eating Index for stores and households

7.3 Measuring the retail food environment

7.4 Association between healthfulness of household purchases and retail food environment

7.5 Concluding remarks

8: Conducting cost-benefit analyses using scanner and label data

Abstract

8.1 General approach

8.2 Examples of ex ante analysis applications

8.3 Concluding remarks

Index

Copyright

Academic Press is an imprint of Elsevier

125 London Wall, London EC2Y 5AS, United Kingdom

525 B Street, Suite 1650, San Diego, CA 92101, United States

50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States

The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

© 2020 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN 978-0-12-814507-4

For information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Publisher: Candice Janco

Acquisition Editor: Candice Janco

Editorial Project Manager: Sara Valentino

Production Project Manager: Debasish Ghosh

Cover Designer: Miles Hitchen

Typeset by SPi Global, India

About the authors

Mary K. Muth, PhD, is director of RTI International’s Food, Nutrition, and Obesity Policy Research Program. She conducts research studies to analyze the impacts of policies, regulations, and other initiatives affecting food and agriculture particularly in the areas of nutrition, food pricing, food labeling, food reformulation, food safety, and food waste. She has analyzed scanner and food label data for almost 20 years for a broad variety of food policy applications. She is also an adjunct associate professor in the Department of Agricultural and Resource Economics at North Carolina State University.

Abigail M. Okrent, PhD, was a research economist at the US Department of Agriculture Economic Research Service, where she investigated the role of food and farm policies on food choices and diet quality, when this book was written. Her research has used household and retail scanner data to analyze determinants of food choice and its implications for health outcomes. She has also conducted studies to investigate the statistical properties of scanner data.

Chen Zhen, PhD, is an associate professor of agricultural and applied economics at the University of Georgia (UGA), where he holds the UGA Athletic Association professorship in Food Choice, Obesity, and Health Economics since 2015. Between 2006 and 2015, he was a research economist at RTI International’s Food, Nutrition, and Obesity Policy Research Program. His research agenda has a strong emphasis on scanner data applications. He has studied topics including properties of scanner data sets, sugar-sweetened beverage taxes, shelf nutrition labels, the Supplemental Nutrition Assistance Program, panel food price indexes, and development of tractable large demand system models. His research on sugar-sweetened beverage taxes was featured in the New York Times and NPR.

Shawn A. Karns is a senior public health analyst in RTI International’s Food, Nutrition, and Obesity Policy Research Program where she constructs and analyzes a broad range of data for food policy research. She has extensive experience managing and analyzing household and retail scanner data and food label data for studies on food costs, food waste, food reformulation, and food labeling.

Preface and acknowledgments

Mary K. Muth, Research Triangle Park, NC, United States

Abigail M. Okrent, Washington, DC, United States

Chen Zhen, Athens, GA, United States

Shawn A. Karns, Research Triangle Park, NC, United States

Our first foray into working with scanner data began almost 20 years ago when a couple of us developed the FDA Labeling Cost Model using IRI store scanner data and, later, updated the model to use Nielsen store scanner data. When we first started, few researchers had worked with scanner data, and much was unknown about their properties. Prior to that time, scanner data had primarily been used only for commercial market research purposes. We began a deeper dive into using scanner data when the US Department of Agriculture’s Economic Research Service (ERS) asked us to document the statistical properties of Nielsen household scanner data in 2006. Our work involved indepth discussions with Nielsen staff, comparisons to government expenditure surveys, and comparisons of attitudes between respondents to nationally-representative surveys and the Nielsen panelists. We began working extensively with IRI data, including its store scanner data, household scanner data, and label data when ERS asked us to examine and document the statistical properties of the data. As part of that work, in April 2016, we conducted a workshop sponsored by ERS in Washington, DC with researchers across the United States to discuss approaches and review results of research that used scanner data and food label data.

Our research with scanner data has included analyses of food choices and the nutrition and health outcomes associated with those choices, food policy evaluation including food taxes and subsidies and food access, and spatial food price estimation. The unique features of scanner data allowed us to examine some of the food policies that were impossible to analyze with traditional sources of data. For example, by merging Nielsen data with Gladson and manufacturer data on nutrition facts, we published a series of original research articles on sugar-sweetened beverage demand that were among the first to predict the likely effects of a sugar-sweetened beverage tax. These would not have been possible with conventional budget surveys because of a lack of information on nutrient content or prices.

As the complexity and depth of scanner and food label data have increased and their uses have increased dramatically, we felt there was a need to provide information to researchers interested in using store or household scanner data and associated label data. Our intent is to help jumpstart researchers’ understanding of the data for designing and conducting analyses. A wealth of research is being conducted using various types of scanner data, so the chapter covering literature by food policy topic is a good starting place for anyone embarking on new research using scanner data. We feel that the information in this book could be useful to graduate students considering dissertation research topics, government analysts that need real market data for policy analyses, and early career researchers that are just beginning to work with scanner and food label data.

Although our experience is based primarily on using data in the United States, we also discuss data available in other countries in the chapters describing the various sources of data and the relevant published literature. In addition, the approaches to using scanner data are generally transferable to different country contexts. Scanner data from less developed countries are becoming increasingly available, and we anticipate researchers will find future opportunities to apply scanner data from those countries in policy analyses.

We would like to acknowledge and thank the organizations that have funded our research using scanner data including the US Food and Drug Administration, the US Department of Agriculture (particularly Aylin Bradley, Mark Denbaly, Ephraim Leibtag, and Megan Sweitzer), Robert Wood Johnson Foundation (particularly Mary Story), and RTI International. Without their support, we would not have had the opportunity to develop the depth of experience needed to write this book. We also would like to thank the individuals at companies that produce scanner data and food label data for answering all of our detailed questions and providing information that has not previously been documented. We also thank Sharon Barrell for her excellent editing of the original text of the book, and Diane Kim, who assisted us in the early stages of the literature review. Finally, we acknowledge the support of our families, friends, and colleagues who give us the strength to pursue our research.

The views expressed here are those of the authors and not necessarily those of USDA.

1

What is scanner data and why is it useful for food policy research?

Abstract

Scanner data obtained from stores or household panels allow for much more detailed analyses of food purchase behavior than previously possible using aggregated data. The data are recorded at the scannable barcode level and can be linked to detailed information on characteristics of products, purchasers, and stores. The main commercial suppliers of store and household scanner data across the globe are Kantar, IRI, and Nielsen, but sometimes researchers obtain data directly from stores. The data are increasingly being used for a broad range of food policy research applications. When using the data, researchers should have an understanding of the data collection procedures used by the data vendors; the extent of coverage across geographies, stores, households, and products; and potential barriers or other practical considerations.

Keywords

Store scanner data; Household scanner data; Universal Product Code (UPC); Global Trade Item Number (GTIN®); Price Look Up (PLU) code; Kantar; IRI; Nielsen; SPINS; GS1®

Before scanner data became available for use, researchers conducting studies on food purchasing behavior and the effects of food policies relied primarily on highly aggregated data from government data sources (National Research Council (NRC), 2005; Nayga, 1992). The level of aggregation in prior data sources meant that it was generally not possible to tease out differences across forms or varieties of a product, types of stores, regions of the country, seasons of the year, and other factors. When store scanner data became available (either from individual stores or from scanner data companies that obtain data from stores and sell syndicated datasets), researchers were able to conduct analyses at a much more granular level across multiple dimensions than previously possible. With the availability of scanner data collected from household panels, researchers were able to analyze research questions that relied on having a record of purchases for a household over time that is tied to demographic information about the household (Box 1.1). The availability of scanner data also meant that new research methods and techniques had to be developed to handle extremely large datasets and to construct relevant variables for analyses.

Box 1.1

Store versus household scanner data

Store scanner data are collected directly at the point of purchase at retail establishments and are often referred to as point-of-sale data. In contrast, household scanner data are collected from a household panel that records their purchases on a periodic basis.

Both types of data represent what researchers often refer to as food at home (FAH) because most purchases are prepared and consumed at home. Purchases at restaurants and other food-away-from-home (FAFH) locations are not represented in the scanner data described in this book.

When working with scanner data, it is important to understand that scanner data companies such as IRI, Kantar, and Nielsen collect the data for commercial purposes. When developing data collection processes, these companies’ primary intent is to conduct analyses and prepare reports for use by their customers, which are generally manufacturers and retailers of consumer packaged goods. Selling their data directly for use by researchers is a relatively small portion of their revenue and outside of their typical business models. The sample of stores and households is generally selected based on convenience rather than use of statistical sampling methods to ensure representativeness. However, the scanner data companies calculate survey weights or projection factors that researchers can use in analyzing the data or provide data that have already been weighted to represent a geographic area or country.a When conducting analyses for their own clients, the scanner data companies may integrate data from other proprietary sources to ensure alignment with manufacturer shipments data or other targets. However, these other weights or adjustments are not available to the research community. Given this situation, researchers need to have a clear understanding of the nature of the data they are using for an analysis and how it affects the analysis approach and interpretation of analysis results.

Although scanner data are available for purchase soon after they are generated, the cost of the most recent data often puts it out of reach for many researchers. Typically, the most recent data that food policy researchers are able to access is from the prior year, and often the data are from several years past. Although the age of the data may prohibit the ability to analyze events immediately after their occurrence, the data are valuable for numerous other applications related to food policy research.

1.1 Understanding barcodes—A necessary condition for working with scanner data

The ability to collect and use scanner data for analyses arose with the adoption of scannable product codes, or barcodes, that are now used globally on nearly all packaged food and other types of products in retail stores. Scannable barcodes represent a unique manufacturer, brand, product, flavor, and size of a product. The global term for scannable barcodes is Global Trade Item Number (GTIN®), and these barcodes are overseen by the not-for-profit information standards organization GS1 US® (GS1 US, 2018). In the United States, the consumer-level GTIN has 12 digits and is commonly referred to as a Universal Product Code (UPC). In other countries, the GTIN has 13 digits and is commonly referred to as European Article Numbering (EAN) System code, or EAN-13. Most packaged fruits and vegetables and other perishable items also have 12- or 13-digit scannable barcodes, but products that are sold as random weight, meaning that consumers select individual items and pay for them by weight, may be tracked using Price Look Up (PLU) codes or barcodes used only within the store. GTINs, UPCs, or EAN codes are different from stock-keeping units (SKUs) that retailers used to track inventory for internal operations, but they can be used like SKUs in a retailer's inventory database (Hudson, 2017; Box 1.2).

Box 1.2

Types of scannable barcodes

Depending on the context, scannable barcodes are referred to as the following:

●Universal Product Code (UPC)

●European Article Numbering (EAN) Code

●Global Trade Item Number (GTIN)

●Price Look Up (PLU) code

Scannable barcodes comprise a company prefix identifying the brand owner (e.g., manufacturer or retailer), a set of digits that identifies the specific product, and a check digit (Box 1.3). The prefix is the initial 6- to 10-digit code licensed to the brand owner by GS1. Brand owners assign the remaining digits of the code (called the item reference number) to each of its products. The last digit, called the check digit, is calculated from the preceding digits and used to ensure the barcode is entered correctly in the rare instances when a barcode is hand-keyed because of an unreadable bar code.b For private-label (or store-brand) products, the prefix is typically associated with the retailer, although a contract manufacturer may manufacture the product. Because the brand owner assigns the portion of the barcode that identifies the specific product, there is no central database that can be used to identify products by barcode except for the scanner data described in this bookc.

Box 1.3

An example of a scannable barcode

Note: Used with permission from GS1.

In scanner data, the barcode is accompanied by a product description field that includes the brand name and a unique abbreviated format of the product name, form or variety, and package size. GS1 provides guidance to manufacturers regarding when to assign a new barcode to a modified product, such as changes in declared formulation or functionality, changes in declared net content, gross weight changes of more than 20%, and addition or removal of a certification mark (GS1 US, n.d.; Fernandez, 2018). Thus, if the characteristics of a product change substantially, a manufacturer might assign a new barcode to a product (Martinez & Levin, 2017). However, the ultimate decision is at the manufacturer's discretion. In cases in which the manufacturer retains the barcode for a modified product, a data vendor might assign a generation code to the barcode to track the change (Muth et al., 2016). Although GS1 strongly discourages the practice, manufacturers might also reuse a barcode for substantially different products; thus, researchers might need to verify that a product is the same across years of data by comparing the product descriptions.

In contrast to barcodes that are used for packaged products, PLUs are used for random-weight products and other items in retail stores (Box 1.4). For most types of products, retailers determine the PLUs for use on their products. However, a set of universal 4- and 5-digit PLU codes has been developed for nonpackaged, random-weight fresh produce. Originally developed by a volunteer committee of the Produce Marketing Association in 1988, the PLU codes are now governed and maintained by the International Federation for Produce Standards (IFPS) (IFPS, 2017; Treacy, 2018). Nearly all random-weight produce in the United States, Canada, Australia, and New Zealand is sold using IFPS standard codes, and several other countries such as Chile and the Netherlands use them on most exported products (Treacy, 2018).

Box 1.4

An example of a scannable PLU code

Note: Used with permission from the Washington Apple Commission.

The PLU codes are used to identify fruits and vegetables by commodity, variety, and size.d Currently, the 4-digit codes represent conventionally grown produce; the addition of a 9 in front of the code indicates the produce was organically grown. However, in the future, some conventional and organic product PLUs will start with an 8 to allow for an increase in the number of PLU codes available for use (Treacy, 2018). A portion of the produce codes are designated as retailer assigned PLUs for products not included in the IFPS standard codes and therefore vary across retailers. In North America, some random-weight products have a DataBar Omni Directional Stacked barcode that represents a 14-digit GTIN with a brand owner prefix and the 4- or 5-digit PLU code (Treacy, 2018). In addition, a generic set of barcodes for packaged produce combines the PLU code with the generic prefix of 033383, which is licensed to the Produce Marketing Association (Treacy, 2018). In scanner data, sales of PLU products may be represented using the GTIN, actual PLU code, or a pseudo-UPC assigned by the data vendor. If a pseudo-UPC is used, a translation to the PLU code or a description of the product should accompany the dataset.

1.2 Overview of types of scanner data

In higher income countries, scanner data are of two types—data collected from stores and data collected from a household panel. Both types of data can be linked by scannable barcode to detailed label information including nutrient content; health claims; and, in some cases, ingredient lists. In more recent years, store and household scanner data have become available from several low- and middle-income countries, and data vendors will likely continue to increase the number of countries in their data collection efforts. However, in many low-income countries, the limited use of scanning technology and the reliance on traditional markets mean that scanner data are typically not available, but efforts are being made to collect comparable data through other means.

1.2.1 Store scanner data

IRI and Nielsen are the predominate suppliers of store scanner data across the globe. Both companies supply store scanner data in the United States. In addition, SPINS collects scanner data from specialty and natural food stores. Outside of the United States, IRI collects store scanner data in several European countries, Australia, and New Zealand, and Nielsen collects store scanner data from most of these same countries in addition to several low- and middle-income countries in Central and South America, Africa, and Asia. Aside from the commercial scanner data companies, some researchers have negotiated agreements with individual stores or chains to obtain data for specific research purposes.

To collect store scanner data, data vendors establish relationships with store chains and independent stores to obtain scanner data feeds on a weekly or other periodic basis (Muth et al., 2016). Typically, if a store chain agrees to provide its data, it provides data for all the stores in the chain. Smaller, independent stores must have electronic scanning capability to participate; thus, in some cases, a scanner data company might help a store install scanning capability (Muth et al., 2016). Note that the selection of stores is not randomized in the way that one might do for a nationally representative survey or an experimental study.

At their most disaggregated level, store-based scanner data contain the sales value (in dollars) and the quantity of products sold (in units) for an individual barcode at a specific store in a given week. Alternatively, store-based scanner data might represent the total sales value and quantity for an individual barcode for all stores in a market or region in a given week. Generally, each individual barcode is represented at its most disaggregated level, thus representing a specific brand, flavor, and size. However, in some cases, particularly for private-label or store-brand products, a record may represent an aggregation across similar products in a product category (e.g., canned tomatoes in different package sizes). Depending on the source of the data, a record will also contain the package size (i.e., weight or other measure of volume of the product) as a separate field, or the package size information must be extracted from the text description of the product. With the package size information, researchers can calculate unit values, such as price per ounce, by dividing the total value of sales by the package size measure.

Store data are relatively easy to manipulate and analyze given the structure of the data and can be useful for measuring the quantity sold and average prices of different types of foods sold over time and across markets. If the records contain information on nutrition content from the label, they can be used to track the effects of changes in quantity sold on measures of specific nutrients available in the food supply.

Results of analyses of scanner data are only representative of the stores that provide the data to a researcher or a scanner data company that compiles them from multiple stores. However, some syndicated data sets are weighted using projection factors so that the total sales value and total quantity sold can be interpreted as representative of a market or country. When using store scanner data, researchers must be sure to interpret the results of analyses appropriately based on the extent of coverage of the stores in the dataset and whether the data have been weighted to be representative.

1.2.2 Household scanner data

In the United States, IRI and Nielsen are the sole suppliers of household scanner data. Both data vendors derive their data from the National Consumer Panel (NCP), which is an operational joint venture owned equally by IRI and Nielsen (Muth et al., 2016; National Consumer Panel (NCP), 2018).e The NCP recruits, equips, and provides incentives to a panel of US consumers and collects the data using handheld scanning devices and other means, and IRI and Nielsen use the data for their separate purposes (NCP, 2018). Outside of the United States, Kantar and Nielsen supply household-based scanner data for multiple low-, middle-, and high-income countries, sometimes through collaborations with other companies. Note that in less developed countries, household purchase data are often obtained through nonscanning methods including paper and pencil diaries and bin collection of food packaging from households; in these cases, the nature of the data is fundamentally different from the data collected using electronic methods.

To collect household scanner data, scanner data companies recruit households to join a panel and record their purchases using a handheld scanning device or smartphone application (Muth et al., 2016; Muth, Siegel, & Zhen, 2007). Households may be recruited through a variety of means such as online advertising, social media, and direct email marketing (Muth et al., 2007, 2016). After households are recruited, they are asked to complete a detailed questionnaire that includes demographic information. Households are then selected by the scanner data company to join the panel with the goal of achieving specified demographic targets to ensure representativeness