Statistics for Applied Behavior Analysis Practitioners and Researchers

By David J. Cox and Jason C. Vladescu

Statistics for Applied Behavior Analysis Practitioners and Researchers provides practical and useful content for individuals who work directly with, or supervise those who work directly with, individuals with ASD. This book introduces core concepts and principles of modern statistical analysis that practitioners will need to deliver ABA services. The organization of the book works through the flow of behavior analytic service provision, aiming to help practitioners read through research, evaluate intervention options, incorporate statistics in their analysis of time-series intervention and assessment data, and effectively communicate assessment and intervention effects using statistics.

As professionals who provide applied behavior analysis (ABA) services are required to use evidence-based practices and make data-based decisions regarding assessments and interventions, this book will help them take a modern, scientific approach to derive knowledge and make decisions based on statistical literacy.

• Describes the logic behind the many variations to statistically describe human behavior
• Explains the conditions under which variations in statistical description are most appropriate
• Highlights common methods for quantitively determine the effectiveness of an intervention
• Discusses the unique challenges of the time-series data and reviews current solutions
• Covers current standards in writing and presenting statistical effects of interventions
• Reviews the application of statistical description for both single-case and between-group experimental designs

• Language: English
• Publisher: Academic Press
• Release date: Jun 17, 2023
• ISBN: 9780323985277

David J. Cox

David J. Cox, Ph.D., M.S.B., BCBA-D currently works as the VP of Data Science at RethinkFirst and is faculty at the Institute for Applied Behavioral Science at Endicott College. Dr. Cox has earned a M.S. in Bioethics from Union Graduate College; a PhD in behavior analysis from the University of Florida; a post-doctoral fellowship at the Behavioral Pharmacology Research Unit of Johns Hopkins University School of Medicine; and a post-doctoral fellowship at Insight! Data Science. Since 2014, Dr. Cox’s research and applied work has focused on how to effectively leverage technology, quantitative modeling, and artificial intelligence to ethically optimize behavioral health outcomes and clinical decision-making. Based on his individual and collaborative work, he has published 50+ peer-reviewed articles, three books, and 150+ presentations at scientific conferences.

Book preview

Preface

David J. Cox and Jason C. Vladescu

Wherever there is number, there is beauty.

–Proclus

Scientists love data. To make sense of data, you need numbers. And, to make sense of many numbers, you need mathematics. Practical mathematics do not need to be complicated. Simple mathematical functions help us balance our bank accounts and predict the cost of our dinner. But the world is often a complicated place and, sometimes, more complicated mathematical functions are better at helping us describe the universe around us. This book is about one area of mathematics—statistics—and how it can be practically used to describe the wondrous relations between behavior and the environment. Before you dive into the content and immerse yourself in these cool waters, it’s important to understand what this book is, what it is not, and the reasons we chose to write it in the first place.

What this book is (and isn’t)

This book is an introduction to statistics for behavior analysts. Behavior analysts are scientists and, assuming the first paragraph is true, they also love data. We use data for just about every decision we make—from understanding behavior contextually—to answering whether our intervention is effective—to seeking to convince others to continue funding our services or research. Data are the lifeblood of our work. Despite our love of data, we don’t always use it to its fullest capacity. Like simple addition and subtraction in finance, graphical displays depicting single-case experimental designs are the analytic workhorse of modern behavior analysis. They get the job done well enough, they are historically reliable, and they allow us to discuss our work easily with other behavior analysts. But you can likely do more with your data than you realize. There are patterns of behavior-environment relations waiting to be uncovered, knowledge about the universe yet to be described, and ways of helping people that can be illuminated if we take the time to play creatively with our data. Statistics are one way to do this. And, by creating single-case experimental design graphs, you are already behaving statistically even if you didn’t know it!

Yet, it’s possible to do statistics illogically. Statistics are verbal behavior constrained by the field of mathematics which are further constrained by the field of logic. Thus, at its core, there are rules people must follow when doing statistics for its underlying logical system to be, well, logically coherent. This book attempts to help outline how some of this logic can be applied to the work behavior analysts do. Specifically, the book is organized sequentially around the decisions behavior analysts make when collecting data, turning their observations about behavior and the environment into numbers, aggregating them within and across clinical or experimental sessions, and then making sense of the behavior-environment relations they examine. Each of these steps involves numbers, statistics, rules around how to do it logically, and—perhaps most excitingly—additional ways that behavior analysts might creatively squeeze more juice out of their data. As an introductory text, the primary goal is to introduce behavior analysts to the role of statistics in the decisions they make every day.

As an introductory text, this book is also designed for those who are new to the language of statistics or who pushed their statistical knowledge to some long-forgotten corner of their behavioral repertoire. We tried to cover the material as simply as we were capable of and to focus more on helping behavior analysts merge the way they think about the world with the language of statistics. As a result, this book does not get into advanced statistical modeling, statistical simulations, optimization procedures, machine learning, or any of the other hot statistical topics of our day. If these are the topics you are looking to dance with, this is not the book for you. Similarly, a note of framing for the statisticians who felt the need to pick up this book for some odd, strange reason. The focus of this book is how statistics can be used as a tool by behavior analysts. Thus, we described how behavior analysts can wrap statistics around what they do. (i.e., statistics play a supporting role). We purposely did not try to use statistical theory to change what behavior analysts do. We feel statistics are merely a one instrument in the scientist’s toolbelt, rather than their guiding light; and we wrote the book accordingly.

Lastly, we would be remiss not to comment about the tone of the book. Mathematics, statistics, and logic can be intimidating and tough sledding for the initiated and uninitiated alike. As Steven Hawking is claimed to have stated, Someone told me that each equation I included in the book would halve the sales. If true, then any book on statistics is likely to have only a handful of readers (often the authors’ parents and significant others). To turn this drudgery to—gasp!—fun, and (we hope) increase readership, we attempted to employ a more informal, engaging, and—in some instances dare we say—a humorous tone. As academics, we know this is sometimes looked down upon. But we’re more interested in function over form. And it’ll be worth it if we can get even a small portion of behavior analysts feeling comfortable using the beauty of mathematics to creatively explore their data.

Onward, Ho!

Regardless of where you are at in your comfort using mathematics and statistics, we sincerely hope you find this book useful in your research and practice. We’re also really excited to see the unique, creative, exciting, and thought-provoking patterns you uncover about behavior-environment relations as you gain fluency in using statistics to play with your data. Without further ado, let’s get to the good stuff.

Chapter 1

The requisite boring stuff part I: Defining a statistic and the benefit of numbers

Abstract

A hallmark of behavior analytic research and practice is the collection, analysis, interpretation, and presentation of data aggregated above and beyond each single response occurrence. Unbeknownst to many behavior analysts, the previous sentence is the definition of statistics. Despite its often notorious reputation in behavior analytic circles, behavior analysts use statistics on a daily basis and in most (all?) of their applied and experimental work. In this chapter, we review what statistics are, why scientists and practitioners find them useful, the small branch of statistical analyses that should carry skepticism in null hypothesis significance testing, and common myths and misconceptions about statistics. Once the basics are reviewed, we outline how much of what statistics and data-based theory boils down to its comparisons of models about how an individual’s behavior is influenced by the environment in which they live. With this reframe about what statistics involve behind us, we then close the chapter by highlighting examples of statistics in applied behavior analysis to further drive home the point that behavior analysts do, in fact, use statistics with regularity and fervor.

Keywords

Statistics; applied behavior analysis; data collection; data measurement; models; model comparisons; statistical myths

In God we trust. All others must bring data.

Robert Hayden

Introduction

People seem to have one of two reactions to the word, Math. In both situations, the pupils dilate and pulse quickens. For some, this happens from dread; and, for others, from pure delight. Regardless of how you’ve found this book, we hope by the end you’ll always fall in the latter camp. After all, math and statistics are just verbal behavior so there really isn’t anything to be afraid of once you learn the language. We’re excited you’re here to learn about what math and statistics have to offer as one method for describing environment-behavior relationships.

We intentionally sought to craft a text that is different from previous statistics books you may have encountered. To begin, we wrote this book for applied behavior analytic practitioners and researchers¹. The field of applied behavior analysis (ABA) is newer to the inclusion of certain statistical practices in the published literature. Many traditional statistics books begin with statistical theory and attempt to sprinkle in relevant examples along the way. Rather than approaching each topic through the lens of statistical theory, we attempt to approach each topic through a practical lens by asking: Why conduct these analyses in the first place? Under what conditions might this way of thinking about our data be useful? And, under what conditions might this way of thinking about our data add little?

This book is also different in that we center the book around practical decisions that behavior analysts make in their work. Sometimes, statistics books try to force research methodologies and clinical questions into a sequence of statistical theory. For example, the book may start with an introduction to probability theory, use the classic black and white balls pulled from an urn example, and then try to make a parallel with examples in healthcare or business that fit into that a similar framework. Anyone who has experienced the disgruntled statistician, lamenting the fact you didn’t come to them first and lecturing you about how you should design your experiment around the eventual statistical tests is likely familiar with this way of thinking: Use statistical theory to guide what you do.

But that’s rarely how real-world decisions are made. No knock to any strategic planning dynamites out there but, in our experience, many applied decisions often arise in the absence of a neat, tightly controlled experimental arrangement. Often, we have a question about a behavior-environment relationship or some challenge we are trying to solve; we have some set of related data that varies in its usefulness; and we need to come to an informed conclusion using those data that allows us to make the best decision we can. So, throughout this book, we attempt to approach each topic framed around the questions with which the topic is likely to coincide.

Lastly, behavior analysts primarily collect and use a very specific type of datasets (i.e., within-subject time series datasets). As you’ll see in the following chapters, these types of datasets have unique characteristics that violate some of the assumptions we might often make about behavioral processes. Once pointed out, a host of interesting questions arise for behavior analysts that might be hard to shake. Heavy use of within-subject time series datasets is somewhat unique among health professionals² and so we dive into it out of pure necessity and to raise some questions the field of ABA may want to (re)consider. But, before we can get there, we’ll follow the advice of the King when the White Rabbit asks him where to begin: "Begin at the beginning and go on till you come to the end." So to the beginning we go.

Defining a statistic

Statistics have a historically bad reputation in behavior analysis. For example, Skinner (1938) notoriously eschewed statistical analysis of behavior-environment relations in favor of experimental approaches. The basic idea was that few, if any, things in the universe remain constant and unchanged. As behavior analysts, we often observe variability in behavior across time, contexts, and people. As such, Skinner argued scientists should use experimentation to understand the functional determinants of variability rather than mathematically controlling for variability through statistical techniques (Skinner, 1956).

That early skepticism of statistical analysis has carried through the field to the present. The resulting dogma around statistics being a dirty word has, thus, influenced many behavior analysts and the training they receive during their education and practicum experiences. As a result, behavior analysts who use the word statistic within their published manuscript or in conversation with colleagues can sometimes be treated as less than pure. But, as noted by many over the last several decades (e.g., Branch, 2014; Ioannidis, 2005; Young, 2018), the logical challenge to which Skinner spoke occurs with a specific set of procedures called null hypothesis significance testing (NHST). We will talk in depth about this later on. But what’s important to note is that NHST comprises one small branch of the large field that is statistics. Ironically, even professional statisticians have decried their overuse in many areas of science (e.g., Wasserstein & Lazar, 2016). And, so what is statistics?

Statistics is a branch of mathematics whose topic is the collection, analysis, interpretation, and presentation of aggregate quantitative data (Merriam-Webster, 2021). Hey—wait a second! Behavior analysts collect, analyze, interpret, and present aggregate quantitative data all the time. Does this mean that behavior analysts regularly use statistics? Yes, they do. In fact, behavior analysts use statistics so frequently that we wrote a book about it to provide guidance for those with less formal training (and because it’s fun to talk about).

Just about every graph used in practice or displayed in published behavior analytic journals and books includes statistics. For example, the percentage of responses that are correct is a single number that describes the aggregate of correct responses divided by the aggregate of opportunities available to respond. As a second example, response rate does not provide you with information about any single response that occurred during a session. Rather, response rate aggregates all the occurrences of a specific response topography, or set of topographies, into a single number where we control for time. Even the cumulative record shows the aggregated count of responding as a function of time. Percentage of responses that are correct, response rates, and the cumulative number of responses within some time period are—by definition—statistics.

The benefits of numbers

So why use statistics? It turns out that the everyday languages used by humans are comparatively imprecise. For example, consider the daily difficulty you would encounter if numbers were not used to describe the weather outside, the speed with which your car travels, or the time of day. Life would likely look very different than it does now. The weatherperson’s description of temperature would likely be relative to their experience and may not help us dress comfortably. Safely slowing down for blind curves and for pedestrians in school zones might be difficult without numbers to guide the ideal speed. And, scheduling meetings would be very tricky for those who work with people spanning time zones or when weather conditions make the position of the Sun ambiguous. In each instance, statistics provide a precise and useful aggregate description of environmental events through the use of numbers.

The utility of conveying more information, more precisely, and using less words is particularly useful to scientists (e.g., Dallery & Soto, 2013). Modern science leverages a theoretically simple and intuitive set of procedures. Identify something in the universe (independent variable [IV]) that you think influences something else (dependent variable [DV]). Systematically present, remove, or vary that IV while trying to hold everything else constant. Then, use tools of some kind to observe and measure the degree to which the IV is present, the degree to which the DV is present, and how well you’ve held everything else constant. Numbers and statistics give us the means to capture these observations in a few lines, paragraphs, or pages rather than requiring our clinical or lab notebook to explode into hundreds or thousands of pages. And, because the verbal stimuli referred to as numbers have a similar or identical basic function for people the world over, it makes it very easy to communicate those observations accurately and precisely.

Behavior analysts benefit from using numbers and statistics, too. (After all we, too, are scientists.) Behavior analysts’ subject matter expertise is the behavior of the individual, and we analyze how patterns of changes in the environment correspond with changes in behavior. By conducting analyses using numbers, we can describe, predict, and control the behavior of an individual (e.g., Catania, 1960; Cooper et al., 2020)—quite the challenging task. Behavior, by definition, is always occurring and always changing. The environment includes dozens, hundreds, or maybe thousands of environmental events that also are always occurring and always changing. Numbers and statistics help behavior analysts more succinctly describe and communicate the occurrence and degree of behavior and environmental stimuli, how behavior-environment occurrences covary, and what behavior we predict we will observe if we change the environment in a particular way.

The following brief thought exercise may help demonstrate how the benefits of numbers and statistics extend to the professional practice of behavior analysis. To start, think of an intervention you recently implemented with a client or an experimental preparation you recently conducted. On the lines below, describe of the effects of that intervention or your experimental preparation using only descriptions of moment-by-moment occurrences (or nonoccurrences) of the behavior, the programmed (or unprogrammed), antecedent and consequent stimuli, and without using numbers:

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

Compare what you wrote with the following, Aggression decreased from ten responses per minute to one response per month by the end of the intervention. We obviously have no clue what you wrote down. But we’re (err, well, at least David is) willing to wager a drink at the next behavior analytic conference that the 18-word phrase we used is more precise and uses fewer words. Statistics are the culprit. Statistics allow us to easily aggregate information from many observations while maintaining some degree of precision of those observations. In short, statistics typically allow us to convey more information using less words.

Models and model building

Another important idea to sciences, generally, and the use of statistics, specifically, is the notion of models. A model can be defined as a miniature representation of something (Merriam-Webster, 2022). Said differently, models describe relationships between variables. Similar to statistics, we use models regularly even though we may not use the word model in our everyday behavior analytic vernacular. For example, the fundamental unit of analysis in behavior analysis, the three-term contingency, is a model (Moxley, 1996; Skinner, 1938, pp. 178–179). Models help researchers specify specific broader patterns in our observations so that they can be tested more directly and improved upon.

The three-term contingency is one way that behavior analysts describe the relationships between events that occur before a behavior (antecedents), the behavior that we’re particularly interested in, and the changes in the environment that follow a behavior (consequences). But it’s important to remember that the three-term contingency is not a physical thing with a physical existence in the universe. The three-term contingency is a miniature verbal representation of the totality of the environment surrounding behavior unfolding in time. Stated differently and more eloquently by the physicist Richard Feynman, If our small minds, for some convenience, divide this…universe into parts…remember that nature does not know it! (Feynman, 1965, para. 34). Other examples of models in behavior analysis include the four-term contingency (e.g., Michael, 1982), the discounting equation (e.g., Mazur, 1986; Rachlin, 2006), and the generalized matching equation (e.g., Baum, 1974; Baum & Rachlin, 1969).

So why build models? The short story seems to be about efficiency and utility. Rarely are scientists interested in simply describing the things they observe. Often, scientists want to know why something happens. As many behavior analysts are likely familiar with, understanding why something occurs requires repeated observation and experimentation with the variables relevant to our phenomenon of interest. But the universe is large and vast and we are unable to observe and measure everything. Models—built up over time through experimentation and scientific communication—help the model user to focus their efforts on measuring and manipulating only the IVs that actually matter (efficiency). And, these miniature verbal descriptions of the universe only survive to the extent that they effectively help the model user understand, predict, and control the phenomenon they’re interested in (utility).

Models can be described in various categorical ways. Fig. 1.1 shows these different ways to think about models and how different models from the published behavior analytic literature fit this schematic. One way to categorize models is based on the function of the modeler’s behavior (x-axis of Fig. 1.1). Here, we can use the commonly espoused continuum of scientific understanding: description, prediction, and control (e.g., Cooper et al., 2020; Skinner, 1938). For behavior analysts, descriptive models serve the function of describing a pattern of behavior-environment relationships as precisely as possible. As we move away from descriptive models, we start to get toward models that provide causal explanations of behavior-environment relationships. Here, we start to play with models that allow us to predict whether behavior will occur and to what degree (i.e., predictive models). And, arguably, the most useful causal models are those that allow us to prescribe courses of action for the model user (i.e., prescriptive or decision models).

Figure 1.1 Types of models in behavior science.

A second way to categorize models is based on the topography of the modeler’s behavior (y-axis of Fig. 1.1). Here is where the most variability might be observed and the most opportunity for future creativity might lie. For example, we can simply use written or spoken words to describe a likely causal relationship between environmental variables and behavior (e.g., the three-term contingency). We might visually or graphically display information to describe the relationship between the environment and behavior (e.g., reversal design plot contrasting baseline with intervention). We may use mathematical expressions to describe the relationship between the environment and behavior (e.g., matching law; Baum, 1974; McDowell, 2005). Or, as a final example, we might even use metaphorical language to highlight the relationship between the environment and behavior (e.g., response strength; Skinner, 1938, 1953, 1974). Note here that the topography is not relevant per se. All of these are instances of verbal behavior (Cox, 2019; Marr, 2015). And, as we know about verbal behavior (e.g., Skinner, 1957), topography is less important than the function that response serves for the speaker and listener (e.g., describe, predict, control).

The role of statistics becomes clearer when we combine the function and topography of the modeler’s behavior. Generally speaking, the use of numbers and quantitative relations between the environment and behavior allow for greater precision in our descriptions, predictions, and control of behavior (e.g., Dallery & Soto, 2013). Statistics are one category of verbal behavior that behavior analysts use to precisely describe and predict interactions between the environment and behavior. If this all still sounds a bit vague, no worries—later in this chapter we provide examples of statistics in the wild of which behavior analysts are likely familiar. But, before we get there, it’ll likely help to talk about what statistics are