Inquiry, Logic, and International Politics

By Benjamin A. Most and Harvey Starr

Through the use of logic, simulation, and empirical data, Benjamin A. Most and Harvey Starr develop and demonstrate a nuanced and more appropriate conceptualization of explanation in international relations and foreign policy in Inquiry, Logic, and International Politics. They demonstrate that a concern with the logical underpinnings of research raises a series of theoretical, conceptual, and epistemological issues that must be addressed if theory and research design are to meet the challenges of cumulation in the study of international relations (or any area of social science). The authors argue for understanding the critical, yet subtle, interplay of the elements with a research triad composed of theory, logic, and method.

• Language: English
• Publisher: University of South Carolina Press
• Release date: Sep 29, 2015
• ISBN: 9781611175936

Benjamin A. Most

Benjamin A. Most was an associate professor of political science at the University of Iowa at the time of his death in 1986. After receiving his doctorate from Indiana University, he taught at Brown University before moving to Iowa. Most distinguished himself as an expert in the study of war and international conflict, methodology, and research design and published in the area of comparative public policy, focusing primarily on Latin America. His work was presented in numerous articles and chapters appearing in the leading journals of the field.

Book preview

Inquiry, Logic, and

International Politics

Studies in International Relations

Charles W. Kegley, Jr., and Donald J. Puchala,

Series Editors

Marvin S. Soroos

Beyond Sovereignty: The Challenge of Global Policy

Manus I. Midlarsky

The Disintegration of Political Systems:

War and Revolution in Comparative Perspective

Lloyd Jensen

Bargaining for National Security:

The Postwar Disarmament Negotiations

Yale H. Ferguson and Richard W. Mansbach

The Elusive Quest:

Theory and International Politics

William R. Thompson

On Global War:

Historical-Structural Approaches to World Politics

Inquiry, Logic, and International Politics

BENJAMIN A. MOST

and HARVEY STARR

WITH A NEW PREFACE BY

HARVEY STARR

© 2015 University of South Carolina

Published by the University of South Carolina Press

Columbia, South Carolina 29208

www.sc.edu/uscpress

24 23 22 21 20 19 18 17 16 15

10 9 8 7 6 5 4 3 2 1

Library of Congress Cataloging-in-Publication Data

can be found at http://catalog.loc.gov/

ISBN 978-1-61117-592-9 (paperback)

ISBN 978-1-61117-593-6 (ebook)

"Time is too valuable, life too short,

to waste my time on dead ends."

Benjamin A. Most (1986:14)

To the memory of BEN MOST

student and teacher, colleague and friend

CONTENTS

List of Illustrations

Preface to the 2015 Edition

Acknowledgments

1. INTRODUCTION: Cumulation, Theory, and the Logic of Inquiry

2 OPPORTUNITY AND WILLINGNESS: A Pre-Theoretic Framework

3. BASIC LOGIC AND RESEARCH DESIGN: Conceptualization, Case Selection, and the Form of Relationships

4. CONCEPTUALIZING WAR: Attributes and Process

Appendix to Chapter 4

5. FOREIGN POLICY SUBSTITUTABILITY AND NICE LAWS: Integrating Process and Theory

Appendixes to Chapter 5

6. THE LOGIC OF INTERNATIONAL STRUCTURE: Power, War, and Micro-Macro Linkages

Appendix to Chapter 6

7. CONCLUSION: Closure, Cumulation, and International Relations Theory

Notes

References

Index

ILLUSTRATIONS

FIGURES

TABLES

PREFACE TO THE 2015 EDITION

Inquiry, Logic, and International

Politics Twenty-five Years On

THE EXPANDED COVERAGE OF

A CRITICAL LOGIC

Harvey Starr

The year 2014 marks the twenty-fifth anniversary of the publication of Most and Starr’s Inquiry, Logic, and International Politics (hereafter noted as ILIP). In this book we introduced a set of elements that were claimed to be useful—indeed critical in several senses—for good model building, research design, and the assessment of the validity of systematic empirical inquiry. While explicit in our thinking about how to do good research that would more efficiently and profitably lead to better cumulation in the research of international relations scholars, we were less explicit in framing the book as a truly critical commentary on the logic of inquiry and the research practices of many IR scholars. In retrospect I see ILIP as a significant piece of critical analysis whose impact can be traced in a number of ways. Because it is still read, assigned in classes, and cited in scholarly work, and because some of the ideas it introduced are now so widespread that they are often used without citation (such as substitutability or opportunity and willingness), recognition of a quarter-century in print is indeed the appropriate time to step back and assess the book’s contributions and impact. I do that briefly here, as well as introduce and highlight some of the important aspects of the book to follow.

In Inquiry, Logic, and International Politics Ben Most and I address a set of impediments to cumulation, which include a self-critical appraisal of the failure of quantitative scholars to match theory and logic to research design and method (see also Starr 2005). We identify these three elements—theory, logic, and method—as the components of the research triad, which serve as the core of our presentation (see chapter 1). We argue that scholars need to recognize the existence of a research triad . . . and that each leg of this triad is critical for advancing our knowledge of international phenomena (ILIP, 2). Scholars are likened to jugglers as each element of the triad needs to be held in the air at the same time in a complex set of interrelationships, indicating that for the juggler to be successful, all of the balls (elements of the triad) must be kept going simultaneously (ILIP, 10). Because a number of the issues we raise in the book have been continued by such scholars as Gary Goertz, Bear Braumoeller, Claudio Cioffi-Revilla, Phil Schrodt, and Cliff Morgan, among others, I can do no better here than to continue the call for attention to the research triad and the promotion of creative and rigorous research design—no matter what subfield is under investigation, what specific theoretical framework is used to guide the research, or what specific methodologies are used to evaluate and compare theoretical frameworks.

SUMMARIZING ILIP:

CUMULATION/PROGRESS AND THEORY

Understanding the extent and nature of and the reasons for the cumulation of knowledge in a specific discipline or area was a central motivating factor for ILIP, in which we argue that theory must be central to research and propel the research process. While ILIP points to a set of ongoing feedback loops among the elements of the triad, the driving element has always been theory. Thus theory must be at the forefront of our endeavors, whatever the subfield or methodology employed (clearly, I do not think this is unique to ILIP; see also Starr 2002). The centrality of theory in the research enterprise is also key to Zinnes’s (1976) notion of integrative cumulation or Bueno de Mesquita’s (for example, 1989 or 1985) focus on progress in a Lakatosian sense. The obvious position that students of research design must support is that the methods selected by any researcher must be appropriate to the questions being investigated. Those questions, in turn, must be theory-driven. Thus any researcher must begin with well-grounded and well-specified theory.¹ Therefore, ILIP stresses in chapter 1 the dynamic feedback loops that exist among theory, research design, and findings, as each informs and modifies the other. Theory thus not only affects the design and the research product but also is itself continually modified and updated by the research process and the results of that process, in a dynamic combination of induction and deduction. As the research process unfolds and theory is modified, the researcher may be led to investigate additional phenomena originally omitted from the study.

This view means that theory is central to the actual enterprise of empirical research. Given the current state of international relations theory at some U.S. institutions as well as abroad, it is important that the basic issues concerning methodology (and epistemology) that link theory, research design, and the actual enterprise of research are discussed and returned to the forefront of the scholarly enterprise. This must be argued explicitly because for some academics, the idea of theory has been and too often continues to be used to mean only broader philosophical world-views or ideologies. As such, theory has been divorced from research by those who use the term in this way. Social scientists need to be reminded that theory’s central contribution rests in the way in which it informs, shapes, and is in turn shaped by empirical research—that theory is a tool in the study of the world of politics. The theory-research loop is crucial here. This loop is irrelevant when theory is conceived of as ideology, where the answers are known, and thus research is irrelevant. Only when theory is seen as central to research, and not used as ideology, can it be the basis for integrative cumulation. While this point is not included in ILIP, concern with theory should be directed at two basic questions about theory that I have always stressed. Both are extremely important but are not necessarily related to each other: first, where does theory come from? and second, how are we to evaluate and compare theories?²

This theory-research feedback loop is a prominent component of ILIP. My work in that book, and with other colleagues since then, has been concerned with a set of key issues in the logic of inquiry that directly addressed this relationship. These issues also helped to comprise a truly critical logic that challenged, and still challenges, researchers to examine the theoretical bases of their research. This challenge involved clarifying the logical consistency of the relationship between theory and research design. It also involved researchers considering the logical structure of the components of their research designs. In sum, the issues found in ILIP that comprise the critical logic include the following: the form of the relationship under study; matching the logic of theory/models to the logic of research design; development of process models; broader versions of the agent-structure problem; identifying the appropriate units of analysis; case selection; and such vital linking and synthesizing concepts as substitutability and nice laws. All of these issues have become part of broader dialogues, debates, and progressive Lakatos-like developments in the study of international politics. To discuss all of these items would be to go beyond the scope of this preface. However, in regard to the impact of ILIP, discussions of (1) the form of the relationship and (2) the concepts of substitutability and nice laws—especially as each is related to necessary and sufficient relationships—will be developed more fully below.

Perhaps most importantly for both cumulation and synthesis, the critical logic—along with its subcomponents and its attendant issues—is not limited to any specific area of international politics or political science (see, for example, the title of Cioffi-Revilla and Starr 1995). Nor is it limited to any specific methodological tradition. In this sense ILIP preceded such important work as King, Keohane, and Verba’s (1994) unified approach to inference or Gerring’s (2005) unified framework for dealing with causation. ILIP’s critical logic was used to develop principles of valid and useful research design that could be applied to any empirical study, to any study that involved observations used to investigate the implications of theoretical statements that in some way, shape, or form touched the world (James Rosenau’s potential observability, 1990, 33). For example, since 1989 I (and collaborators) have worked to elaborate and develop more fully the analyses and arguments that Most and I present in ILIP. I have, for example, demonstrated the ways in which ILIP addresses Lave and March’s (1975) three rules of thumb for creative model building: (1) think ‘process’; (2) develop interesting implications; and (3) look for generality (see Starr 1997). Thus, applying the critical logic to research design has been absolutely central to addressing the second question about theory noted above: how can we evaluate and compare theories?

While I was dealing with critiques of systematic empirical analysis within the framework of standard research design or statistical applications, it became clear to me that our critical logic is also able to deal with most of the critiques directed at systematic empirical work from outside the scientific-method-based tradition of inquiry. That is, most of the critiques raised by those critical of positivism (always an ambiguously used concept) can be subsumed under issues raised in the critical logic of ILIP.³ Using this critical logic, the scholar need not stand outside the tradition of the scientific method to raise and deal with such issues as choice posing as truth or reality as a social construction or the broad issue of relativism (see Vasquez 1995).⁴ To repeat, the key point here is that the critical logic and its components apply across methodologies and to all aspects of research design—as long as one’s true enterprise is to investigate some aspect of the empirical world.

THE IMPACT OF ILIP: A FIRST CUT

In chapter 1 of ILIP we refer to a long tradition of introspection and stock-taking in the scientific study of international politics (to use Zinnes’s term). This is consistent with Zinnes’s concern with cumulation, which was a prime motivation for the Most and Starr project and subsequent book-length treatment. I see a brief review of the impact that this book has had in the study of international politics as consistent with this tradition.

Some feel for the impact of ILIP can be given in a few broad strokes. The book has been widely cited across a range of studies and topics in international relations, as well as in comments on a specific project’s research design and/or methodology. Note also that chapter 2 of ILIP is a revised version of Starr’s original 1978 article that introduced the opportunity and willingness framework. For a large number of international relations scholars, ILIP was their introduction to opportunity and willingness. As one form of an agent-structure model, the opportunity and willingness framework was well suited for helping ILIP discuss the running use of the study of war as a substantive example of the logic of inquiry arguments that are presented (with chapter 4 specifically directed at war). Such an agent-structure framework was also useful in providing a range of research questions that went beyond those depending on standard linear correlation and regression-type analyses.

The use of the opportunity and willingness framework was also crucial to introducing the concept of substitutability (in chapter 5), which has been one of Most and Starr’s (and ILIP’s) most important contributions to IR theory and research, and one that has received substantial attention. Similarly, ILIP’s argument that greater attention must be paid to the form of the relationship—which had to reflect the theoretical model being used and had to be logically connected to the research design—is most clearly reflected in chapter 3’s discussion of necessity and necessary causal relationships. This chapter has been reprinted in the Goertz and Starr—edited volume Necessary Conditions: Theory, Methodology and Applications (2003b), which has been used extensively in comparative politics graduate courses and in scholarly research associated with the American Political Science Association’s Section on Qualitative and Multi-Method Research. Discussions of necessary conditions as well as references to Goertz and Starr have been frequent in the Qualitative and Multi-Methods Newsletter and at the annual summer Institute for Qualitative and Multi-Method Research.

As a result, and in part reflecting these aspects of ILIP, I have located around eight hundred citations to ILIP (utilizing Google Scholar Citations, looking at ILIP as well as the previously published articles and presented conference papers that went into the final book manuscript for ILIP). In addition, for a book that was assigned almost solely in graduate seminars, its sales of almost twenty-four hundred new copies can be seen as another useful indicator of its impact. It is not quite clear how many courses have adopted ILIP as required or recommended for purchase, but across the years I have seen a number of Web-available course syllabi containing the book or various chapters that have been assigned in both U.S. and international institutions and in methods courses across the subfields of political science⁵ and courses in international relations.⁶

THE IMPACT OF ILIP: FORM OF THE RELATIONSHIP

While the primacy of theory is a central theme in ILIP, Most and I direct the reader to look at a set of related issues. Perhaps first and foremost is understanding the logical form of the relationship generated by a theory (which includes the logical causal structure such as necessity and sufficiency), which affects most of the other research design aspects as well—for example, should we expect linear or curvilinear results, and what is the shape of the relationship between independent and dependent variables? For example, using logic, we indicate (in chapter 1 and especially in chapter 3) how scholars had set up research hypotheses based on sufficient relationships but then went on to test them with designs that could deal only with necessary relationships, and vice versa. In brief (and to be elaborated below), our initial questioning of how researchers have tried to test or evaluate necessary relationships has been taken up more recently in work by Goertz and collaborators (for example, Braumoeller and Goertz 2003; see also Goertz and Starr 2003a, 2003b; Goertz 2003, 2005; Mahoney and Goertz 2006; Goertz, Hak, and Dul 2013; Goertz and Mahoney 2012).⁷ Deriving in part from ILIP, one central issue raised in several chapters of Goertz and Starr (2003b) is that the use of standard statistical techniques will not capture the effects of necessity.

Regarding the form of the relationship, I return here to basics and build up from there. Social science, as all science, is a continuous quest for an explanation and understanding of the world around us. Most and I thought that this was done through the research triad of theory, logic, and research design, where logic and theory are central to both hypotheses and results. Perhaps the central element of explanation and understanding is causation. Social scientists are concerned with causation as applied both to individual events or cases and to classes or groups of events. The causal relationship may take many forms. Two of the most prominent, important, and commonly used forms of the causal relationship involve necessary and/or sufficient relationships. As stressed in ILIP, analyses must be concerned with the form of the relationship. David Hume’s classic definition of cause involves the constant conjunction of an object followed by another, where all objects similar to the first are followed by objects similar to the second. Causation is seen as including three elements, the first being the existence of correlation or association between two factors or variables (the constant conjunction). That is, changes or attributes in one factor are associated with changes or attributes in another. Correlation is one possible form of any specific relationship. Correlation itself may take several forms, such as linear or curvilinear. The second element of cause involves the temporal dimension: the proposed causal factor must take place before the effect or the phenomenon to be explained. Cause must precede effect. The third element is the most rigorous and difficult: the elimination of other explanatory factors beyond those proposed (the central idea of control).

Necessity and sufficiency are themselves different forms of the causal relationship, representing different forms of constant conjunction. Scholars have argued that research designs (the nature of the theoretical logic and research hypotheses being employed) built around these different forms of causation will affect—and be affected by—the nature and types of cases selected, the controls employed for dealing with possible other (explanatory) factors, and the methods used for evaluating the theory and proposed research hypotheses. All three elements of the research triad—theory, logic, and research design—are thus affected by the form of the relationship, especially looking at necessary relationships in distinction from sufficient relationships or even simple correlational relationships. The differences between how necessary conditions and sufficient conditions are treated are additionally important in regard to the research designs needed to investigate inference in the small-n studies that characterize much of comparative politics (and qualitative analyses more generally).

While ILIP has been one of the first works to stress the logical forms of relationships and has had a major impact on crafting a valid and useful logic of research design, the subsequent work by Goertz and collaborators also demonstrates the impact of ILIP on contemporary research. Goertz (2003) speaks of two causal universes with two different views of causality. The first takes the form of "the greater X the greater or more likely Y. This view of sufficiency, he argues, easily becomes a quantitative, probabilistic, correlational view of causality (Goertz 2003, 48). The second takes the form of necessary conditions: a necessary condition by definition is something without which the effect cannot occur"; that is, if and only if X then Y.⁸ The form of the relationship leads to different types of hypotheses, and that ‘necessary condition’ does not equal ‘correlation’ (Goertz 2003, 48). Using the economic requisites of democracy as an example, Goertz (2003, 49) provides these two contrasting hypotheses:

Necessary Condition Hypothesis: A minimum level of economic development is necessary for democracy.

Correlational Hypothesis: The higher the level of economic development, the more likely a country is a democracy.

While both concern a relationship between economic development and democracy, they are not asking the same question, or assuming the same type of relationship. Part of Goertz’s argument is that the former hypothesis is directed more toward the causes of specific events, while the latter is more concerned with causes of classes of events. A necessary condition approach can make strong predictions regarding the question "what does the absence of X say about the likelihood of Y occurring? (Goertz 2003, 53). The correlational hypothesis approach, in contrast, is interested in the mean causal effect" (Goertz 2003, 55). Chapter 3 of ILIP uses a set theory approach to analyze necessary and sufficient conditions (one of five approaches set out in Goertz and Starr 2003a), providing a number of jumping-off points for Goertz’s subsequent research project on necessary conditions. This includes his critique of King, Keohane, and Verba (1994) for discussing research design almost entirely in terms of probabilistic linear correlation hypotheses of sufficiency.⁹

Subsequent to ILIP and arguing many of these same types of themes is Schrodt’s critique of the linear frequentist orthodoxy as exemplified by King, Keohane, and Verba (see Schrodt 2006). I quote at length from Schrodt (2006, 336):

KKV establishes as the sole legitimate form of social science a set of rather idiosyncratic and at times downright counterintuitive frequentist statistical methodologies that came together—following bitter but now largely forgotten philosophical debates by Fisher, Neyman, Pearson, Savage, Wald, and others in the first half of the 20th century (Gill 1999)—to solve problems quite distant from those encountered by most political scientists. . . . In political science, the canonization of frequentism has been coupled with a statistical monoculture that relies almost exclusively on models involving linear combinations of independent variables. . . . The linear frequentist orthodoxy elevates to a philosophical imperative a method of inference that was originally designed for the analysis of very large, homogeneous populations that could be subjected to repeated tests and had only a small number of characteristics of interest. . . . But outside a small number of issues, most of the problems of interest to most political analysts do not fit.

Thus, Schrodt notes that pathologies quickly emerge when linear models are applied in many political analyses (2006, 337). In full Schrodtlike style, one subsection of a 2010 paper exclaims, Enough Already with the Linear Models!¹⁰ Goertz (2006, 224) notes simply, There is widespread feeling cutting across both qualitative and quantitative methods that standard, additive-linear-in-variables statistical methods often do a poor job.

Echoing the call in ILIP and reflecting many of the concerns of Goertz about the form of the relationship, Schrodt (2006, 337) explicitly notes, The work by Goertz, Starr, and others (Most and Starr 1989; Braumoeller 2003; Goertz and Starr 2003) on necessary and sufficient conditions has provided insights into the systematic study of logical regularities, which often as not are more important to theories of political behavior than the coefficients of a least squares fit. As noted in the work of Braumoeller (for example, 2006), while we are trained to model changes in central tendency, necessary conditions imply a complex range of patterns that have nothing to do with central tendency. I think the lesson here is that the empirical scholar needs to beware one-size-fits-all, standard, linear frequentist applications. Following ILIP, the scholar needs carefully to check the logical form of the argument and its causal claims and to make sure that it is being captured in one’s designs and methods.¹¹

This brief overview of the work of Goertz and the use of the observations of methodologists such as Schrodt and Braumoeller is meant only to exemplify the critical issue of the form of the relationship raised in ILIP. The message that Most and I wanted to send was that the research triad, with the explicit use of logic to link theory and research design, alerted us to problems in hypothesis formation, case selection, and methodological tools that threatened the validity and utility of our research; these were problems that often could not be solved through standard statistical practice but could in fact be made much worse.

We also wanted to exhort researchers to look carefully at their theory and the research questions generated and to apply only those methods of analysis appropriate to the broader issues/questions that concerned them as researchers. To do so meant and still means that we should be expansive and inclusive in terms of the items we include in our methodological toolboxes. In turn this means that the methods we choose will reflect the theory used, the aims of the research (from description and association to causation, explanation, and/or prediction), and the nature of the causality involved, rather than tailor our theory and the aims of research to the methods we want to use. Our methods should not be the proverbial hammer in search of a nail, nor should we act as the child with the hammer who discovers that everything needs to be pounded.

THE IMPACT OF ILIP: DEALING WITH SUBSTITUTABILITY

As noted, chapter 2 of ILIP presents a revised discussion of opportunity and willingness, one type of agent-structure model that describes the set of relationships between an entity and the environment that surrounds it (based on the work of Harold and Margaret Sprout, for example, 1969). Although it was elaborated on in chapter 2, I note briefly here that the concept of opportunity was developed to represent the possibilities available to any entity within any environment, that is, the total set of environmental constraints and possibilities. While opportunity represents macro-level (environmental and structural) factors, willingness represents the choice processes that occur at the micro level, that is, the selection of some behavioral option from a range of alternatives. Thus both structure-environment and choice-decision processes are required. Therefore opportunity and willingness are concerned with the relationships that nest decision makers within their surrounding environments, and as demonstrated in ILIP, opportunity and willingness are jointly necessary for any action to occur.

As discussed in chapter 5, the concept of