Log-Linear Modeling: Concepts, Interpretation, and Application

By Alexander von Eye and Eun-Young Mun

An easily accessible introduction to log-linear modeling for non-statisticians

Highlighting advances that have lent to the topic's distinct, coherent methodology over the past decade, Log-Linear Modeling: Concepts, Interpretation, and Application provides an essential, introductory treatment of the subject, featuring many new and advanced log-linear methods, models, and applications.

The book begins with basic coverage of categorical data, and goes on to describe the basics of hierarchical log-linear models as well as decomposing effects in cross-classifications and goodness-of-fit tests. Additional topics include:

• The generalized linear model (GLM) along with popular methods of coding such as effect coding and dummy coding
• Parameter interpretation and how to ensure that the parameters reflect the hypotheses being studied
• Symmetry, rater agreement, homogeneity of association, logistic regression, and reduced designs models

Throughout the book, real-world data illustrate the application of models and understanding of the related results. In addition, each chapter utilizes R, SYSTAT®, and §¤EM software, providing readers with an understanding of these programs in the context of hierarchical log-linear modeling.

Log-Linear Modeling is an excellent book for courses on categorical data analysis at the upper-undergraduate and graduate levels. It also serves as an excellent reference for applied researchers in virtually any area of study, from medicine and statistics to the social sciences, who analyze empirical data in their everyday work.

• Language: English
• Publisher: Wiley
• Release date: Aug 21, 2014
• ISBN: 9781118391761

Book preview

CHAPTER 1

BASICS OF HIERARCHICAL LOG-LINEAR MODELS

In this chapter, we pursue four goals. First, we introduce basic ideas and issues concerning hierarchical log-linear modeling. This introduction leads to an understanding of the situation a researcher faces when dealing with categorical variables, and to an appreciation of the data material to be processed. Second, the basic assumptions that need to be made for proper log-linear modeling are discussed. Third, we talk about effects in a table, various approaches to analyzing these effects, odds ratios, and first elements of log-linear modeling. Fourth, we end the chapter with a discussion of hierarchical log-linear models using the opensource environment R [183], SYSTAT [203], a standard, general purpose software package, ℓEM [218], a specialized program for categorical data analysis.

1.1 SCALING: WHICH VARIABLES ARE CONSIDERED CATEGORICAL?

Before deciding which variables can be considered categorical, we briefly review parts of the discussion on scale levels. This discussion has been going on since the mid-twentieth century, and there is still no conclusion. Log-linear models are typically estimated for categorical data. Therefore, a review of this discussion is of importance.

A scale results from measurement, that is, from assigning numbers to objects. Among the best known results of statistics are the scale levels that Stevens [201] proposed in 1946. Stevens proposed a hierarchy of four scale levels. To introduce this hierarchy, McCall [156] discussed three properties of scales. The hierarchy results from combining these properties.

The first property is that of magnitude. Scales that possess magnitude allow one to judge whether one object is greater than, less than, or equal to another object. For example, if a scale of schizophrenia assigns a score of 7 to the first patient and a score of 11 to the second, one can conclude that the second patient is a more severe case than the first only if this scale possesses magnitude.

The second property is that of equal intervals. Scales of equal intervals allow one to interpret the size of differences between scores. For example, if Person A has an IQ score of 120 and Person B has an IQ score of 100, there is a difference of 20 IQ points. Now, if Person C has an IQ score of 80, than the difference between A and B is the same as the difference between B and C.

The third property is that of the absolute zero point. Scales with absolute zero points allow one to indicate that nothing of a particular characteristic is observed. For example, if a car is measured to move with a speed of zero, the car is standing still.

Combining these three properties yields Stevens’s four scale levels. We present this hierarchy beginning at the bottom, where the least complex mathematical operations are possible. With each higher level, new operations become possible.

The bottom feeder of this hierarchy is the nominal scale level. This scale (some authors state that the nominal level does not even qualify as a scale) possesses none of the three properties discussed above. This scale is used to label and distinguish objects. For example, humans are labeled as females and males. In data analysis, every female is assigned the same number, and every male is assigned the same number. In more elaborate systems, species are labeled and differentiated, and so are diseases, theories, religions, types of chocolate, and political belief patterns.

At the nominal level, every individual with a particular label is considered identical to every other individual with the same label. All researchers can do at this level is ask whether two objects share the same label or characteristic. One can thus determine whether two objects are the same (=) or different (≠). Any other operation would require different scale properties.

The second level in the hierarchy is that of an ordinal scale. At this level, scales possess magnitude. However, they do not possess equal intervals or absolute zero points. For example, when the doctor asks the patient how she is feeling and she responds better than at my last visit, her response is given at the ordinal level. Another example involves rank-ordering faculty based on their contributions to the department. The operations possible at the ordinal scale level lead to statements as to whether one comparison object is greater than another (>), less than another (<), or equal to another (=). Clearly, the > and < are more differentiated ways of saying ≠. At the ordinal scale level, correlations are possible; averages, however, and variances cannot be calculated. Examples of ordinal scales include questionnaire scales that ask respondents to indicate the degree of their agreement on a scale from 1 to 7, grades in school, and the Olympic medal ranking in figure skating. In none of these cases, is the distance between scale points defined and interpretable.

Clearly, at the ordinal level, the doctor does not know how much better the patient feels. At the next higher scale level, the interval level, this increase can be quantified. The interval scale possesses magnitude and equal intervals, but no absolute zero point. As was indicated above, equal interval implies that the distance between two scale points can be determined, and that the distance units are the same over the entire range of admissible scores. At this scale level, means and variances can be calculated, and we can, thus, perform an analysis of variance, a factor analysis, and we can estimate a structural equation model. Sample scales include most psychometric tests, such as intelligence, aggression, and depression.

The top level in the hierarchy of measurement scales is the ratio scale. This scale possesses all three of the properties above: magnitude, equal intervals, and the absolute zero point. At this level, one can, in addition to the operations that are possible for the lower level scales, perform all other arithmetic operations, and one can use all tools of higher mathematics. For example, scores can be added to each other, multiplied by each other, and one can transform scores using logarithmic transformations.

While consistent in itself, Stevens’s classification has met with critical appraisal. In an article by Velleman and Wilkinson [214], we find, among other issues, five points of critique (see also Hand [95]):

1. Stevens’s classification represents an oversimplified and overpurified view of what measurements are like; such a view must not be allowed to dictate how data are analyzed.

2. Meaningfulness to the data analyst and his/her intentions should guide the selection of methods of analysis.

3. Statistical methods cannot be classified according to Stevens’s criteria.

4. Scale types are not precise categories; on the contrary, scale scores can have more than one characteristic, even at the same moment.

5. Not all characteristics of scale scores are reflected by Stevens’s system.

Indeed, in parts of the literature, Stevens’s classification is not used any more. An example can be found in Clogg [32]. The author discusses the following possible scales instead (p. 314):

continuous (quantitative)

restricted continuous (due to censoring or truncation)

categorical dichotomous

categorical nominal (multiple categories, but no ordering)

categorical ordinal (with levels as in a Likert scale)

categorical quantitative (levels are spaced with distances between scale points known in principle).

In addition, Clogg discusses mixed scale types, for example, scales that are partially ordered. Examples of such scales include categorical ordinal scales that, for instance, also include a don’t know response option.

What can we conclude from this discussion? Most important, categorical variables come in many different forms. One form is constituted by variables with naturally distinct categories. Consider, for example, the (numerical) categories of the variable Car Brand. These categories are distinct in the sense that there is no ordering, and scale values between those that label the categories are not meaningful. It makes no sense to say that a car is of the brand 1.7. Distinct categories can also be found in rank orders, but here, cases can be assigned to averaged ranks (thus violating the tenet that ranks cannot be averaged; see above). Even interval level or ratio scale scores can be categorical. For instance, clinical diagnoses are often based on symptom scores that can be at the interval level, but the classification as case versus not a case is categorical.

In this book, we consider variables categorical when the number of scale values is small. One virtue of the methods discussed here is that scale characteristics can be taken into account. For example, there are models for scale scores that are ranked (e.g., see Section 9.3.6), and for scale scores that reflect different distances from one another (e.g., see Chapter 13). Thus, researchers are very flexible in the options of taking scale characteristics into account. In the next section, we begin the introduction of technical aspects of the analysis of categorical data. We discuss cross-classifications of two and more variables.

1.2 CROSSING TWO OR MORE VARIABLES

Crossing categorical variables yields cross-classifications, also called contingency tables. These representations allow one to analyze the joint frequency distribution of two or more variables. To introduce cross-classifications, let us use the following notation. Observed cell frequencies are denoted with m, and estimated expected cell frequencies are denoted with . Later, we will use subscripts to indicate the exact location of a frequency in a cross-classification. In this section, we introduce two forms to display cross-classifications, the matrix form and the tabular form.

Table 1.1 Frequency Distribution of the Four-category Variable A

images/978-1-1181-4640-8_1_image_5_1_1.jpg

For single variables, the two forms are the same. Consider, for example, Variable A with the four categories a1, a2, a3, and a4. The frequency distribution for this variable is given in Table 1.1. This table presents, for each category ai, the observed frequency mi, where i indexes (or counts) the categories of Variable A, with i = 1, ..., 4. Equivalently, a vertical arrangement could have been used instead of the horizontal arrangement. An example of such an arrangement appears in Table 1.3. Completely crossing two categorical variables yields a representation in which one can see the number of times that a category of one variable was observed for each category of the other variable, and vice versa. Cross-classifications are thus comparable to completely crossed analysis of variance designs. Under certain circumstances, not all combinations of variable categories are possible. The resulting cross-classification will then be incomplete. This issue will be discussed later, in Section 9.3.4.

To give an example of a cross-classification of two variables, consider again the above Variable A, and Variable B, which has the two categories b1 and b2. Crossing the four categories of A with the two categories of B yields a 4 × 2 contingency table. This table is given in Table 1.2. As can be seen from the cell entries, we now need two subscripts to indicate the location of a frequency in a table. By convention, the first subscript indicates the row and the second indicates the column in which a frequency is located. For example, the observed frequency m32 is located in the third row and the second column of the table. The last column of the table displays the row marginals (also called sum or row total), that is, the sum of the frequencies in a row. The last row of the table displays the column marginals (also called sum or column total), that is, the sum of the frequencies in a column. The cell in the lower right corner of the table contains the sum of all frequencies, that is, the sample size, N.

The entries and the marginals (totals) of the two-dimensional (2D) array in Table 1.2 can be explained as follows. Entry mij is the frequency with which Category ai (row i) was observed jointly with Category bj (column j), with i = 1, ..., 4, and j = 1, 2. Summing over the second subscript, that is, over the columns, yields the row totals images/978-1-1181-4640-8_1_image_5_6.jpg , and summing over the first subscript, that is, over the rows, yields the column totals, images/978-1-1181-4640-8_1_image_5_7.jpg . Summing over both rows and columns yields the overall total, that is, the sample size, images/978-1-1181-4640-8_1_image_5_8.jpg . Both Tables 1.1 and 1.2 could have been transposed, that is, in the case of Table 1.2 presented as a 2 × 4 table instead of a 4 × 2, without change in meaning of the corresponding entries or marginals.

Table 1.2 Cross-classification of the Four-category Variable A and the Two-category Variable B

images/978-1-1181-4640-8_1_image_6_1_1.jpg

Table 1.3 2 × 2 × 3 Cross-classification of the Three Variables X, Y, and Z

images/978-1-1181-4640-8_1_image_6_3_1.jpg

Now, in many cases, researchers are interested in analyzing more than two variables. Cross-classifications of three or more variables can also be presented in matrix form, for example the staggered matrices in Table 1.4. One way of doing this for three variables is to present a two-variable table, that is, a 2D table for each category of the third variable. Accordingly, for four and more variables, 2D arrangements can be created for each combination of the third, fourth, and following variables. These tables can be hard to read. Therefore, many researchers prefer the so-called tabular representation of contingency tables. This representation contains, in its left column, the cell indices. In the next column, it contains the corresponding observed cell frequencies. In possibly following columns, the expected cell frequencies can be given, residuals, and the results of cell-wise evaluations or tests. Table 1.3 presents the tabular form of the 2 × 2 × 3 cross-classification of the three variables X, Y, and Z.

Table 1.4 3 (Concreteness) × 2 (Gender) × 2 (Correctness) × 2 (Example Use) Cross-classification for the 10th Statement

images/978-1-1181-4640-8_1_image_7_1_1.jpg

EXAMPLE 1.1

The following example uses data from a study on verbal understanding by von Eye, Jacobson, and Wills [237]. A sample of 178 adults (48 males) was presented with 10 proverb-like statements. The task involved statement interpretation. Here, we ask whether the interpretation of the 10th statement (1) was concretely worded, (2) used an example, (3) was gender-specific, and (4) was correct. The concreteness scale had three categories, with 1 = concrete, 2 = concrete and abstract, and 3 = abstract words used. The example used scale was coded as 1 = yes, and 2 = no. Gender was coded as 1 = female and 2 = male. Correctness was coded as 1 = right and 2 = wrong. Table 1.4 presents the 3 (Concreteness) × 2 (Gender) × 2 (Correctness) × 2 (Example Use) cross-classification in a staggered matrix form, for the 10th proverb (the proverb was Love is a collaborative work of art.).

Table 1.4 is arranged in a slightly different way than Table 1.3. It is arranged in panels. It is the goal of this book to introduce readers to methods of analysis of tables of the kind shown here.

The following are sample questions for the variables that span Table 1.4. These questions can be answered using the methods discussed in this book:

1. Is there an association between any two of the four variables Correct Interpretation, Concreteness, Example Use, and Gender? [Answer: yes; for instance, there is an association between Concreteness and Correctness]

2. Is there a 3-way association or interaction among any three of the four variables? [Answer: no]

3. Is Gender related to patterns of association between Example Use and Correctness of Interpretation? [Answer: no]

4. Is there a path that originates in Gender, goes to Example Use, and then to Concreteness? [Answer: no]

5. Is Example Use a mediator between Gender and Concreteness? [Answer: no]

6. Are there individual cells that contradict particular, simple models? [Answer: no]

7. What is the most parsimonious model that can be used to describe the data in Table 1.4? [Answer: one fitting and parsimonious model contains the four main effects and the three 2-way interactions [Gender, Correctness], [Concreteness, Correctness], and [Example Use, Concreteness].

There are many more questions that can be asked. This book will present methods of analysis for many of them. In the next section, we discuss these questions from a more general perspective.

1.3 GOODMAN’S THREE ELEMENTARY VIEWS OF LOG-LINEAR MODELING

Goodman [82] discusses three elementary views of log-linear modeling. In the context of 2D tables, the author states (p. 191) that "log-linear modeling can be used

1. to examine the joint distribution of two variables,

2. to assess the possible dependence of a response variable upon an explanatory or regressor variable, and

3. to study the association between two response variables."

When more than two variables are studied, these views carry over accordingly. In addition, these three views can be taken, as is customary, when an entire table is analyzed or, as it has recently been discussed (Hand & Vinciotti [96]), local models are considered (see also Havránek & Lienert [99]). Local models include only part of a table, exclude part of a table, or contain parameters that focus on parts of a table only.

When the joint distribution of variables is modeled, results are typically stated by describing the joint frequency distribution. For example, a cross-classification of two variables can be symmetric with respect to the main diagonal of the table (von Eye [220]; von Eye & Spiel [253]). Consider a square, that is, an I × I cross-classification with cell probabilities πij, with i, j = 1, ..., I. This cross-classification is axial symmetric if πij = πji. Other concepts of symmetry have been discussed (Bishop, Fienberg, & Holland [15]), and there are many forms of joint distributions.

When dependency relations are modeled, results are typically expressed in terms of conditional probabilities, odds ratios, or regression parameters from logit models. As is well-known and as is illustrated in this text, log-linear models can be used equivalently. Path models and mediator models can be estimated.

When association patterns are modeled, results are typically expressed in terms of associations or interactions that can involve two or more variables. To analyze associations and interactions, variables do not need to be classified as dependent and independent. All variables can have the same status.

In this text, we discuss methods for each of these three views. Before we delve into technical details, however, we briefly discuss, in the next section, assumptions that need to be made when analyzing cross-classifications (see Wickens [261]).

1.4 ASSUMPTIONS MADE FOR LOG-LINEAR MODELING

As will be discussed later (see Section 3.1), X²-tests can be used to appraise the correspondence of model and data. X²-statistics are asymptotic statistics that approximate the χ² distribution well if certain assumptions can be made. Three of the most important assumptions are that the cases in a table (1) are independent of each other, (2) have similar distributions, and (3) are numerous enough. We now discuss each of these assumptions.

The independence assumption, also called the assumption of probabilistic independence, is most important, and this assumption is made in many other contexts of statistical data analysis. It implies that no case carries more than random information about any other case. This assumption is usually reduced to the requirement that cells must be filled with responses from different cases. However, such a requirement is not always sufficient to guarantee independence. For example, if the results of political elections are predicted, the vote cast by Voter A must not determine the vote cast by Voter B. This, however, is rarely guaranteed. There are entire districts in which voters traditionally vote for a particular party. In these districts, the vote cast by one voter allows one to predict the votes cast by a large number of other voters. Accordingly, family members often vote for the same party, and friends often agree on voting for a particular candidate. This is legal and does not jeopardize the validity of elections or statistical analysis.

Clearly, if the same person goes into a table more than once, this assumption is violated. Therefore, repeated measures analysis is a different beast than the analysis of cross-sectional data. What is the damage that is done when cases fail to be independent? In general, bias will result. It is not always clear which parameter estimate will be biased. However, it is easy to demonstrate that, in certain cases, severe bias can result for both mean and variance. Consider the case where two candidates run for office. Let the true voter distribution, one week before the elections, be such that 50% tend to vote for Candidate A, and the other 50% for Candidate B. Now, a TV station that predicts the outcome of the elections asks 100 voters. For lack of independence, the sample contains 75% supporters of Candidate A.

Table 1.5 Example of Mean and Variance Bias in Polling Example

images/978-1-1181-4640-8_1_image_10_1_1.jpg

Table 1.5 shows how the mean and the variance of the true voter distribution and the one published by the TV station compare. It shows that, in this example, the mean is dramatically overestimated, and the variance is dramatically underestimated. In general, any parameter estimate can be affected by bias, and the direction of a bias is not always obvious.

The second assumption to be made when assessing goodness-of-fit using X²-based statistics concerns the distribution of scores. It is the assumption that the data were drawn from a homogeneous population. If this is the case, the parameters are the same for each individual. If, however, data come from mixed distributions, or individuals respond to different effects, the aggregate of the data can be problematic (for discussion and examples, see Loken [143]; Molenaar & Campbell [166]; von Eye & Bergman [228]; Walls & Schafer [259]). First, it is unclear which effects are reflected and which are not reflected in the data. Second, the aggregate of data from multiple populations could have the effect that the X² calculated from the data approximates the χ² only poorly.

Therefore, if the provenance of frequencies varies and is known, it needs to be made part of the model. For example, one can add a variable to a model that classifies cases based on the population of origin. Mantel–Haenszel statistics can then be used to compare associations across the populations (see Section 14.1). If, however, the provenance of data is unknown, but researchers suspect that the data may stem from populations that differ in parameters, methods of finite mixture distribution decomposition can be applied to separate the data from different populations (Erdfelder [53]; Everitt & Hand [56]; Leisch [133]).

The third assumption concerns the sample size. To obtain sufficient statistical power, the sample must be assumed to be sufficiently large. Two lines of arguments need to be pursued. One involves performing standard power analysis (Cohen [38]) to determine sample sizes before data are collected, or to determine empirical power when the sample is given. In the other, rules are proposed, for instance, the rules for 2D tables proposed by Wickens [261]; see also the discussion in von Eye [221]:

1. For models with df = 1, each of the ij should exceed 2.

2. For models with df > 1, images/978-1-1181-4640-8_1_image_11_7.jpg can be tolerated, for a few cells.

3. For large tables, up to 20% of the cells can have images/978-1-1181-4640-8_1_image_11_8.jpg less than 1.

4. The total sample size should be at least 4 times the number of cells in the table, and much larger when the marginal probabilities deviate from a uniform distribution.

It goes without saying that other sources propose different rules. For example, sample size requirements based on Cohen’s[38] power calculations can be dramatically different, and much larger samples may be needed than estimated based on Wickens’s [261] rules. In addition, it is unclear how these rules need to be modified for cross-classifications that are spanned by three or more variables. A debate concerns rule number 3. If images/978-1-1181-4640-8_1_image_11_9.jpg is clearly less than 1 and mij > 0, the Pearson X² component for Cell ij will inflate, and a model will be rejected with greater probability. Therefore, it has been proposed that Delta option be invoked in these cases, that is, by adding a small constant, in most cases 0.5, to each cell. Some software packages, notably SPSS, add this constant without even asking the user. In Section 3.1, on goodness-of-fit testing, the effects of small expected cell frequencies on the two most popular goodness-of-fit tests will be illustrated.

An alternative to the Delta option involves performing exact tests. This is an option that is available, in particular, for small tables and simple models. In this text, we focus on the approximate X² tests because, for large tables and complex models, exact tests are less readily available.

CHAPTER 2

EFFECTS IN A TABLE

This is the first section in which we discuss how to model what is going on in a table. The what is going on element is expressed in terms of effects. These effects can be compared to the effects in analysis of variance. There are (1) intercepts, (2) main effects, (3) interactions, (4) covariates, and (5) special effects such as the ones that specify scale characteristics of variables that span the table. In this section, we focus on main effects. Consider the I × J table that is spanned by the variables A and B. The probability of Cell ij is πij, with i = 1, . . . , I, and j = 1, . . . , J. The marginal probabilities of the categories of Variable A are denoted by images/c02_image_001.jpg , and the marginal probabilities of the categories of Variable B are denoted by images/c02_image_001.jpg . For the probabilities of the cross-classification, it holds that Σij πij = 1.

2.1 THE NULL MODEL

In this and the following sections, we develop a hierarchy of models, beginning with the most parsimonious model. This model proposes that no effect exists whatsoever. It is called the null model. If this proposition holds, deviations from average are no stronger than random. The expected probability for Cell ij is images/978-1-1181-4640-8_2_image_2_3.jpg The corresponding expected cell frequencies are images/978-1-1181-4640-8_2_image_2_4.jpg

Table 2.1 Cross-classification of Concreteness (C) and Wordiness (W) of the Interpretations of Statement 8; Expected Frequencies Estimated for the Null Model

images/978-1-1181-4640-8_2_image_2_1_1.jpg

EXAMPLE 2.1

In the study by von Eye, Jacobson, and Wills [237] on statement processing, the variables Concreteness of Interpretation and Wordiness of Interpretation were included. Concreteness was coded as 1 = concrete words used, 2 = abstract and concrete words used, and 3 = abstract words used. Wordiness was coded as 1 = used above average number of words for interpretation of statement, 2 = used about average number of words, and 3 = used below average number of words. Table 2.1 displays the 3 × 3 cross-classification of Concreteness and Wordiness for the 8th statement (the statement was Discretion is the better part of valor). The sample included 164 respondents.

The expected cell frequencies were estimated as images/978-1-1181-4640-8_2_image_2_9.jpg = 164/9 = 18.22. The z-score in the fourth column is the standardized residual, also called standardized deviate, calculated as images/978-1-1181-4640-8_2_image_2_10.jpg The comparison of the observed with the estimated expected cell frequencies shows that the null model does not correspond well with the data. The discrepancies between the observed and the estimated expected cell frequencies are large, and the standardized residuals (the z-scores) indicate that each cell deviates significantly from expectancy. That is, each z is greater than 1.96, the threshold for α = 0.05. We conclude, not unexpectedly, that the frequency distribution in this cross-classification does reflect effects. In the next two sections, we introduce row and column effects.

Table 2.2 Cross-classification of Concreteness (C) and Wordiness (W) of the Interpretations of Statement 8; Expected Frequencies Estimated for the Concreteness Effects-only Model

images/978-1-1181-4640-8_2_image_3_1_1.jpg

2.2 THE ROW EFFECTS-ONLY MODEL

The row effects-only model proposes that knowledge of the row marginals is sufficient to explain the frequency distribution in a table. This would allow one to estimate the expected cell probabilities based on the marginal probabilities of the rows, images/978-1-1181-4640-8_2_image_3_8.jpg , as images/978-1-1181-4640-8_2_image_3_9.jpg . Table 2.2 shows the analysis of the statement data in Table 2.1 under the Concreteness effects-only model.

The estimated expected frequencies in Table 2.2 reflect the propositions of the Concreteness effect-only model. This model proposes that the cell frequencies are proportional to the row probabilities, but not to the column probabilities. Specifically, the cell frequencies are estimated to be images/978-1-1181-4640-8_2_image_3_10.jpg .

The magnitude of the residuals suggests that this model is much better than the null model. It may not be good enough to explain the data in a satisfactory way, but the discrepancies between the observed and the estimated expected cell frequencies, as well as the standardized residuals, are much smaller than in Table 2.1. In fact, seven of the nine residuals are now smaller than 1.96, thus indicating that, for these seven cells, there is no significant model–data discrepancy. We can conclude that the knowledge that (1) most respondents interpret statements using mostly abstract words and (2) the smallest portion of respondents uses concrete words for interpretation, allows one to make a major contribution to the explanation of the data in Table 2.2. Whether or not this contribution is significant will be discussed later. In the next section, we discuss the Wordiness effects-only model.

2.3 THE COLUMN EFFECTS-ONLY MODEL

In a fashion analogous to the row effects-only model, the column effects-only model proposes that knowledge of the marginal column probabilities helps explain the frequency distribution in a table. Specifically, this model proposes that images/c02_image_001.jpg . Table 2.3 shows the analysis of the statement data in Table 2.1 under the Wordiness effects-only model.

Table 2.3 Cross-classification of Concreteness (C) and Wordiness (W) of the Interpretations of Statement 8; Expected Frequencies Estimated for the Wordiness Effects-only Model

images/978-1-1181-4640-8_2_image_4_1_1.jpg

The results in Table 2.3 suggest that Wordiness effects make less of a contribution to the explanation of the data than the Concreteness effects. It shows that the discrepancies between the observed and the estimated expected cell frequencies, expressed in comparable units by the standardized residuals, are larger than in Table 2.2. All of the z-scores in this table are greater than 1.96. Still, they are, on average, smaller than those in Table 2.1, that is, those for the null model. We conclude that Wordiness makes a little contribution. As for the row-effects model, we do not know whether this contribution is significant. All we know, at this point, is that it is greater than nothing.

It is worth noting that, in the hierarchy of models that we are developing, both the row-effects model and the column-effects model are one level above the null model, and operate at the same hierarchy level. This is of importance when models are compared based on their ability to explain data. The model discussed in the next section operates at the next higher level of the model hierarchy.

2.4 THE ROW- AND COLUMN-EFFECTS MODEL

To introduce the model that proposes that both the row and the column effects exist, we briefly review the definition of independent events. Let the probability of Event A be pA, and the probability of Event B be pB. Now, Events A and B are stochastically independent if and only if pAB = pApB, where pAB indicates the probability of the co-occurrence of A and B. If this relation does not hold, the two events are called statistically dependent, or stochastically dependent.

In the context of contingency table analysis, an event is the observation of a category of a variable. For example, it is considered an event that the interpretation of a statement is judged as abstractly worded, or it is considered an event that the interpretation of a statement is judged as wordy. The co-occurrence of these two events is the observation of an interpretation that is both abstract and wordy. If, in a model, the effects of both Concreteness and Wordiness are taken into account, both the row effects and the column effects are part of the model.

Note that we currently still use the example of an IJ table, which is a 2D table. The models that we discuss in this context can easily be generalized to tables that are spanned by more than two variables. In addition, if both the row and the column effects are taken into account, possible interactions are not (yet) taken into account. Row effects and column effects are termed main effects. Therefore, the model that takes both row and column effects into account is also called the main effect model.

Considering that no statement is made about a possible interaction between the row and the column variables, the main effect model is an independence model. From this, we can specify how to estimate the expected cell probabilities. We obtain images/978-1-1181-4640-8_2_image_5_6.jpg . This formulation reflects (1) both main effects and (2) the assumption of independence of the row variable from the column variable.

We now apply the main effect model to the statement data. Table 2.4 shows the analysis of the statement data in Table 2.1 under the independence or main effect model. The expected frequencies were estimated as images/978-1-1181-4640-8_2_image_5_7.jpg = Nπij, where N indicates the sample size.

Readers will notice that the cell frequencies for the independence model are estimated just as for the good old Pearson X²-test. In fact, the Pearson X²-test of the association between two variables is identical to the X²-test of the log-linear model of variable independence. If this model is rejected, which is the case when the X² indicates significant model–data discrepancies, the two variables are associated. This interpretation is possible because the association (interaction) of the row variable with the column variable is the only effect that is not taken into account when the expected cell frequencies are estimated.

Table 2.4 shows that, on average, the standardized residuals are smaller than for any of the simpler models discussed in the last sections. None of the z-scores is above 1.96. If this model describes the data well, the two variables that span the table can be assumed to be independent, because the discrepancies between the estimated expected frequencies, which conform to the independence model, and the observed frequencies are no larger than random. If, in contrast, this model must be rejected, an association can be assumed to exist. Taking into account scale characteristics of the data can reduce the discrepancies. If this is not enough for the main effect model to survive, an association must exist. This issue will be discussed in more detail later, in Section 9.3.7.

Table 2.4 Cross-classification of Concreteness and Wordiness of the Interpretations of Statement 8; Expected Frequencies Estimated for the Main Effect Model

images/978-1-1181-4640-8_2_image_6_1_1.jpg

2.5 LOG-LINEAR MODELS

In this section, we move from the probability notation used in the last sections to the log-linear notation. The log-linear notation has several advantages. First, it makes it easier to identify models as members of the family of generalized linear models (see Section 6.1). Second, the models have a form parallel to the form used for analysis of variance models. Therefore, their form facilitates intuitive understanding for readers familiar with analysis of variance. Third, log-linear models mostly contain additive terms. These are easier to read and interpret than the multiplicative terms used in the last section. This applies, in particular, when there are many variables and a model becomes complex. Fourth, the relationship between log-linear models and odds ratios can easily be shown (Section 4.3).

To introduce the log-linear form of the models considered in this book, consider the independence model introduced in Section 2.4, with images/978-1-1181-4640-8_2_image_6_10.jpg . The expected cell frequencies for this model were estimated from the data as images/978-1-1181-4640-8_2_image_6_11.jpg which is equivalent to images/978-1-1181-4640-8_2_image_6_12.jpg . As before, let the first variable be denoted by A, and the second variable by B (the order of these variables is, in the present context, of no importance). Taking the natural logarithm of images/978-1-1181-4640-8_2_image_6_13.jpg yields

images/978-1-1181-4640-8_2_image_6_9.jpg

which we reparameterize as

images/978-1-1181-4640-8_2_image_6_8.jpg

that is, the log-linear form of the main effect model of variable independencel. Using the log-linear form, it is easy to also represent the null model, the model that only takes the main effect of variable A into account, and the model that only takes the main effect of variable B into account. Specifically, the null model is

images/978-1-1181-4640-8_2_image_7_16.jpg

The main effect A-only model is

images/978-1-1181-4640-8_2_image_7_17.jpg

and the main effect B-only model is

images/978-1-1181-4640-8_2_image_7_18.jpg

As we said above, the model that takes both main effects into account is

images/978-1-1181-4640-8_2_image_7_19.jpg

Using the log-linear form, one can specify a model that also takes into account the association between Variables A and B. This model is

images/978-1-1181-4640-8_2_image_7_20.jpg

where the last term represents the interaction between A and B. This term is based on interaction variables that are created in a multiplicative way comparable to interaction variables in analysis of variance.

For the two variables A and B, the model that takes both the main effects and the interaction between the two variables into account, is the saturated model. Saturated models have zero degrees of freedom. All possible hierarchical effects are taken into account.²

In each of these models, the λ’s are the model parameters that are estimated from the data. The number of these parameters can be large. Consider the saturated model for a 2 ×2 table, that is, a table with four cells. This model comes with the following nine model parameters: images/978-1-1181-4640-8_2_image_7_21.jpg images/978-1-1181-4640-8_2_image_7_22.jpg