Specific Comparisons
Prerequisites
Difference
Between Two Means (Independent Groups)
There are many occasions on which the comparisons
among means are more complicated than simply comparing one mean
with another. This section shows how to test these more complex
comparisons. The methods in this section assume that the comparison
among means was decided on before looking at the data. Therefore
these comparisons are called planned comparisons.
A different procedure is necessary for unplanned
comparisons.
Let's begin with the made-up data from a hypothetical
experiment shown in Table 1. Twelve subjects were selected from
a population of high-self-esteem subjects (esteem = 1) and an
additional 12 subjects were selected from a population of low-self-esteem
subjects (esteem = 2). Subjects then performed on a task and (independent
of how well they really did) half were told they succeeded (outcome
= 1) and the other half were told they failed (outcome = 2). Therefore
there were six subjects in each esteem/success combination and
24 subjects altogether.
After the task, subjects were asked to rate (on
a 10-point scale) how much of their outcome (success or failure)
they attributed to themselves as opposed to being due to the nature
of the task.
The means of the four conditions are shown in
Table 2.
There are several questions we can ask about the
data. We begin by asking whether, on average, subjects who were
told they succeeded differed significantly from subjects who were
told they failed. The means for subjects in the success condition
are 5.500 for the low-self-esteem subjects and 7.333 for the high-self-esteem
subjects. Therefore, the mean of all subjects in the success condition
is (5.500 + 7.333)/2 = 6.667. Similarly, the mean for all subjects
in the failure condition is (7.833 + 4.833)/2 = 6.083. The question
is, how do we do a significance test for this difference of 6.667
-6.083 = 0.084?
The first step is to express this difference in
terms of a linear combination of a set of coefficients and the
means. This may sound complex, but it is really pretty easy. We
can compute the mean of the success conditions by multiplying
each success mean by 0.5 and then adding the result. This works
because adding two numbers and then dividing by 2 is the same
thing as multiplying each number by 0.5 and then adding the products.
For example to get the mean of the numbers 1 and 3, we could add
them together and divide by 2: (3+1)/2 = 2. Or, we could multiply
each by 0.5 and add: (.5)(1) + (.5)(3) = .5 + 1.5 = 2.
We therefore can compute the difference between
the "success" mean and the "failure" mean
by multiplying each "success" mean by 0.5, each failure
mean by -0.5 and adding the results. In Table 3, the coefficient
column is the multiplier and the product column in the result
of the multiplication. If we add up the four values in the product
column we get
L = 2.750 + 3.667 - 3.917 - 2.417 = 0.083
This is the same value we got when we computed
the difference between means previously (within rounding error).
We call the value "L" for "linear combination."
Now, the question is whether our value of L is significantly different
from 0.
The formula for L is:

where ci is the ith coefficient
and Mi is the ith mean. For this example,
L = (0.5)(5.50)+(0.5)(7.33)+(-0.5)(7.833)+(-0.5)(4.833)
= 0.083.
The formula for testing L for ignorance is shown
below

In this example,

MSE is the mean of the variances. The four variances
are shown in Table 4. Their mean is 1.625. Therefore MSE = 1.625.
The value of n is the number of subjects in each
group. Here, n = 6.
Putting it all together,

We need to know the degrees for freedom in order
to compute the probability value. The degrees of freedom is
df = N - k
where N is the total number of subjects (24) and
k is the number of groups (4). Therefore, df = 20. Using the Online
Calculator, we find that the two-tailed probability value is 0.874.
Therefore, the difference between the "success" condition
and the "failure" condition is not significant.
Online
Calculator: t distribution
A more interesting question is whether the effect
of outcome (success or failure) is different depending on the
subject's self esteem. Perhaps, the effect of the outcome depends
on the subject's self-esteem such that compared to high-self-esteem
subjects, low-self-esteem subjects are less likely to attribute
success to themselves and more likely to attribute failure to
themselves.
To test this, we have to test a difference between
differences. Specifically, is the difference between the two self-esteem
groups different in the success condition than it is in the failure
condition? The means shown in Table 5 show that this is the case.
For the success condition, the low self-esteem subjects attributed
the outcome to themselves less than did the high-self-esteem subjects.
The opposite pattern occurred in the failure condition where the
low-self-esteem subjects attributed the outcome to themselves
more than did the high-self-esteem subjects. For the success condition,
the difference between the high-self-esteem mean and the low-self-esteem
mean is 7.333-5.500 = 1.833. For the failure condition, the difference
is 4.833- 7.833=-3. The difference between differences is 1.833-(-3)=4.83.
The coefficients to test this difference between
differences are shown in Table 5.
If it is hard to see where these coefficients
came from, consider that our difference between differences were
computed this way:
(7.33 - 5.5) - (4.83 - 7.83)
= 7.3 - 5.5 - 4.83 + 7.83
= (1)7.3 + (-1)5.5 + (-1)4.83 + (1)7.83
The values in parentheses are the coefficients.
To continue the calculations,


The two-tailed p value is 0.0002. Therefore, the
difference between differences is highly significant.
Multiple Comparisons
The more comparisons you make, the greater your
chance of a Type I error.
|