Types of reliability

• Stability: test-retest. You give a test, let time pass, and give the *same* test again. Not some other form of it.

• Equivalence: parallel forms. You give two distinct forms (form A and form B) of a test. Both forms are supposed to be representative samples of the same domain.

• Stability and equivalence: You give form A in September and form B in May.

• Internal consistency: how well do the individual items correlate with one another (like Coefficient Alpha)? The old way was split-half; now we use KR20 and KR21 because computers can handle this easily.

• Inter-rater reliability: two observers have different subjective judgments of a measurement procedure; how well do they correlate with each other?

Other important points

• If you have all high ability people grouped together, you lower the variability, and that lowers the reliability of the test.

• The maximum reliability coefficient - like any correlation - cannot be greater than |1.0|.

• Correlations are mean free. On two occasions, if you take the same test, your position remains the same if there is a correlation of 1.0. That does not mean that your exact score remains the same.

• On Laura's test - read the questions very carefully. 95% means two sigma.

• As the reliability decreases, the SEM increases, and vice versa. It is an inverse relationship.

Back to Statistics course