Effects and principles

• “Context effects arise when the appearance of a particular item has an effect on the difficulty of a subsequent item.”

For example, let the two following questions form possible test questions:¹⁾

If the test creating algorithm would be able chose between one, none or both of the questions, students who would get to solve both questions would have less trouble solving question 2 than students who would get to solve question 2 but not question 1. This issue can be dealt with using testlets.

Tests should be designed to start with the easier questions and progress to the more difficult ones (power tests). If so, examinees with lower knowledge levels will be encouraged by initial success and try harder to solve more difficult tasks later.There is some empirical evidence for this phenomena.²⁾

Content balancing refers to equal or desired shares of questions covering each topic in the test.

• “For example, in an arithmetic test we might want to have 25% of the items deal with addition, 25% with subtraction, 25% with division, and 25% with multiplication.”³⁾

Informal content specifications refer to, for example, question context.

• “Test developers have found that it is not wise to have too many problems dealing with the same topic… nor even the same general area… Thus, in addition to filling specifications regarding the formal content, test developers must be careful to balance the test with respect to the informal contents.”