Test-wiseness

If you are designing a multiple-choice or true-false quiz to assess students' knowledge and mastery of some material, there is a chance that part of what you are assessing is not knowledge of the content but rather how good students are at taking tests (test-wiseness). Basically, students can guess the correct answer or narrow down the options based on things such as wording and grammar (secondary cues) rather than knowledge of the topic being assessed.

For example, in answering the nonsensical multiple-choice question below, you can infer the correct answer by matching the end of the question stem ("an") with the beginning of the choices. The correct answer must be the one that starts with a vowel:

 A zieb always comes after an:
   a) burb
   b) kond
   c) arph
   d) rhoo

Test-wiseness Cues

Here are some criteria that may be used to make test questions less susceptible to test-wiseness:

Categorical Exclusive

Distracters contain words such as "all", "always", or "every." If a choice contains the word "all" or or "every", a student may more easily rule it out. Try not to use those terms in the question or in the choices.

Phrase-Repeat

The correct answer contains a key sound, word, or phrase that is contained in the question’s stem.

Absurd/Implausible Answer

Distracters are unrelated to the stem. Some incorrect choices may be more obviously wrong and easier to eliminate.

Precision/Specificity

Correct answer is more precise, clear, or qualified than the distracters. It is very detailed and provides no room for ambiguity. That suggests it is the correct answer.

Length

Correct answer is longer than the distracters. Try to make each of the choices relatively equal in length.

Grammar

Distracters do not match the verb tense of the stem, or there is not a match between articles ("a", "an", "the").

Give-Away

Correct answer is given away by another item in the test. Sometimes the answer to question 2 can be found in the stem of question 1, for example.

Order of Answer

Don't use a predicatible pattern for the answers (for example, making the correct answer usually be C).

Number of Options and Guessing

Out of a hundred multiple-choice items with three answer options each, a student with no knowledge of the content will get 33 correct simply by guessing blindly. However, the more choices you add, the more susceptible the question becomes to test-wiseness, hence four items is most common.

Odd Man Out

If you make two or more choices difficult to distinguish from one another conceptually, that usually helps you rule them out. If three answers are very similar and one is very different, the different one may likely be the correct answer.

Spelling

Another one mentioned is that test authors are more likely to make spelling errors on incorrect choices, since they mainly review the correct answers.

Examples

1. Robert Runt has a webpage with tips for students on taking tests. At the bottom of the page is a sample multiple-choice quiz illustrating many of the above principles.
2. Gibb's 1964 dissertation contains a 70 item Experimental Test of Testwiseness.
3. Mahamed et al. created a 20 item test-wiseness questionnaire for pharmaceutical students.

History

Thorndike (1951) first conceptualized "test wiseness" as a variable that can affect test scores. Thorndike viewed test wiseness as a general and lasting characteristic of the individual and a prominent source of score variance on cognitive tests second in its influence only to in­dividual cognitive ability. More specifically, Millman, Bishop, and Ebel ( 1965 , p. 707) defined test wiseness as the "subject's capacity to utilize the characteristics and formats of the test and/or the test taking situation to receive a high score. Test wiseness is logically indepen­dent of the examinee's knowledge of the subject matter for which the items are supposedly measures."