Guidelines for designing tests

Tests are a series of short answer questions and learner selected responses, usually used to assess a learner's underpinning knowledge or understanding. Best suited to:

  • off the job environments, but may be used
  • on the job, in a quiet area away from noise or disruptions.

Preparing a test
Short answers
Label diagrams
Fill in the gaps
Multiple choice questions
Matching and alternate choice questions
True/false questions
Templates

Advantages

  • a useful tool for generating evidence to supplement other types of assessment
  • useful where assessment methods are restricted due to safety requirements or infrequent occurrence of specified events / activities
  • useful for assessing underpinning knowledge and understanding
  • test conditions can be standardised for skills tests
  • time for testing can be allocated

Issues

  • individuals can react differently in test situations
  • often removed from realistic work situations
  • tests need to be well planned and structured so that they are actually testing what they aim to assess

Preparing a test

When you start to prepare a test, decide on the unit(s)of competency that you are assessing and then write or design questions that best assess them. The following is a list of the different types of questions / items you may choose:

When you develop questions you should consider developing an item bank. An item bank is a file of test questions that makes it possible to develop a good test with minimal time and effort. All available test questions can be stored by topic. Sets of instructions or directions can be added and the item bank can be updated easily.

Hints

  • prepare clear and concise instructions
  • include easy as well as difficult questions (arrange questions in order of difficulty)
  • use separate answer sheets to retain test questions for future use and to make marking easier
  • keep the language and numeracy skills of the learners in mind when writing the questions

Short answers

A short answer item is a question used to find out about what the learner knows. It requires the learner to provide an answer rather than to select an answer. Short answer items use questions which restrict the answer to a short paragraph, sentence, or single words. Short answer questions are more appropriate for some outcomes than others. They are useful for recall of facts, analysis of data and solving problems. Examples of short answer items could include lists of items or equipment, brief descriptions of a procedure, sequence of an activity or reasons to explain an event.

Advantages

  • easy and relatively quick to write
  • less influenced by guessing than multiple choice or true/false questions
  • easy to score/mark with a high reliability of marking, especially with model answers or a marking guide
  • provides more scope for the learner to show their knowledge than with a limited response question.

Issues

  • short answer questions should give the learner clear guidance on the target length of their answers
  • when a marking guide is developed, the test designer/assessor should recognise that there are likely to be correct answers in addition to those expected and that these should be added to the marking guide over time

Examples

Describe (in not more than 50 words) what is meant by.......
(2 marks)

Summarise the three main elements that ..........
(2 marks)

What do each of the following initials stand for:
xxxxx......
yyyyy.....
zzzzz......
(2 marks)

Hints

  • ask for specific information
  • word the question precisely and concisely
  • ensure that there is a single or defined number of correct answers
  • avoid clues to the answer
  • indicate the marks for the item / questions

Label diagrams

This type of question requires the learner to look at a drawing or picture and write the correct name in a blank space. This could also be a practical test with actual items of equipment or components.

Advantages

  • useful in measuring the learner's knowledge of tools, supplies or components
  • assists learners with low literacy skills by use of pictorial materials.

Issues

  • preparation of clear diagrams or pictorial material is difficult.
  • NOTE: diagrams and photographs can be inserted into the templates in this kit using clip art or inserting an object.

Examples

Name the tool below...................
Identify the features that................

insert diagram or photo of a tool

Fill in the gaps

"Fill in the gap" questions are another variation of the short answer question. The question consists of a true statement in which one or two important words have been left out.

Advantages

  • Little scope for the learner to guess the correct answer
  • Useful for assessing recall

Issues

  • Often difficult to write questions that have only one correct answer
  • Sometimes the word(s) omitted make it difficult to understand the meaning of the sentence

Hints

  • Omit only a relevant key word
  • Use straight forward sentences
  • Check that learners can infer the meaning even without the deleted word
  • Do not use a or an to provide a clue
  • Give credit for all potential answers that make sense

Multiple choice questions

A multiple choice item is a question or incomplete statement followed by four or five options from which the learner selects the best answer(s). Incorrect options are called distractors. Distractors are used to side track learners who may not know the correct answer.

Advantages

  • Marking / scoring is uniform, standardised and usually quick
  • Objectivity in marking reduces any teacher variability factors
  • Assesses a learner's knowledge as well as their ability to discriminate amongst several possible alternatives
  • Can test judgment as well as memory
  • Provides a better sample of the topic being assessed than most other formats given the same period of time
  • Allows for higher order questions to be asked
  • Can assess more complex cognitive tasks

Issues

  • Relies on the learner having appropriate reading skills
  • Construction of effective questions is often difficult and time consuming
  • The quality of the question often depends on appropriate"distractors" or incorrect options
  • Economies of scale can only be achieved with large numbers of learners

Examples

Direct question
a
b
c
d		 Incomplete statement
a
b
c
d
Best answer
a
b
c
d		 Knowledge (comprehension, application, analysis, synthesis)
a
b
c
d

Hints

  • Allocate approximately one multiple choice question per minute
  • Questions should be meaningful and represent a specific problem in the stem of the question
  • Questions should be stated in positive rather than negative terms
  • Distribute answers equally amongst positions of a,b,c,d etc
  • Avoid the use of grammatical clues at the end of a stem, for example:a, an, are, is, this, these
  • Avoid the use of "none of the above" as an option
  • Try to include at least four responses for each question
  • Use between 5-10 questions to assess each criteria or learning outcome
  • Prepare more questions than you need as not all will be useful

Matching and alternate choice questions

The matching question is a type of multiple choice question which is helpful for assessing knowledge of related information. It consists of two lists or columns of related information from which the learner is asked to match appropriate items.

Advantages

  • A lot of content can be covered in one question
  • Easier to write than multiple choice questions
  • Useful variation in questioning
  • Objective and easy to mark/score.

Issues

  • Need a large number of items to match within the question
  • Specific instructions/guidelines should be given for matching items

Hints

  • Need 7-10 items to match within the question

True/false questions

The true-false question is basically a statement that has to be identified as correct or incorrect.

Advantages

  • Useful for assessing the basic knowledge of facts

Issues

  • Can be prone to the effects of guessing
  • Questions can be tests of recall of trivial information

Hints

  • Ensure the statement is entirely true or false
  • Include only one idea in each question
  • Use clear language and avoid double negatives
  • Place true and false answers in random order
  • Use false answers so that they represent misconceptions, and true answers for correct ideas
  • Include many questions on a topic to reduce the impact of guessing answers(approx 50 questions for a true-false test)
  • Avoid the terms, usually, always, none
  • Avoid trick questions

Templates

Download the tests template (Word - 48 Kb)
Download the questions template (Word - 25.5 Kb)

    Print this page
  

Need help?

Related links