Recommendations for writing multiple-choice tests

Read the recommendations and use them as guidelines to explain mistakes in the examples of ill-correct multiple-choice items.

1. Avoid writing absurd distractors which everyone can see are wrong.
2. Remember that your distractors should appear correct to any Ss who are not sure of the answer.
3. Use distractors which will make Ss infer the information from the text.
4. All the distractors should be written within the Ss’ range of proficiency and at the same level as the correct option.
5. Always make sure that each multiple-choice item has only one correct answer.
6. There should be enough context to understand the meaning of the sentence.
7. Avoid questions which can be based only on the Ss’ background knowledge.
8. Avoid items which only require Ss to match words in the text.
9. Avoid distractors which do not relate to anything in the paragraph.
10. Make all the options grammatically correct if you use incomplete sentences in a multiple-choice item.
11. Avoid grammatical cues in any item.
12. Make your options approximately equal in length in the same multiple-choice item. Avoid making your right answer longer even if you want to make absolutely sure that it is correct.
13. Minimize the number of negatively stated items.
14. Avoid generalizing the correct option so that it has wider application than the distractors.
15. Avoid correct options that contain familiar or stereotyped phraseology.
16. Vary the position of the key for each item.
17. Avoid the options all of the above and none of the above.
18. Ensure each option independent of each other.
19. Avoid incorrect options which contain language or technical terms with which candidate may be unfamiliar.
20. If the stem is testing the definition of a word, put the word to be defined in the stem and make the options alternative definitions or meanings.

 

 

 

Advantages and disadvantages of cloze tests

 

Advantages

Disadvantages

Rather easy to prepare Cost-effective at the lesson Measures overall abilities and general language proficiency Scoring is reliable, economical and rapid. Easy to administer. Does not need a professional tester. Can successfully be used to measure the reading difficulty of the text. Very good for testing grammar, vocabulary and reading comprehension. Can be used for graded learning. Have many forms and modifications for different purposes and levels (e.g. modified cloze test, open cloze, multiple-choice cloze, C-cloze, etc.) All the items can be pre-tested.

There may be an element of subjectivity because of more than one acceptable answer. It may be difficult to foresee and include in the key all possible answers. Poor knowledge of the topic may affect Ss’ performance. They test only receptive skills. They are better for teaching prediction than for testing it. Not a communicative type of test. They do not test creative writing and speaking.

 

Other techniques and definite tasks with them as well as those with the above-mentioned techniques will be analysed at your seminars.

 

LECTURE 3

Warmer: Associations with ELT.

E: spoken-written, literary – slang, AE, BrE, Austr, etc

L: skills-knowledge, competence-performance, grammar, vocabulary, pronunciation –language areas; R, WR, L, S – skills, productive – receptive

T: teacher-student, approach, method, technique, teaching-testing

Testing knowledge (language areas – use of English) – testing skills (today just receptive)

Testing use of English.

The aim of testing use of English is to demonstrate knowledge and control of the

language system by completing a number of tasks, some of which are based on specially written texts. Testing use of English is important but we should remember that while grammar and vocabulary contribute to communicative skills, they are rarely to be regarded as ends in themselves.

Testing includes testing recognition of vocabulary items and grammar structures, manipulation of structures and collocations, understanding of meaning in context and awareness of appropriacy in context.

Testing of vocabulary productively is difficult, so it’s practically never attempted in

proficiency tests but is done in achievement tests.

Main task formats for testing vocabulary can include:

· Multiple choice

· Matching/multiple matching (two sets of stimuli have to be matched against each other, e.g. words and definitions, synonyms, antonyms, hyponyms and hyperonyms etc)

· Word attack (candidates are asked tp work out the meanings of a selection of highlighted or boxed words within the context of a text)

· Odd man out

· Gap filling

· Cloze/systematic deletion (together with reading skills)

· Lexis classification (match the words with the topics)

· Negative cloze/cloze elide/intrusive word (candidates must delete words which the tester had added to the text but which do not belong there) (together with reading skills)

· C-test (candidates are to restore words in the text from their parts (together with reading skills)

· Word formation (together with reading skills)

· Editing/proofreading (together with reading skills)

· Listing

· Labeling (together with reading or listening skills)

· Translation

Main task formats for testing grammar can include:

· Multiple choice

· Editing/proofreading (together with reading skills)

· Translation

· Alternative response (correct/incorrect)

· Sentence completion (opening brackets)

· Sentence combination

· Addition

· Ordering

· Transformation

· Gap filling (modernized cloze)

· Table completion (three forms of verbs, degrees of comparison of adjectives)

· Matching (sentence halves, structures and their names)

· Identifying structures

· Cued sentences (from pictures or words)

· Personalization

Advantages and disadvantages of each technique will be analysed at your practical lessons.

 

 

2. Testing listening comprehension.

Micro-skills and macro-skills or skills and sub-skills are tested.

Skills: understanding of gist, listening for specific information, follow the instructions, interpreting the information (appreciate the speakers’ attitudes and intentions), inferring the information (the underlying meaning) etc.

Sub-skills: recognition of function of structures, interpretation of intonation patterns etc

When choosing materials for testing listening as well as the way they are presented one should bear in mind the following guidelines.

Use more than one text – to get a better sample of students’ abilities (dialogue and monologue, description and narration etc)

Qualities of text:

1. Topic should be interesting to students.

2. Texts should be similar to texts that students listen to in real life.

3. Texts should reflect aim and objectives of the course.

4. Texts should be challenging.

5. Ideas in the text should not be too dense or difficult.

6. Do not choose the text which was written to be read silently.

7. Think about the function of the text (description is usually easier than narration, narration is easier than abstraction)

8. Think about organization of text (temporal and spatial relations, cause and effect relations etc)

9. Think about language of text: grammar, vocabulary.

10. Think about the register: variety gradually grows, non-standard accents can be used except at low levels.

11. Text should be authentic, recorded by native speakers (though in FCE may be non-native speakers).

12. Make sure that students would listen to different voices, both male and female voices.

13. Background noise may be avoided at low level but it depends on the pragmatic function of the text(e.g. announcements in airport)

14. Text should be characterized by natural fluency, pauses may be longer at lower levels, at the lowest – may be slower than normal speaking.

Length of text: There is no ideal length for a text. You should consider the following:

1. What skills do you want to test? How long does the text need to be to allow these to be tested?
2. What is the weighting of this component compared to others?
3. How much time is available?
4. Age of students?
5. Their proficiency level?

Text difficulty: to determine the difficulty of a test you have to consider the combination of the text that is used, the skills that the items are testing, the complexity of the technique and the constraints under which the students are working. You may have the following combinations amongst others: easy text, easy items; easy text, difficult items; difficult text easy items; difficult text, difficult items.

Way of presenting: live or recorded; number of repetitions; need for visual aids; instructions both written and listened to, at lower levels may be in the native language).

Some more guidelines.

Give a context before giving the passage.

Give listeners a purpose for listening before they listen: let them see the questions first.

Do not put undue demands on memory.

Give the listeners a task to do while listening.

Do not make listeners read too much.

Do not write items which depend on other items.

 

Possible test formats:

• True/false
• Multiple choice
• Information transfer (trace the route, complete the table, draw a diagram etc)
• Labeling
• Sequencing (texts, pictures)
• Text completion (complete the information about the film)
• Short answer questions
• Discrepancies (together with reading skills)
• Note taking

Advantages and disadvantages of each technique will be analysed at your practical lessons.

 

 

Testing reading.

What kinds of texts to use? Should reflect the age of learners, their level of proficiency

and course objectives.

At the elementary level: signs, notices, simple texts.

Intermediate: messages, letters, advertisements, magazines, reports, articles.

Advanced – academic, literary texts, newspaper, magazine, articles.

Which ability to test?

Elementary students – to show ability to skim factual texts for gist and to scan them for specific information.

Intermediate – to identify the text type, to recognize attitudes, emotions expressed by the writer, to see what the writer’s purpose is and understand the text structure.

Advanced – to decide whether a text is based on fact or opinion, to select information and use it to perform a task, to infer meaning from context, to appreciate style.