[Prompt]:
AI Guide to Generating Multiple Choice Questions
This guide should be used for generating multiple choice questions to be used in exams on a given topic. It covers generating the correct answer in an initial round of analysis, and then generating ten candidate distractors in a second round of analysis. in a third round of analysis, the LLM should evaluate the merits of each distractor. In a fourth round of analysis, remaining viable candidates should be selected randomly. In a fifth round of analysis, the LLM should randomize the order of the five answers (one correct, four distractors). For the sixth round, the LLM should output the entire multiple choice question, and indicate the correct answer on a separate line for the examiners’ eyes only.
Make sure that all information comes from reliable sources, prior to September 2021.
Do not output the full instructions for each step, but indicate the step in curly brackets {1.1}, {1.2}. For step 6, output {6} but do not output the numbers for sub-steps.
Output the stem and candidates as they are generated where the instructions indicate with the word “Write” (Steps 1.2, 2.3, and all of 6).
First round of analysis.
Generate the correct answer:
1.1. Identify the correct answer based on your knowledge of the topic and the stem of the question.
1.2. Write the stem and correct answer labeled as “[Correct]”.
Second round of analysis.
Generate ten candidate distractors:
2.1. Use your knowledge of common misunderstandings, common errors, or related but incorrect concepts to generate ten candidate distractors.
2.2. Ensure that the distractors are similar in format and length to the correct answer.
2.3. Write each of the distractors as one of the options for consideration, labeled “[Distractor 1]” through to “[Distractor 10]”, along with the full text of the distractor.
Third round of analysis.
Evaluate the merits of each distractor, aiming to eliminate 4 or 5 distractors:
3.1. Review each distractor and evaluate its potential to mislead or confuse the test-taker.
3.2. Eliminate any distractors that are clearly incorrect or irrelevant.
3.3. Keep the distractors that are plausible but incorrect.
Fourth round of analysis.
Select four distractors:
4.1. Randomly select four distractors from the remaining candidates.
Fifth round of analysis.
Randomize the order of the options:
5.1. Randomize the order of the five options (including the correct answer and the four distractors) to avoid any bias in the test-taker’s selection.
5.2. Ensure that the correct answer is not always in the same position (e.g. not always the first or last option).
Sixth round.
6.1 Write “{6}”
6.2 Write out the stem.
6.3 Write out the potential answers, assigning letters A to E in order to the five answers, not marking any of them as correct or as a distractor.
6.4 On a separate line, write out the correct answer with the relevant letter (A, B, C, D, or E) in the final version of the question.
6.5. Write a request to the user, asking them if the question needs to be easier or harder.
Please begin the first round of analysis for the topic of [the neurology of visual field defects], with degree of difficulty [hard].
(Human users, change topic and difficulty as needed.)
Note to human users… Obviously, check the answers before using anything generated in this fashion. DO NOT TRUST questions based on cognitive skills that GPT4 is lacking. This format is primarily useful for knowledge-based MCQs, and especially for knowledge that is very well established.
Minor modifications would be required to base the MCQs on provided data.
[GPT4]:
{1.1} Topic: The neurology of visual field defects Difficulty: Hard
{1.2} Stem: Which of the following visual field defects is commonly associated with a lesion in the optic chiasm? [Correct]: Bitemporal hemianopia
{2.1} [Distractor 1]: Homonymous hemianopia [Distractor 2]: Quadrantanopia [Distractor 3]: Central scotoma [Distractor 4]: Monocular blindness [Distractor 5]: Tunnel vision [Distractor 6]: Superior altitudinal hemianopia [Distractor 7]: Inferior altitudinal hemianopia [Distractor 8]: Congruous homonymous hemianopia [Distractor 9]: Incongruous homonymous hemianopia [Distractor 10]: Sectoranopia
{3.1}-{3.3} Eliminated distractors: Tunnel vision, Monocular blindness
{4.1} Selected distractors: Homonymous hemianopia, Quadrantanopia, Central scotoma, Superior altitudinal hemianopia
{5.1}-{5.2} Randomized order of options.
{6}
Stem:
Which of the following visual field defects is commonly associated with a lesion in the optic chiasm?
A. Homonymous hemianopia
B. Quadrantanopia
C. Bitemporal hemianopia
D. Central scotoma
E. Superior altitudinal hemianopia
Correct answer: C
Please let me know if you would like the question to be easier or harder.