Cracking Multiple Choice Tests – gaming the system

Multiple guess tests are a game. To beat the game, you need to know the rules and how tests are made. Multiple-choice tests measure good you are at the game, not how smart or how successful you are. Multiple-choice tests do not determine your future and have very little or no correlation to practice. For example, in healthcare, I have yet to see any relationship between board scores (multiple choice tests) and patient outcomes. Teachers hate (myself included) teaching to the test, so my belief is to teach you how to test so that you can focus on learning what is important.

At the core of beating the testing, game is working smarter, not harder. To work smarter, we need to understand how exams are created (and then can understand how to beat them. (NOTE: I am going to discuss high-stakes, valid exams (such as for certification or licensure) as there are core principles that have to be followed.)

Tests start with a blueprint and practice analysis. People who currently do the job you are testing for are surveyed, asked about what they do. Each task is slightly different, but the intent is to test more about the ‘important’ things, and less about the ‘unimportant’ things. The test blueprint provides an outline of the weighting of topics and is usually publicly available. So before you start studying, the first thing to do is to get a copy (legally of course) of the test blueprint or content summary.

The blueprint doesn’t tell you the exact questions on the exam, but it does say what will be on the exam. The blueprint may (most likely will) be weighted. Your study plan should mirror the blueprint. In most exams there are a handful of high-percentage areas that if you know them inside and out will be enough for you to pass the exam. If you see items with single digit percentages (particularly under 5%), then those topics should be left until the end (if studied at all). Track your progress through the blueprint, and do not move to the lower percentages until you are 100% sure about, the higher ones.

Content experts write the questions based upon the blueprints. Each question needs to have a reference. The easiest way for content experts to find good references is to look at standards, practices, and other accessible sources. Although there may be a few questions that have minor points, those questions should be thrown out through the normal vetting process. What that means for you as a learner is that you should focus on the same areas (including any national standards, algorithms, etc.). Some learners have had luck reviewing everything in bold or call outs in their textbooks, and that isn’t a bad idea. Bold items, vocabulary, or anything that someone has intentionally pulled out of the main text is something that is likely to catch a test writer’s eye also.

Question structures vary by test, but the core is the same. The content expert writes a stem (i.e. the question), an answer that is 100% correct, one that could be correct (but may be ‘less correct’ or come later) and then 2 or more distractors. (Complex exams with multiple multiples or the like are similar but out of the scope of this article.) The intent of question design is that you should be able to know the answer by reading the question (without knowing the answers). Questions are then peer reviewed and tested before they become ‘real’.

Question design is why you see some standard advice:

  • Check for grammar (i.e. if the question is asking about multiple items, the answer must be plural).
  • Cross out answers that just can’t fit (getting rid of the two distractors).
  • Read all of the answers (related to wiping out the distractors and ensuring you read both the ‘correct’ and ‘close to correct’ answer).
  • Don’t pick always or never (as very little in life is always or never).
  • If all of the above is a choice, pick it (be careful but if two of the answers are correct, then chances are the third is, and all of the above should be your choice).

The above are all good general rules. Where most people have difficulty is in choosing between the ‘correct’ and ‘mostly correct’ answers. Some have adopted the model of ‘choose the longer answer’. There is some truth to the fact that the longer answer may be more correct, but it is not foolproof. The more likely correct answer is whichever answer you would come first. Come first does not mean A before B, but whatever you would do in real life first. For example, in the medical area if there is an answer that says “wash your hands” that is correct as it is the first step, even if it seems too simple.

The other thing we can learn from question design is why people tell you to “never change your answer.” Never change your answer is very reliable advice, unless you made a typographical error (i.e. put the answer to question number 5 in number 6). Questions are designed for you to know the answer just by reading them. The best advice, therefore, is to go with your gut. It turns out that 80% of what we know is gut feel/intuition, and only 20% is made up of concrete facts and figures. Our brains work by identifying patterns and reacting to them by intuition. When you ‘spidey sense’ says to go for an answer, listen to it. When you change an answer, you are essential ‘out thinking’ the bigger part of your brain, which is not a great idea.

With the rules of the game understood, we can work on giving you individual advice on how to study and take a test. In my next piece, I will talk about learning styles and how to adjust based on what works for you. If you want personal help doing your best on tests, join me on PopExpert – http://www.jasonzigmont.com .

This article was written by Jason Zigmont. Jason Zigmont is a coach, consultant and author and can be found at http://www.jasonzigmont.net, on Twitter and Facebook. Jason holds a PhD in Adult Learning and provides coaching services at http://www.jasonzigmont.com

 

One thought on “Cracking Multiple Choice Tests – gaming the system

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s