Intro

We are interested in knowing the probability that a student understands the answer and concept given that the answer selected is correct.

Formulation

Take a multiple choice exam, each question has m answer choices i.e. i, ii, iii, … , m.

Let \(p =\) Probability that the student understands the concept and \(1 - p =\) Probability that the student doesn’t understand the concept.

Then: \(C =\) “Correct Answer” and \(K =\) “Understands the answer and concept.”

Probability

So, $$P\left( {K \vert C} \right) = \frac{P\left( {K \cap C} \right)}{P\left( C \right)} = \frac{P\left( K \right)}{P\left( C \right)}$$. Why are we able to disregard C in the numerator? By the definition of the probability spaces, we note that the intersection of KC is only K since K is contained in C. That is to say that K is a subset of C.

Therefore, only the values in K will be in C for the intersection KC. Continuing on… $$\frac{P\left( K \right)}{P\left( {C \vert K} \right)P\left( K \right) + P\left( { C \vert {K^c} } \right)P\left( { {K^c} } \right)} = \frac{\left( p \right)}{\left( 1 \right)\left( p \right) + \left( { \frac{1}{m} } \right)\left( {1 - p} \right)} = \frac{mp}{mp + 1 - p} = \frac{mp}{1 + \left( {m - 1} \right)p}$$

Application

What if we only had 4 options per question with a \(p =.5\)? Then, $$\frac{\left( {4 \cdot .5} \right)}{\left[ {1 + \left( {4 - 1} \right) \cdot .5} \right]} = .8$$

If a student doesn’t understand the problem and receives credit, $$P\left( { {K^c} \vert C } \right) = 1 - P\left( { K \vert C } \right) = 1 - .8 = .2$$

For kicks, what happens if there are infinite options per question? $$\mathop {\lim }\limits_{m \to \infty } P\left( {K \vert C} \right) = \frac{mp}{1 + \left( {m - 1} \right)p}\overbrace = ^{LH}\frac{p}{p} = 1$$

The takeaway: increasing the amount of answers per question decreases the likelihood of the examinee receiving the right answer if they do not understand the problem.