Interpreting the Quiz Item Analysis report

The Quiz Item Analysis report provides instructors with actionable feedback on quizzes and individual questions. These measures are the standard for certain types of high-stake quizzes. For the report to be valid, it must comply with the following assumptions:

  • The report is assessing results for first quiz attempt only.
  • Quiz only contains multiple-choice questions.
  • Each quiz question has only one correct answer.
  • The question order can be random, but all learners must receive the same set of questions.
  • Quiz questions relate to a coherent subject area to ensure applicability of the quiz reliability measure.

    Quiz Item Analysis report

The Quiz Item Analysis report interprets data using the following formulae:

Formula

Description

Upper 27%

This calculates how many of the quiz takers ranking in the top 27% of the class passed the question.

For example, if learners 1, 2, and 3 are the top 27% of performing learners on the quiz, and learners 2 and 3 passed question 1, then 66.67% of the top 27% performing learners passed the question.

Lower 27%

This calculates how many of the quiz takers ranking in the bottom 27% of the class passed the question.

For example, if learners 4 and 5 are the bottom 27% of performing learners on the quiz, and learner 4 passed question 1, then 50% of the bottom 27% performing learners passed the question.

Discrimination Index

This calculates any possible discrimination in the question between the higher and lower performing learners by subtracting the question's lower 27% from its upper 27%. The higher the difference, the better the question discriminates between high and low performing learners.

For example, the discrimination index on question 1 is 66.67%-50%=16.67%.

A negative value indicates that more low performing learners are correctly answering the question than high performing learners. This would most likely indicate a flawed quiz question.

Point Biserial

This calculates any possible correlation between the question score and the total quiz score. It measures the strength of the correlation between the two values, and shows either a positive or a negative relationship ranging from -1.00-+1.00.

Point Biserial displays a "N/A" value when all users answer a specific question correctly.

Standard Deviation

This calculates how much the individual quiz question score varies from the average quiz question score. Standard deviation increases as less people answer a specific question correctly.

Standard Deviation displays a "0.00" value when all users give the same answer a specific question, correctly or incorrectly.