Learning Theories

Q-matrix is a matrix describing relations of questions and concepts required for their understanding. It is a domain-independent model of knowledge represented by a binary matrix showing the relationship between test items and latent or underlying attributes, or concepts.

Q-matrix is a MxN matrix, where M equals the number of questions in an assessment, and N equals the total number of concepts required for understanding all questions. The matrix element A[i,j] equals 1 if the i-th concept is required for correctly answering j-th question and 0 if the i-th concept is NOT required for correctly answering j-th question. Alternatively, matrix values can be not just {0,1}, but real numbers from the interval [0,1], describing the probability that a student who knows i-th concept will correctly answer j-th question.

An example of a q-matrix is shown in the following image.

From the matrix, one can read that knowledge of concept 1 is required in order to answer correctly questions 3, 4 and 5. One can also read that questions 1 and 2 test only the knowledge of concept 2.

Furthermore, one could also say that the ideal response of a student taking the test formed of those 5 questions who knows only concept 1 should be “00001”. This is so since he does not know concept 2 which is required for questions 1 and 2 (therefore the leading “00”). Yet this concept is also required for correct ansqering of questions 3 and 4 so he can not answer those questions neither (therefore the following “00”). Finally, 5th question requires only knowledge about the concept 1 so the student can answer this question correctly (therefore the ending “1”, forming all together “00001”).

• “method, which examines the inputs of many students to automatically extract relationships between questions and underlying concepts, and then uses those relationships in diagnosing and correcting student misconceptions.”
• domain-independent knowledge model
• originally a binary matrix showing the relationship between test items and latent or underlying attributes, or concepts
• To build the q-matrix, experts constructed a relationship between test questions and concepts (referred to as attributes) and students taking the test were assigned knowledge states based on their test answers and the constructed q-matrix ¹⁾

The goal of q-matrix construction is to extract underlying, or latent, variables, which account for studentsí differential performance on questions.

Approaches:

• Hand construction of the q-matrix by experts' assigning concepts to questions and then comparing student answers to closest matrix responses. Problems: a q-matrix is a much more abstract measure of the relationships of questions to concepts. We might assume that the questions designed to test students are a more accurate reflection of the teaching objectives than an abstract construct which relates questions to underlying concepts.
• The alternative to this strategy is to design a method to extract a q-matrix, which explains student behavior, and reveals the underlying relationships between questions. Experts can examine the resulting q-matrix 25 to ensure that the extracted relationships seem to be valid, and then use that q-matrix to guide the generation of new problems.

Factor analysis: How to automatically determine concepts? Using covariance matrix. Number of concepts should be smaller than number of questions. Still, this methos has proven to be less fault tollerant.

The q-matrix method is a simple hill-climbing algorithm that creates a matrix representing relationships between concepts and questions directly. The algorithm varies c, the number of concepts, and the values in the q-matrix, minimizing the total error for all students for a given set of n questions. To avoid of local minima, each hill-climbing search is seeded with different random Q-matrices and the best of these is kept.

When forming a correlation matrix, we lose individual student data in favor of calculating average relationships between questions. The q-matrix method is optimized to assign each student the most appropriate knowledge state, using all available response data for each student.

As Sellers found in her research, the results obtained through q-matrix analysis seem to describe relationships among variables in interpretable ways. Factor analysis and principal components analysis, on the other hand, do not readily offer interpretable results.

Later researchers found that, although the q-matrix model was a good way to compare student data to a concept model, expert-constructed q-matrices did not correspond to student data any better than random q-matrices did.

the findings in Brewer's previous research, which found that the factor analysis method performed poorly in comparison with the q-matrix method when fewer observations were available.