Measurement model: Understanding the errors

A theoretical model, measurement model in research describes and evaluates the quality of measurement to improve its usefulness and accuracy. In quantitative research, we need to measure variables because variables operationalize^[1] constructs/concepts. Measurement is the process of assigning numbers to the variables according to rules. Variables represent attributes or properties of subjects or treatments. We use a specific device or procedure to measure a variable, generally referred to as an instrument. No matter what the instrument is, there is always a chance of measurement error. That is, we might not get a real true measure. Let us represent this theoretical measurement model below as

Observed score (X) = True score (T) + Error (E)
or
X = T + E

Let’s take an example. Suppose we are interested in measuring the weight of each child in a classroom. We use a measuring scale to record the weight of each child. But we may not be satisfied with the accuracy of this measure since many factors can influence the accuracy of the measure. For example, an uneven floor, carpet on the floor, accuracy of the scale, or perhaps the scale was not set to zero, and so on. We can classify all these sources of error as measurement error which distort the true score.

Measurement model – Components of error

There are two types of error.

Random error and
Non-random error

Random errors: Random errors are unsystematic. They are chance factors that affect the accuracy of our measurement. A unique characteristic of random error is that it might not affect every observation, and it might not affect every observation in the same way. Random errors are equally likely to underestimate or overestimate the true score. That is, random errors are truly random, making the data measurement less precise. To circumvent this problem one can use statistical procedures. Since random errors are equally likely in either direction, by taking multiple measurements, random errors should average out and their effect is likely to be nullified. For example, we might have repeat assessments of blood pressure as follows


Time Point	X	T	E
T1	122	120	+2
T2	119	120	-1
T3	121	120	+1
T4	118	120	-2

Non-random error: On the other hand, non-random errors are systematic and affect all observations in one direction more than the other. In the above example of weight measurement, when the scale is kept on the carpet, it softens the floor leading to likely underestimation of all scores. Or if the measurement scale faulty or not set to zero but instead is set to 2, it will overestimate all the scores by 2 pounds. We call this bias. Bias occurs due to distortions in procedures and characteristics of instruments, observers, and investigators (Spector, 1981).

What about unintentional errors that arise from instruments and procedures. We call this error of measuring the wrong construct. For example, a researcher is interested in measuring the overall health of participants. If they ask the doctors to evaluate the health by measuring all the physiological measures, would that be enough? By restricting the measure to just physiological measures (intentionally or unintentionally) they are leaving out the psychological measures of health, such as mental health, emotional health, and so on. So the operationalization process does not measure the construct completely. Such an error will result in measuring a construct that is incomplete at best or incorrect at worst.

There is no simple way to process and average out the effect of non-random error. This makes non-random error more problematic as it limits our ability to draw valid inferences. Researchers have to make an effort to uncover such non-random errors and careful research design can minimize non-random errors. We can now expand our theoretical measurement model below by expanding the error term with E(r) representing the random error, B for bias, and E(w) for the error of measuring the wrong construct.

X = T + E(r) + B + E(w)

Random error, E(r), affects the reliability of the measure. Reliability refers to the consistency of a measuring device. That is, it measures how consistent the measuring device or instrument is when multiple observations are taken when the true score is unchanged. Reliability serves to quantify the precision of the instrument (accuracy on repeated trial), and thus the trustworthiness of the scores.

Non-random error, B, and E(w) affect the validity of the measurement. Validity refers to the degree to which the measure accurately reflects the desired attributes of the construct. In other words, are we measuring what we think we are measuring? For a measure to be useful, it has to be both, valid and reliable.

Footnote:

[1] Operationalization is the actual process through with a construct is measured.

Bibliography

Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Beverly Hills, Calif.: Sage Publications.

Dooley, D. (2001). Social research methods (4th ed.). Upper Saddle River, New Jersey: Pretence Hall.

Price, L. R. (2017). Psychometric methods : Theory into practice. New York ; London: The Guilford Press.

Spector, P. E. (1981). Research designs. Beverly Hills: Sage Publications.

Stevens, S. (1946). On the theory of scales of measurement. Science, 103(2684), 677-680. Retrieved November 10, 2020, from http://www.jstor.org/stable/1671815