9.3.1 Systematic Error, Random Error and Internal/External validity

Describes What is Meant by Systematic Error (selection and measurement bias), Random Error (chance) and Internal Validity and External Validity

When evaluating the quality of research, it is important to consider potential sources of bias and the validity of the study. Here is a teaching piece on the concepts of systematic error, random error, and internal and external validity:

Systematic error:

Systematic error refers to errors or biases that occur consistently in the same direction throughout a study, resulting in incorrect or misleading conclusions. Two common types of systematic error are selection bias and measurement bias.

Example: A long-term observational study is conducted to determine the relationship between exercise frequency and heart disease. However, the questionnaire used to assess exercise frequency tends to overestimate light activities as moderate exercise due to ambiguously phrased questions. As a result, participants who engage primarily in light activities (like slow walking) are inaccurately classified as engaging in moderate exercise. This systematic error in the measurement tool leads to consistent overestimation of exercise intensity across the study population, potentially skewing the study’s findings regarding the protective effects of exercise against heart disease.

Selection bias:

Occurs when participants are not representative of the population being studied, leading to incorrect conclusions about the relationship between exposure and outcome.

Example: A study on the effectiveness of a new cancer treatment only enrolls patients from a high-income urban hospital, excluding those from rural or lower-income areas. This leads to selection bias, as the study population does not represent the broader population of cancer patients.

Measurement bias:

Occurs when measurements are inaccurate or imprecise, leading to incorrect conclusions about the true relationship between exposure and outcome.

Example: In a study assessing the impact of a dietary intervention on cholesterol levels, the cholesterol is measured using a method known to underestimate cholesterol levels in overweight individuals. This introduces measurement bias, as the results might not accurately reflect the true cholesterol levels of overweight participants.

Random error:

Random error refers to the variability or chance inherent in any measurement. It is not possible to eliminate random error completely, but it can be reduced by increasing sample size, improving measurement tools, and implementing standardized procedures.

Example: In a large-scale epidemiological study, variations in individual responses due to factors like slight day-to-day changes in diet, mood, or physical activity introduce random errors. These errors are not systematic but occur due to natural variability among participants.

Internal validity:

Internal validity refers to the extent to which a study design and implementation minimize the potential for systematic error. Internal validity is important for determining whether the results of a study are valid and can be generalized to the population of interest.

Example: A randomized controlled trial (RCT) testing a new diabetes drug maintains strict control over variables like participant selection, dosing, and monitoring of outcomes. The high level of control ensures that the results (e.g., improved blood sugar control) can be confidently attributed to the drug, demonstrating strong internal validity.

External validity:

External validity refers to the extent to which the results of a study can be generalized to other populations, settings, or circumstances. The external validity of a study can be influenced by factors such as the study population, study setting, and study design.

Example: A clinical trial for a new arthritis medication includes a diverse range of participants in terms of age, gender, ethnicity, and severity of arthritis. The results of this trial can be generalized to the broader population of arthritis patients, indicating high external validity.

Concept	Description
Systematic error	Errors or biases occur consistently in the same direction throughout a study, resulting in incorrect or misleading conclusions. Two common types are selection bias and measurement bias.
Random error	Variability or chance inherent in any measurement cannot be completely eliminated but can be reduced through increasing sample size, improving measurement tools, and implementing standardized procedures.
Internal validity	The extent to which a study design and implementation minimize the potential for systematic error, which is important for determining whether the results of a study are valid and can be generalized to the population of interest.
External validity	The extent to which the results of a study can be generalized to other populations, settings, or circumstances, which can be influenced by factors such as the study population, study setting, and study design.

References:

1. Chalmers, I., & Glasziou, P. (2009). Avoidable waste in the production and reporting of research evidence. The Lancet, 374(9683), 86-89.
2. Guyatt, G. H., Sackett, D. L., & Cook, D. J. (1994). Users’ guides to the medical literature II. How to use an article about therapy or prevention. A. Are the results of the study valid? Jama, 271(5), 389-391.
3. Portney, L. G., & Watkins, M. P. (2009). Foundations of clinical research: Applications to practice. FA Davis.