Confounding — RWE’s Fundamental Limitation

Every study design covered in Lessons 15.4 and 15.5 exists partly in response to the same underlying problem: confounding. Patients who receive a specific drug differ systematically from patients who don’t receive it, in ways that are not random and, critically, may not be fully measurable in the available data — differences in disease severity, comorbidities, contraindications to alternative treatments, and socioeconomic factors that influence both what treatment a patient receives and their underlying risk of the outcome being studied. These differences create associations between drug exposure and an outcome that look causal in a naive analysis but are actually attributable to the underlying disease or patient characteristics, not the drug itself.

Confounding by indication — already mentioned in Lesson 15.4 as cohort design’s dominant validity threat — is the specific, most common form of this problem in pharmacovigilance: a drug prescribed preferentially to sicker patients will show an apparent elevated risk for almost any adverse outcome, purely because the population receiving it started at higher baseline risk. A naive comparison of "drug users" versus "non-users" without accounting for this will systematically overstate the drug’s actual contribution to the outcome.

Three main methods exist to control for confounding, and understanding what each one actually does — and doesn’t do — matters. Propensity score methods statistically model the probability of receiving the drug given a patient’s measured characteristics, then match or weight patients so the comparison groups are balanced on those measured characteristics, mimicking some of what randomisation achieves in a trial, but only for the confounders that were actually measured and included in the model. Active comparator design sidesteps part of the problem structurally, by comparing drug users against users of an alternative treatment for the same indication rather than against non-users — both groups share the indication, which controls for confounding by indication specifically, even before any statistical adjustment is applied. And instrumental variable analysis uses a third variable, associated with treatment choice but not directly with the outcome except through that treatment choice, to approximate a more randomised-like comparison — a more specialised, less commonly used technique requiring a genuinely valid instrument to work correctly.

The honest limitation, stated plainly, is that all three methods reduce confounding from measured, known confounders — none of them can fully eliminate confounding from a variable that was never measured or recorded in the underlying data at all. This is precisely why this module’s framing, consistent since Lesson 15.1, treats RWE as most valuable when used as a complement to spontaneous report data (Module 7) and clinical trial evidence, rather than as a standalone causal-inference tool capable of settling a safety question entirely on its own. Lesson 15.9’s triangulation principle builds directly on this limitation: multiple independent evidence streams, each with different weaknesses, converging on the same conclusion is a stronger basis for regulatory action than any single RWE study, however well-designed, standing alone.