Detecting critical treatment effect bias in small subgroups

Randomized trials have traditionally been the gold standard for informed decision-making in medicine, as they allow for unbiased estimation of treatment effects under mild assumptions. However, there is often a significant discrepancy between the patients observed in clinical practice and those enrolled in randomized trials, limiting the generalizability of the trial results [12, 43]. To address this issue, the U.S. Food and Drug Administration advocates for using observational data, as it is usually more representative of the patient population in clinical practice [30, 39]. Yet, a major caveat to this recommendation is that several sources of bias, including hidden confounding, can compromise the causal conclusions drawn from observational data. In light of the inherent limitations of randomized and observational data, it has become a popular strategy to benchmark observational studies against existing randomized trials to assess their quality [4, 13]. The main idea behind this approach is first to emulate the procedures adopted in the randomized trial within the observational study; see e.g.