From Efficacy to Effectiveness: Evaluating Psychedelic Randomized Controlled Trials for Trustworthy Evidence‐Based Policy and Practice

Regulatory evaluation of psychedelic-assisted therapies, such as MDMA for PTSD and psilocybin for depression, faces methodological challenges because the standard requirement for two successful phase 3 randomized controlled trials (RCTs) lacks agreement on what constitutes success when psychoactive drugs are given alongside psychotherapy. This arrangement undermines the internal validity of estimated treatment effects compared with conventional controls. The paper reviews assumptions behind RCTs' gold-standard status in evidence-based medicine, highlighting limits of randomization and blinding and warning against the extrapolation fallacy. Trustworthiness that efficacy in RCTs will predict effectiveness in target populations depends on the type of treatment: low for stand-alone drugs, high for drug-assisted psychotherapies, due to different causal claims and external validities.

Study at a glance

Abstract

ABSTRACT The recent review of a new drug application for MDMA‐assisted therapy for posttraumatic stress disorder by the United States' Food and Drug Administration (FDA) highlighted epistemological and methodological challenges for evidence assessments. Similar challenges will also be faced in reviews of other compounds in early‐ and late‐stage development, like psilocybin for depression. The regulatory demand for two successful phase 3 randomized controlled trials (RCTs) seems problematic, given a current lack of agreement on what constitutes “success”, particularly when psychoactive drug administration is concomitant with (psycho)therapy. These complex arrangements challenge the internal validity of estimated average treatment effect through comparison with conventional control conditions. This paper reviews the assumptions behind RCTs' current “gold‐standard” status in the hierarchy of evidence‐based medicine (EBM). Recapitulating known epistemic limits of randomization and blinding, it emphasizes the urgent need to avoid the extrapolation fallacy. The resulting argument is that the degree of trustworthiness that efficacy—reported in RCTs—will reliably predict effectiveness—in target populations outside RCTs—depends on what type of psychedelic treatments will be regulated. If “stand‐alone” drugs for large‐scale prescription and consumption, trustworthiness should be graded low. On the other hand, for regulation of drug‐assisted (psycho) therapies, the degree of trustworthiness can be considered high. The reason being that these two treatment approaches are based on different causal claims with distinct external validities. Therefore, careful assessment of support factors in each is recommended to prevent detrimental consequences, from potential rejection of effective therapies up to medical reversal of eventually approved drugs.