‘Equal-unblinding’ meta-analysis of psychedelic therapy vs. antidepressants for the treatment of depression

Williams and colleagues frame the analysis around two contrasting pictures of antidepressant efficacy: traditional antidepressants (tADs, primarily SSRIs and SNRIs) have small treatment–placebo (between-arm) differences on the HAMD-17, while psychedelic-assisted therapy (PAT) trials report much larger between-arm effects. The introduction highlights functional unblinding — patients deducing allocation from drug effects — as especially pervasive in PAT trials because of the drugs' intense subjective effects, and notes that unblinding can inflate between-arm estimates. The authors observe limited direct head-to-head evidence comparing PAT and tAD and argue that comparing PAT trials (effectively unblinded) to truly blinded tAD trials creates an unfair advantage for PAT. This pre-registered meta-analysis therefore sets out to compare PAT against open-label tAD trials so both treatments would equally experience effects related to patients knowing their treatment. The primary aim was to compare within-arm change from baseline to primary endpoint (the patient improvement) on the HAMD-17. The authors also planned to compare blinded versus open-label trials within each treatment class to quantify the influence of blinding. The analysis focuses on outpatient, non-psychotic adult major depressive disorder and converts other depression scales into HAMD-17 equivalents for comparability.

The investigators pre-registered their search and analysis plan and searched PubMed in March 2024 for trials in adults (mean age between 18 and 65) of either open-label tADs (defined as new-generation antidepressants) or PAT using LSD, psilocybin, mescaline (including San Pedro/Peyote), or DMT/ayahuasca. Trials were excluded if conducted in inpatients, psychotic depression, or significant comorbidity (an exception was made for comorbid anxiety). Augmentation/combination trials and trials with run-in periods were excluded, except where a run-in phase itself met inclusion criteria and was treated as an open-label tAD phase. The team excluded open-label antidepressant trials with fewer than 100 participants but applied no minimum sample size for psychedelic trials. Two reviewers (BS and HB) independently screened and extracted data, resolving conflicts by consensus; missing data recovery included contacting study authors. Different depression scales (BDI-I/II, MADRS, HAMD-21, QIDS) were converted into HAMD-17 units; conversions between SDs of endpoint and change scores assumed a conservative correlation coefficient of 0.5. The primary outcome was the within-arm effect (change from baseline to primary endpoint in HAMD-17 units). Analyses used both pre-registered Bayesian and frequentist multilevel random-effects meta-analytic models that accounted for nested outcomes (different scales) and included baseline depression severity as a covariate with a fixed effect and random slope. Models testing the main hypotheses included a binary focal variable for treatment type (PAT vs open-label tAD) or for blinding (open-label vs blinded) as appropriate. Bayesian results are reported as posterior medians with 95% credible intervals (CrI95), and frequentist results as means with 95% confidence intervals (CI95); secondary analyses were conducted using frequentist models for computational efficiency. Additional procedural notes provided by the authors include handling of crossover data (extracting first-period data except for specific fixed-order designs), an explicit decision not to perform a formal risk-of-bias assessment because open-label trials are high risk by definition, and several deviations from the preregistration (for example, restricting 6–12 week primary endpoints to tAD trials only, changing the dependent variable to change scores, and simplifying random-effects structures when complex models did not converge). The included-trials set used for the primary analyses comprised 16 open-label tAD trials and 8 PAT trials from an initial pool of 619 records; an extended dataset added four near-miss PAT studies for robustness checks.

The primary dataset comprised 16 open-label tAD trials (n=9,751) and 8 PAT trials (n=548). Mean time from baseline to the primary endpoint differed between groups: 8.1±1.5 weeks for tAD trials and 3.4±2.2 weeks for PAT trials. Baseline HAMD-17 scores averaged 22.7±1.9 for open-label tAD trials and 21.3±3.9 for PAT trials. Hypothesis 1 (PAT vs open-label tAD): Bayesian models estimated the within-arm change for open-label tAD at approximately −12.5 HAMD units (CrI95 around −12.9 to −12.2; SMD ≈ −2.7) and for PAT a very similar magnitude (SMD ≈ −2.6). The posterior mean for the PAT minus tAD difference was βPAT−tAD ≈ 0.25 HAMD units favouring tAD, and the posterior probability that PAT reduced depression by ≥3 HAMD units more than open-label tAD was only 0.2%. The posterior probability that the true difference lay within ±3 HAMD units (the region of practical equivalence, where 3 HAMD units was pre-specified as the minimal clinically important difference) was 99.1%. An extended dataset produced a similar estimate (βPAT−tAD=0.43, CrI95 [−0.17, 2.55]). In frequentist models the difference was likewise non-significant; a reported frequentist estimate for the between-treatment difference was ~0.3 HAMD units favouring open-label tAD (CI95 [−1.39, 1.98], p=0.730). Sub-analyses comparing PAT with SSRIs, SNRIs, and non-SSRI/non-SNRI tADs found no significant differences. Hypotheses 2 and 3 (blinded vs open-label within treatment classes): For tAD, the Bayesian estimate for blinded within-arm change was about −12.3 HAMD units (SMD ≈ −2.6) and for open-label roughly −12 HAMD units; the frequentist model found a small but statistically significant advantage for open-label tAD over blinded administration with βblind−OL = 1.29 HAMD units (CI95 [0.07, 2.51], p=0.038). The authors note this corresponds to roughly half the pre-specified MCID and that the entire CI95 falls within ±3 HAMD units, implying the magnitude is practically negligible. For PAT, neither the Bayesian nor frequentist models found a significant difference between blinded and open-label trials (frequentist βblind−OL ≈ 0.45, p=0.738), though the CI95 intersected the ±3 HAMD unit region so formal equivalence was not established. Sensitivity analyses using the extended dataset did not materially change these results. Across both treatment classes the within-arm improvement from baseline to primary endpoint was approximately 12 HAMD units, a magnitude the authors describe as both statistically and clinically large.

Williams and colleagues interpret their main finding as a failure to confirm the preregistered hypothesis that PAT would be clinically superior to open-label tAD by at least the pre-specified MCID of 3 HAMD units. Instead, both treatments produced similar, large within-arm symptom reductions of roughly 12 HAMD units, and the between-treatment difference was negligible (~0.3 HAMD units) and statistically non-significant. The authors emphasise that this comparison—PAT versus open-label tAD—was chosen to equalise the advantage that knowledge of allocation can confer, reasoning that open-label comparisons better reflect how medicines are used in routine clinical practice. To reconcile their null finding with prior reports that PAT shows much larger between-arm effects than tAD, the investigators propose two contributors. First, open-label administration of tADs appears to modestly improve outcomes relative to blinded administration (about 1.2 HAMD units on average), which represents some benefit of patients knowing their treatment. Second, pooled evidence suggests placebo response is substantially suppressed in PAT trials (the authors cite an estimated 4.0 HAMD unit lower placebo response), producing an inflated between-arm treatment–placebo difference for PAT. Summed together, these two effects approximately account for the 5 HAMD unit apparent advantage of PAT in between-arm comparisons. The authors describe this suppressed placebo response in PAT as a potential 'knowcebo' effect arising when participants realise they are in the control arm and experience disappointment, and they note that in some PAT trials placebo arms worsened. The discussion acknowledges several limitations the authors report. Some PAT trials enrolled treatment-resistant depression (TRD) cases while no tAD trials did, but baseline severity was controlled for and sensitivity analyses excluding TRD-only PAT trials did not change the result. PAT trials are not fully unblinded in the mathematical sense (correct-guess rates are reported near 95%, not 100%), so a residual unblinding imbalance could remain, though it is likely much smaller than in a blind-versus-blind comparison. The comparison is imperfect because PAT typically includes psychotherapy while the tAD comparators were pharmacological only; the authors note no trials of open-label tAD plus therapy were available for comparison. They also underline that their analysis focused solely on symptom reduction: other outcomes such as social functioning, connectedness, meaning in life, adverse events, and longer-term follow-up metrics may differ between interventions and were not synthesised here. Finally, the authors state they did not perform a formal risk-of-bias assessment because open-label trials are inherently high risk. In their concluding interpretation, the investigators present a tempered view: PAT produces large within-arm improvements that justify cautious optimism, but PAT was not superior to open-label tAD under their equal-unblinding comparison, underscoring the influence of blinding integrity on apparent efficacy and calling for measured expectations about PAT's comparative advantage.