History repeating: guidelines to address common problems in psychedelic science

The paper is a narrative, synthesis‑style contribution rather than a formal systematic review: the extracted text does not report a prespecified search strategy, databases searched, or formal inclusion criteria. Instead, the authors draw on recent clinical trials, meta‑analyses, methodological critiques and exemplar studies to illustrate recurring problems in the literature. They classify identified problems into three tiers based on the perceived ease of remediation (easy, moderate, hard) and relate each problem to one or more of four validity domains: internal, external, construct and statistical conclusion validity. To substantiate their points the authors cite specific trial examples, meta‑analytic findings and audits (for instance, analyses of adverse event reporting and trial registries), and refer to prior methodological guidance and reporting standards (for example CONSORT and registered reports). Proposed remedies combine methodological practices (preregistration, blinded analysis, active placebos, multiverse analyses), organisational measures (team and multi‑centre science, independent arbiters), and resource‑related solutions (larger samples, longer follow‑up, more public funding). The text also summarises a practical checklist intended for diverse stakeholders, but the extraction does not include a formal methods appendix or a reproducible search log.

Van Elk and colleagues identify ten central challenges in contemporary psychedelic research and present them grouped as easy, moderate and hard problems. The easy problems are (1) invalid statistical inferences and questionable research practices, (2) conflicts of interest (COIs), and (3) inadequate reporting of safety and adverse events. Examples illustrating invalid inferences include RCTs or open‑label studies whose abstracts or conclusions overstate efficacy relative to the presented data, outcome switching and flexible selection of analysis time points, and failures to correct for multiple testing across many outcomes. The authors note evidence for publication bias in clinical psychiatry more broadly and highlight that underpowered studies combined with selective reporting inflate apparent effect sizes. On COIs, they cite analyses showing trials with declared COIs are more likely to report positive results and note instances where a majority of trial authors had financial ties; they argue for industry‑independent oversight and transparent public disclosure. For safety, the authors report systematic underreporting—for example, an analysis found 41.5% of serious adverse events listed on clinicaltrials.gov for esketamine were not reported in published articles and 88 of 94 of those events occurred in esketamine arms. They also document instances of serious adverse outcomes in psychedelic trials (for example, hospitalisations in an ayahuasca trial: 4 of 15 participants in the experimental group) and examples of participant harm or boundary violations in therapy settings. Moderate problems comprise (1) frequent absence of control groups, (2) small sample sizes, (3) selection bias, and (4) short or absent long‑term follow‑up. The authors emphasise that many clinical studies are open‑label, so observed improvements (for instance a reported 60% recovery at 12 months in one open‑label psilocybin study) cannot be causally attributed to treatment without comparison to expected natural remission (they reference a 12‑month spontaneous remission estimate of around 53% in untreated depression). Small samples are widespread in clinical and neuroimaging studies, yielding imprecise and potentially inflated effects; the authors note that some highly cited neuroimaging datasets include fewer than 10 participants and have been re‑analysed repeatedly. Selection biases arise from restrictive inclusion criteria (for example excluding comorbidities, suicidal ideation or ongoing medication), self‑selection by motivated or psychedelic‑experienced participants, and predominantly WEIRD samples; the phase III MAPS trial is cited as including only 26% of screened treatment‑seeking participants. Short follow‑up intervals are common—examples include studies with follow‑up measured in hours, days or a week—so claims of sustained treatment efficacy lack long‑term evidence. The hard problems are (1) the breaking‑blind problem, (2) large expectancy and placebo effects, and (3) an unclear mechanism of action. The authors document that blinding is often ineffective: participant blinding was formally assessed in only 8 of 81 studies in a cited review, and examples show high accuracy of participant guesses (for example one trial where 15 of 19 guessed placebo correctly and 12 of 15 guessed active correctly). They explain that breaking blind threatens internal validity via biased outcome assessment, differential therapist behaviour and boosted patient expectancy. On placebo and expectation effects, the paper highlights heightened suggestibility under psychedelics and a cultural “hype” that can inflate expectations, cause demotivation or nocebo effects in control arms, and even bias participant self‑reports. Regarding mechanisms, the authors review competing hypotheses (neuroplasticity, increased brain entropy, loosening of priors, and the therapeutic role of subjective experiences) and stress that the psychotherapeutic component is under‑studied—only 3 of 21 trials compared structured psychological support to a minimal support condition. They note the field lacks standardised psychotherapeutic protocols and therapist training, and that the development of non‑psychedelic psychoplastogens could help disentangle pharmacological from experiential mechanisms. Across sections the authors offer empirical and procedural illustrations (for example outcome‑switching, lack of blinding assessment, underreporting of adverse events) and quantify some issues when possible. They also compile practical remedies tied to each problem class, ranging from routine transparency measures to more resource‑intensive solutions such as multi‑site high‑powered trials and long‑term follow‑ups.

Van Elk and colleagues conclude that the combined impact of the identified threats makes it premature to draw firm conclusions about the safety and efficacy of psychedelic therapies. They argue that many threats co‑occur in the same studies—small, selective samples with short follow‑up, weak or absent controls, ineffective blinding, pronounced expectancy effects and financial COIs—so that bias accumulates and valid causal inference is undermined. The authors position these concerns relative to earlier critiques in the field, noting that deficiencies such as breaking blind, inadequate safety reporting and publication bias have been flagged previously but remain insufficiently addressed. Key limitations and uncertainties acknowledged by the authors include the lack of systematic, long‑term safety data; the field’s reliance on small, often non‑representative samples; the difficulty of maintaining blinding in experiential interventions; and incomplete reporting and preregistration practices. They stress that many proposed mechanisms remain speculative and under‑tested, and that the psychotherapy component—its content, standardisation and therapist training—remains poorly characterised. The authors also highlight practical constraints: remedying several problems requires greater resources, multi‑centre coordination and cultural change in reporting and peer review. Implications they discuss focus on concrete research and policy changes. These include routine preregistration and registered reports, publishing null results, full reporting of all measures and adverse events (with standardised definitions), blinded outcome assessment and blinded analysis, formal assessment and reporting of masking efficacy, independent arbiters for adverse‑event attribution, reduction of COI influence via industry‑independent oversight, use of active placebos or dose‑comparison designs to mitigate breaking blind, and longer follow‑up (preferably up to 1 year) with larger, more diverse samples. The authors recommend adopting open science tools (for example multiverse and many‑analyst approaches), investing in team and multi‑site science, and developing standardised psychotherapy manuals and therapist training. They also present a checklist intended to help stakeholders vet study quality. Overall, the authors urge caution about current hype, while leaving open the possibility that future, more rigorous studies may demonstrate clear benefits for specific patient groups.

The paper ends with a practical roadmap and a checklist aimed at improving rigor in psychedelic clinical science. Van Elk and colleagues call for studies that address multiple validity threats simultaneously—appropriate control conditions, larger and more representative samples, long‑term follow‑up, transparent preregistration, standardised adverse‑event reporting, formal assessment of blinding effectiveness and independence from commercial conflicts. They stress that transparency is the minimum requirement for progress and that many proposed solutions are already available; resolving the harder problems will additionally require methodological innovation and greater resources. The authors conclude with a cautionary note against uncritical enthusiasm: given the current evidence base, strong caution is warranted and history‑repeating patterns of premature optimism should be avoided while the field pursues more rigorous, transparent and adequately powered research.