Discordance between treatment and follow-up phases (part 3 in a series)
(Click here for part 1)
Panacea was a goddess known for having a cure-all, which is why her name now means “a solution to every problem.” This series comprises a critique of an RCT, recently published in JAMA, in which the authors concluded that the treatment was something of a panacea.
Discordance between treatment and follow-up phases
Sometimes a study just doesn’t pass the sniff test. Maybe it isn’t anything overtly problematic, but you notice some data anomalies that cause you to question the authors’ conclusions. For me, one such study was the comparison of long-term cardiovascular outcomes in patients treated with short-term antidepressant therapy, which was published late last year by Kim and colleagues.1 I’ve written an overview of this study’s many problems here, but this page discusses the discordance between the direction of effect during the treatment and follow-up phases.
Let me explain:
Figure 1 shows cumulative incidence curves for the primary outcome, which was a composite of cardiovascular death, all-cause death, myocardial infarction, and percutaneous coronary intervention (PCI, a procedure to treat patients having acute cardiac events). The cumulative incidence curves show the percentages of patients who had any of the outcomes over the entire study period. As you can see, throughout most of the 11-12 year follow-up period, escitalopram appears to have a lower risk of the primary outcome compared to placebo, which was statistically significant. But, the odd thing about these data is that early in the study — when the patients were actually on treatment (and while they were still blinded to which treatment they were getting), their risk of having the composite outcome was actually higher for the escitalopram patients compared to placebo.
Figure 2 shows the same phenomenon for the myocardial infarction outcome, which was the only one of the 4 outcomes from Figure 1 that was statistically significant. (When this happens, we question whether the statistical significance of the composite outcome is completely driven by the one component that was significant.) Here, placebo appeared to out-perform escitalopram for more than two years!
A few months after the publication first came out, a letter to the editor from 3 researchers in the Netherlands was published that called into question some other data anomalies about the original graph presented in Figure 1.2 Kim and colleagues wrote a response to those investigators in which they said that the original curve had been created “inappropriately,” without giving any further explanation.3 They also submitted a corrected figure, shown in Figure 3. The corrected graph did not reflect the same phenomenon. However, the graphs for the 4 component outcomes were unchanged in the corrected paper, which meant that the difference in the composite outcome still appeared to be entirely driven by the myocardial infarction outcome, which appeared to show that placebo was better than escitalopram in the entire first 2 years.
There are some potential explanations for these anomalies. It is possible that there could be two different etiologies for myocardial infarction: one in the treatment period and the subsequent 1-2 years, and another in the rest of the follow-up period. For example, it’s possible that an unknown adverse effect of escitalopram is an acute and transient increase in the risk of myocardial infarction that goes away within 6-18 months after discontinuing treatment, and that could have been followed by a persistent benefit of having briefly treated the depression. Alternatively, the post-treatment period switch in myocardial infarction risk between the two groups, rather than being a function of the protective benefit of having previously and briefly been treated, might have been a function of a Hawthorne effect that was introduced when the patients were unblinded at 12 months. These seem to be an implausible explanations for their findings, and the investigators do not make this case. (In fact, they do not mention this switch in effects between the two periods at all.) Another explanation is that fabricated or altered data could produce these findings. There may be other potential explanations.
Nonetheless, these anomalies in combination with other limitations of the paper, strengthen my conviction that we cannot trust the study’s conclusion that short-term treatment of depression with escitalopram (less than 6 months) results in dramatic long-term benefits (up to 12 years), in terms of reductions in cardiovascular outcomes.
- Kim J-M, Steward R, Lee Y-S, et al. Effect of escitalopram vs placebo treatment for depression on long-term cardiac outcomes in patients with acute coronary syndrome: A randomized clinical trial. JAMA. 2018;230(4):350-7.
- Zimmerman FM, El Farissi M, Tonino PAL. Cardiac outcomes after treatment for depression in patients with acute coronary syndrome: Comment. JAMA. 2018;320(20):2151-2.
- Kim J-M, Steward R, Noon J-S. Cardiac outcomes after treatment for depression in patients with acute coronary syndrome: Reply. JAMA. 2018;230(20):2152-3.