November 8, 2013.
This is a presentation from the International Psycho-Oncology Society Conference in Rotterdam, November 8, 2013 invited by the Early Career Professionals Special Interest Group.* I am grateful for such a relaxed opportunity to speak my mind about some issues that junior researchers in psycho-oncology, like those in many fields, are facing. Senior members of the field have failed you. We need you to undo some of the damage that is being done.
As you enter the field, recognize that you are different from cohorts of researchers who have come before you. On the one hand, you are more methodologically and statistically sophisticated. You are also more digitally savvy, although I am sometimes bewildered by how little you as yet take advantage of the resources of the Internet and social media.
On the other hand, you face new accountability and pressures in terms of the monitoring of the impact factor of journals in which you publish, as well as you having to adhere to reporting standards and preregister your clinical trials before you even run the first patient. Researches who came before you had it easier in these respects.
I’ve done this kind of talk before and I recognize there is an expected obsolescence to what I present. I recall way back when I was junior person in the field, senior faculty warned me not to start using email because it was a total waste of time and inferior to communicating by snail mail. I am sure that much of the advice being offered to you is just as valuable and soon to be obsolete. And, similarly, many of the tools and strategies you will need to acquire first seem a waste of time.
Five years ago, I would have encouraged you to get more comfortable communicating about your work and even self-promoting. I would have suggested you use the now obsolete means, listserves to do so. I would have encouraged you to challenge the gross inadequacies of peer review by writing letters to the editor, which also have the advantage of cultivating critical skills better than journal clubs do. Both listserves and letters to the editor are now obsolete, but the ideas behind these recommendations still hold, maybe even more. You just have to pursue these goals differently and certainly with different tools.
As for myself, I’m undergone a lot of changes in the past five years. Some of my best recent papers have been written with authors gathered from the Internet, often without me first meeting all of them. I was honored that one of these papers won the Cochrane Collaboration’s Bill Silverman Prize, which I guess makes my co-authors and myself certified disruptive innovators.
I now tweet, blog, use Facebook, and champion open access publishing. Later in this talk, I will provide the exciting details of the launch of a trial of PubMed Commons. I had been afraid of having to observe an embargo on discussing this. But fortunately the shutdown of the US federal government ended, and PubMed Commons was launched just in time for me to talk about it in this presentation.
Tweeting and blogging are not distractions or alternatives to writing peer-reviewed papers, they can become the means of doing so. Tweets may grow into blog posts, then a series of blog posts, and eventually even a peer-reviewed journal article. No guarantees, but looking back, that’s how a number of my peer-reviewed papers have developed.
On the other hand, the process can work in reverse. Blogging and tweeting about recent and forthcoming papers is a very important part of how to be a scholar and how to promote yourself in the current digital moment.
Here are some examples of me self-consciously and experimentally promoting recent papers with blogging.
- Is psychotherapy for depression any better than a sugar pill?
- Cows in the rain: the colon strategy for constructing a title for a scientific paper
- Do rising rates of antidepressant prescription translate into lower rates of suicide? Evidence from 29 countries.
My first bit of advice to junior investigators is figure out where such action is occurring. The form and format it takes is constantly shifting. Observe, experiment, and get involved, consistent with your own comfort level. Remain lurking if you’d like, reading blogs, and occasionally expressing approval clicking “like” or “favorite” until you are ready for get more involved.
I invite all of you to join me in participating in disruptive innovation. On the other hand, I realize this is not for everyone, and so I will spell out alternative low road.
The state of the field being what it is, offers clear opportunities for you to conform, and play the game according to the rules that work. Many of you will do so and some of you can rise to the top of a mediocrity.
My second bit of advice is that if everyone likes your work, you can be certain that that you are not doing anything important. That sage advice I got from Andrew Oswald.
The behavioral and social sciences are a mess. We have four or five times the rate of positive findings relative to some of the hard sciences, and I don’t think is because our theories and methods are more advanced.
The field of psycho-oncology is particularly a mess, as seen in rampant confirmation bias and many of our widely acclaimed papers presenting evidence that is interpreted in ways that are exaggerated or outright false. The bulk of intervention studies in psycho-oncology are underpowered and the flaws in their designs provide a high risk of bias. Studies consistently obtain significant results at an impressive, but statistically improbable rate.
At the heart of the special problems of the field is the consistent subordination of a commitment to evidence-based science to the vested interests of those who want to promote and secure opportunities for the clinical services of their professions, regardless of what the evidence suggests. This is most notably seen in the relentless promotion of screening for distress in the absence of evidence that actually improves patient outcomes.
Ultimately, data will provide the basis for deciding whether screening is a cost-effective way of improvements and whether it represents the best use of scarce resources. I think that the evidence will be negative. But I am more worried about the lasting effects on the credibility and integrity of a field in which editing and peer review have been so distorted by the felt need to demonstrate that screening has benefit.
Many celebrated findings in the field of psycho-oncology are really null findings, if you carefully look at them.
- Spiegel, D., Kraemer, H., Bloom, J., & Gottheil, E. (1989). Effect of psychosocial treatment on survival of patients with metastatic breast cancer. The Lancet, 334(8668), 888-891.
- Fawzy, F. I., Fawzy, N. W., Hyun, C. S., Elashoff, R., Guthrie, D., Fahey, J. L., & Morton, D. L. (1993). Malignant melanoma: effects of an early structured psychiatric intervention, coping, and affective state on recurrence and survival 6 years later. Archives of General Psychiatry, 50(9), 681.
- Antoni, M. H., Lehman, J. M., Klibourn, K. M., Boyers, A. E., Culver, J. L., Alferi, S. M., … & Carver, C. S. (2001). Cognitive-behavioral stress management intervention decreases the prevalence of depression and enhances benefit finding among women under treatment for early-stage breast cancer. Health Psychology, 20(1), 20.
- Andersen, B. L., Yang, H. C., Farrar, W. B., Golden‐Kreutz, D. M., Emery, C. F., Thornton, L. M., … & Carson, W. E. (2008). Psychologic intervention improves survival for breast cancer patients. Cancer, 113(12), 3450-3458.
There are important negative trials of supportive expressive therapy and expressive writing being kept hidden in file drawers. Just search clinicaltrials.gov.
Zombie ideas and tooth fairy science still hold sway in the literature and win media attention. I have in mind the notion that psychological interventions can extend the lives of cancer patients and exaggerated ideas about the mind holding sway over the body and defeating cancer.
You are entering a system of publication and awards that is not working fairly. Papers appear in ostensibly peer reviewed journals without adequate review. There’s rampant cronyism in opportunities to publish and widespread sweetheart deals as to whether authors have to address concerns raised by reviewers. There is sandbagging of critics and negative findings. It is an embarrassment to the field that authors of flawed ideas are able to suppress commentary on their work and censor criticism.
The respected, high impact Journal of Clinical Oncology is particularly bad when it comes to psychosocial studies. It shows consistently flawed peer-review, the influence of sloppy editorial oversight, and serious restrictions on commenting on its miscarriage of the review process. Feeble post publication peer review is continually handicapped and silenced. I believe that journal has an ethical responsibility to identify to its readers which articles have evaded peer review and to announce that authors of published papers can exercise veto over any criticism or negative commentary.
If you want to take the low road, you have lots of opportunities to succeed.
- Pick a trendy topic.
- Don’t be critical of the dominant views, even if you see through the hype and hokum.
- Use biological measures, particularly ones that can be derived from saliva, even if they have no or unknown clinical significance.
- Report positive findings, even if you have to spin and torture and suppress data.
- No matter what your results, in your discussion section claim they confirm the dominant view and reaffirm that view, even if it is irrelevant or contradicted by your findings.
When you design studies, have lots of endpoints that you can always ignore later. Pick the one to report that makes your study look best. A lot of the positive findings in literature cannot really be replicated, but you can always appear to do so by pushing aside the results of primary analyses, and favor unplanned secondary and subgroup analyses. If necessary, construct some post hoc new outcome measures you didn’t even envision when you originally designed your study. Prominent examples of these strategies can readily be found in the published literature.
Many of you will do all this, wittingly or unwittingly following the advice and example of your advisors, but you can become more proficient in pursuing this low road.
Alternatively, I invite at least some of you to take the high road and join me and participate in disruptive innovation. Again, it’s not for everyone.
Blog, and if you’re not ready to consistently post your own, join in a group blog. I highly recommend groups like Mental Elf, where you can take turns offering critical commentary on recently published papers.
If you are not ready to blog, you can tweet. You can selectively follow those on Twitter who show they can offer you both fresh new ideas with which you would not otherwise come into contact, as well as a filtering out of much that is hype, hokum and sheer nonsense.
- Join PubMed’s Revolution in Post Publication Peer Review
- What you ought to know about PubMed Commons
- Science buzz and criticism get a powerful boost
- Enter PubMed Commons
As long as you have a paper published in PubMed, even a letter to the editor, you can secure an invitation to comment on any article that has appeared in PubMed. You can have others “like” or add a response to your comment, and you to theirs as part of a continuing process of post publication peer review. With PubMed Commons, we’re taking post publication peer review out of the hands of editors who so often have aggressively and vainly taken control of a process that should be left with readers.
I’m asking you to join with me in pursuing a larger goal of creating a literature that is an honest and reliable guide for other researchers, clinicians, patients, the media, and policymakers as to the best evidence. Let’s work together to create a system where review process is transparent and persists for the useful life of a work. For this last point, I give thanks to Michael Eisen, cofounder of PLOS one and disruptive innovator extraordinaire.
*Special thanks to
Claire Wakefield, Michelle Peate, University of NSW,Sydney Australia
Kirsten Douma and Inge Henselmans, Academic Medical Center Amsterdam, The Netherlands
Wendy Lichtenthal, Memorial Sloan-Kettering Cancer Center, New York City