Making mistakes and owning them: How I submitted corrections to published papers and (currently) live to tell the tale

 

838-02493432

by Dr. Lauren Robinson

It’s the nightmare scenario: you look back at an old bit of code and realize you’ve made a mistake and, to make matters worse, the paper has already been published. This year I lived that nightmare scenario. I had shared my code only to discover that a variable that should have been reverse scored (which boils down to multiplying the number by -1), wasn’t. It was a minor oversight that I’d made as a 1st year PhD student learning new statistics, I hadn’t caught the mistake until now, and, worse still, the code had been used in two papers I wrote simultaneously. I considered changing my name and hiding but as I had a postdoc and my mother claims to like me, I figured it was better to keep my current identity.

‘…the right decisions don’t come without risk….’

Reaching out to the senior author we knew there was only one solution: We had to redo the statistics and submit corrections. As an early career researcher, I was panicked. What if the results were drastically different, was a retraction (possibly two) in my future? Fear aside, a mistake was made, we had to own it, and if we were going to believe in scientific integrity then we had to show ours. It’s been my experience that the most difficult decisions, the ones that I’m truly afraid to make – those are the decisions I know to be right. But the right decisions don’t come without risk and I can’t pretend that I wasn’t, and continue to be, worried that not everyone would see this as a minor mistake. Science is competitive and the feeling of having to be flawless, particularly at this phase of my career, is a weight. As a woman in science I already have to fight to be taken seriously, to be seen as competent, and I had committed a sin, I had made an honest mistake that had been published, twice. Before I could find out the results of my mistake on my career, I had to find out their impact on my papers.

‘As a woman in science I already have to fight to be taken seriously, to be seen as competent…’

I somehow survived three painful hours while I waited to finish work at my postdoc and could get back to where I kept the study data. Upon sitting at my desk (liquid courage in hand) I redid the stats, anxious to find the results. Now look, I’m no slouch with numbers, I know what multiplying by -1 does to them, but panic overrode sense in that moment and I needed to see to believe. First paper: Flipped the direction of effect on a non-significant variable that remained that way. Okay, fairly minor, just requires that the journal update the tables. Second paper: Again, the only thing that changed was the direction of effect, though this variable had been and still was significant, means we had to adjust the numbers, a line in the abstract, and three sentences in the results. Not great, but as variables go it hadn’t even rated being mentioned in the discussion.

Okay, okay, okay (deep breaths, bit more whisky), this could be so much worse I told myself. I screwed up but hey, everyone makes mistakes, I was learning something new, I should’ve have caught it earlier, but it was caught now. Onto the next step, making the corrections, contacting coauthors, and letting the journals know. Time to really live by our ideals. But first! Another moment of panic while I wondered if I had made the same mistake in my two newest papers. Opening code, reading through, and…no, I hadn’t made the mistake again. Somewhere along the way I had clearly learned how to do these statistics correctly, I just hadn’t caught it while I was working on these two papers and had copy-pasted the code across them. Good news, I am in fact capable of doing things correctly.

‘I had lived my nightmare and it felt, as least in this moment…completely survivable…’

Writing the email to my coauthors wasn’t something that I was particularly looking forward to. “Oh hey fellow researchers that I respect and admire, I screwed up and am going to let the journals and the world know. PS, please don’t think less of me and hate me. Okay, thanks.” While that’s not what I wrote, that’s what it felt like. An admission of imperfection, shame, guilt, a desire to live under a rock. However, I’ve been blessed with caring and understanding collaborators, each of whom was extremely supportive. Next, I sent an email to the journals explaining the mistake and requesting corrections be published. Each journal was understanding and helped us write and publish corrections and that was it, it was done. I had lived my nightmare and it felt, as least in this moment…completely survivable. I had imagined anxiety and panic and battling my own shame and guilt. This…this was a feeling of stillness that I was not prepared for.

Prior to contacting the journals and writing this blog, I asked myself how much this would hurt my career. Would a small mistake cost me my reputation, respect, and future in the science I’d already sacrificed so much for? Would writing this blog and openly speaking to the fact that I had made a mistake only further the potential damage to career and respect? Would a single mistake, done at the beginning of my PhD and not since repeated, mean that others didn’t trust my science and statistics, not want to work with me? Would I trust my own skills, and more importantly, myself, again? There was so much uncertainty and so little information available on this experience, yet mistakes like this must happen more than we think, they just go unspoken.

‘…genuine mistakes? We have to make those acceptable to acknowledge, correct, even retract, and speak about, to learn and move on from.’

This, this is the crux of a problem in science, there are unknown consequences of acknowledging and speaking openly about our mistakes and, by failing to do so, we only further increase the chance that mistakes go uncorrected. Let’s hold those that perform purposeful scientific misconduct accountable, but genuine mistakes? We have to make those acceptable to acknowledge, correct, even retract, and speak about, to learn and move on from them. Those who don’t learn from their mistakes? Well, they may be doomed to face the consequences. As a note, if we’re going to move towards openness and transparency in science then we need to be particularly careful that those in underrepresented groups aren’t unfairly punished or scrutinized for admitting and speaking about mistakes as these groups are already under a microscope and face unique and frustrating challenges. We cannot allow openness and transparency to be used as one more excuse for someone to tell us no, not if science is to diversify and progress.

‘What kind of person and scientist do I want to be?’

Of all the questions I asked myself, deciding to write this post came down to one: What kind of person and scientist do I want to be? As an animal welfare scientist, I have long believed in being transparent and open in science, I realized that’s who I am as a person as well. Living by my ideals meant not only correcting my mistake but also talking openly and frankly about it. These choices, challenging as they may have been, are the right ones. To err is human and luckily for me I have divine friends, mentors, and colleagues that forgive me my mistakes and sins. I believe that we should all be so lucky and that mistakes should be openly and transparently discussed. For now, I live to science another day and look forward to the challenges, mistakes (which I intend to catch prior to publication), and learning that come with it.

For those interested in working with me (imperfections and all) when my current postdoc ends this January, feel free to get in touch via ResearchGate (https://www.researchgate.net/profile/Lauren_Robinson7) or Twitter (https://twitter.com/Laurenmrobin).

Links to published corrections:

http://psycnet.apa.org/buy/2016-39633-001

https://www.sciencedirect.com/science/article/pii/S016815911830193X

Read about Lauren’s fascinating research (with lots of monkey photos!) into animal welfare and animal behaviour here.

Monkeys, happiness, and winning debates

By: Lauren Robinson

fgd

Monkeys you say? Tell me more.
Jane Goddsfodall once asked me, “Was it you, was it you who put a monkey in the loo?!” If you’re wondering, no it was not. Thankfully she was referring to a poster rather than an actual monkey. Yet, I take it as a point of pride to have been asked and to be working in a field where I regularly get close enough to monkeys to have been slapped by one (truthfully
it’s more than that but I’ve lost count). It was my fault; I was observing the monkeys and how dare that require looking at them. Primatology, the study of nonhuman primates ckvsn(monkeys, chimpanzees, gorillas, etc.), is not for the faint of heart or slow of reflex. It’s a field I fell in love with (I mean look at the baby Sulawesi macaque on the right, it has a heart shaped bum!) during my Masters dissertation studying Japanese macaques (see: photo above of suckling infant).
There are a lot of different things about primates that I could study (having anecdotally and painfully observed their speed) and the area of primatology that I am most interested is primate welfare. What do I mean when I say “welfare”? Well, I use a very broad definition and define welfare as the mental and physical health of an animal. In order to study animal welfare, researchers, such as myself, use methods that cross between the fields of animal behaviour, psychology, and physiology, among others. We observe animals for unusual behaviours, assess them for increased stress levels, and look for signs of injury and illness. Animal welfare science is a growing field and, with pioneers such as Marian Dawkins (Dawkins, 1980) and Temple Grandin, it is one with multiple strong and well known female scientists to look up to.

Enough of that, let’s talk about me.

My research focuses on the individual animal, which is why I’m currently in a psychology department studying individual differences in animal personality. I take the approach that an animal’s welfare is an individual experience and we need to understand the individual differences associated with it, specifically personality. Most of us have a general idea of what personality is, especially when asked to list the traits we love or hate about other people.fvsdknv Over the last couple decades it has become more accepted to talk about animals having personality as well (Gosling, 2001). It’s rare that someone describes their dog as “consistently approaching unfamiliar people and animals in a nonaggressive manner”. Instead, they say their dog is friendly and sociable. In the case of my dog Juneau (left), we describe her as eccentric and too clever for her own good. While some scientists may be on the fence about animal personality my experience has been that the public isn’t, they get it and they believe that animals have it. In order to understand primate welfare I look for the personality differences that influence it, which is the focus of my research. I want to know if certain personality traits make animals more likely to be do well in captivity, in the same way that people with certain personality traits do better in life. For example, more extraverted and sociable people tend to be healthier and happier (Costa & McCrae, 1980; Deary, Weiss, & Batty, 2010).

I started as PhD student at the University of Edinburgh in 2013 working with Dr Alex Weiss. Alex and I have different scientific backgrounds and naturally, we disagree on some things. Key among the disagreements we’ve had over the years is the difference between welfare and happiness in animals. Alex felt that if an animal had everything it needed in captivity (safety, food, companionship, good physical health) then it had high welfare. He noted then even when animals have all these things they can be unhappy, which to him meant that happiness and welfare did not necessarily go together for animals. Alex based this on the observation that some people appear to have everything they could want for (money, friends, shelter) but aren’t happy. I felt differently. As I said earlier, I take the approach that an animal’s welfare is an individual experience. Therefore, if the animal appears to have everything it needs but is still unhappy then, by definition, that animal has reduced welfare. How to find out who is right though? To the Batcave! Yeah, sadly not. Instead it was off to Google Scholar to research and come up with a way of testing my hypothesis that primate happiness and welfare were one and the same.skndcs

What I found was a great article by Franklin McMillan (2005), who says that there are five main things that influence an animal’s welfare: mental stimulation, physical health, stress, social relationships, and control of physical and social environment. When psychologists look at human happiness they typically use questionnaires (Sandvik, Diener, & Seidlitz, 1993) and there is a questionnaire to measure primate happiness (King & Landau, 2003) but animal welfare scientists don’t typically use questionnaires as there are concerns about the accuracy of ratings.
This hasn’t been well studied though so I took McMillan’s five things and created a questionnaire for staff familiar with animals to fill out. To test if it worked I took my welfare questionnaire and the primate happiness questionnaire and sent them out to zoos and research facilities.

 

Well, you win the debate or not?

Currently, I’m working on finishing my PhD (send whisky for my woes) and have used the questionnaires to study welfare and happiness in three species: Brown capuchins, chimpanzees, and rhesus macaques. First thing I found was that staff familiar with the animals I studied were really good at rating animal welfare. They agreed to the same degree that people do when they rate their friends and family member’s personality. The next thing I found, much to my own happiness, was that welfare and happiness are really one and the same in those three species (I won!). Three species and some pretty compelling results (Robinson et al., 2016; in review; in prep) were convincing enough to get my supervisor to rethink his opinion on happiness and welfare. Did you catch that? The PhD student actually won one! Sure, Alex has taught me a billion things to this one thing I taught him, but I will take it.

So, what about personality and welfare? Personality does influence primate welfare, similar to what we see in people. Animals with certain personality traits have higher happiness and welfare. The brown capuchins that were more sociable, assertive, attentive, and more emotionally stable were those that had higher happiness and welfare. For chimpanzees, seems to be about extraversion and emotional stability. Rhesus macaques, it’s all about confidence; those with more confident personalities had higher welfare and happiness. It’s my hope that now we know more about welfare, happiness, and personality we can use this information to improve the lives of animals. This could be done by using the questionnaire as another tool for measuring animal welfare or by trying to provide more care for animals with personality traits that tend to be related to unhappiness.

Upon reflection…

bfkjdfWhile my research results are better than I could have hoped for the best part of this research were the experiences I gained along the way. As I get to the end of my PhD, and this post, I’m starting to put thesis together and I’m all about reflection about my past three years (when I’m not panicking about the next three). I’ve gotten to study three species of primates, worked in zoos and research facilities (many of you will have thoughts on animals in research, I get that but don’t have room to get into that topic without a separate post), and collaborated with tons of amazing researchers. All of that is fantastic but let’s be honest, the monkeys are the best part.

You may be wondering what monkeys are like. I’ve worked directly with over 100 macaques and there is no doubt in my mind that each one is an individual with very different personality. Some are funny, some are playful, some are grumpy, and plenty are aggressive (learned that the hard way). While I hope that I’ve piqued your interest in primates, their amazing personalities, and their welfare I would be remiss if I didn’t state that primate are not pets (see resources below). I know I’ve spoken of my passion for working with primates but only in professional manner and environment and I never treat them as less than they are, which is wild animals. Primates are far too clever and socially complex to be kept as pets. Anyone that tells you otherwise is flat out wrong. No exceptions to the rule, no anecdotes, no to primates as pets.

Having said my warning, I will finish by acknowledging that while there are a lot of words to describe what I do (science, animal welfare, primatology) the one that always stands out to me is ‘privileged’. Working with primates is a privilege. Studying and working to improve their welfare is the best way I know to show my appreciation of that privilege.

If you’re interested in learning more about primate welfare, there are some public engagement resources that I’m a big fan of:

NC3Rs macaque page

Online tour of German Primate Center

Why monkeys shouldn’t be pets

Animal welfare legislation resources

Costa, P. T., & McCrae, R. R. (1980). Influence of extraversion and neuroticism on subjective well-being: happy and unhappy people. Journal of Personality and Social Psychology, 38(4), 668–678.

Dawkins, M. S. (1980). Animal suffering: The science of animal welfare. Ethology (Vol. 114). New York: Chapman and Hall.

Deary, I. J., Weiss, A., & Batty, G. D. (2010). Intelligence and Personality as Predictors of Illness and Death: How Researchers in Differential Psychology and Chronic Disease Epidemiology Are Collaborating to Understand and Address Health Inequalities. Psychological Science in the Public Interest.

Gosling, S. D. (2001). From mice to men: What can we learn about personality from animal research? Psychological Bulletin, 127(1), 45–86.

King, J. E., & Landau, V. I. (2003). Can chimpanzee (Pan troglodytes) happiness be estimated by human raters? Journal of Research in Personality, 37(1), 1–15.

McMillan, F. (2005). Mental wellness: The concept of quality of life in animals. In Mental Health and Well-Being in Animals.

Robinson, L. M., Waran, N. K., Leach, M. C., Morton, F. B., Paukner, A., Lonsdorf, E., Handel, I., Wilson V. A. D., Morton, F. B., Brosnan, S., & Weiss, A. (2016). Happiness is positive welfare in brown capuchins (Sapajus apella). Applied Animal Behaviour Science, 181, 145-151.

Robinson, L. M., Altschul, D., Wallace, E. K., Ubeda, Y., Machanda, Z., Slocombe, K. E., Llorente, M., Leach, M. C., Waran, N. K., & Weiss, A. (In press). Chimpanzees with positive welfare are happier, extraverted, and emotionally stable. Applied Animal Behaviour Science. 10.1016/j.applanim.2017.02.008.

Robinson, L. M., Capitanio, J. P., Leach, M. C., Waran, N. K., & Weiss, A. (In prep). The influence of personality on rhesus macaque health, welfare, and happiness.

Sandvik, E., Diener, E., & Seidlitz, L. (1993). Subjective Well-Being – the Convergence and Stability of Self-Report and Non-Self-Report Measures. Journal of Personality, 61(3), 318–342.