Making mistakes and owning them: How I submitted corrections to published papers and (currently) live to tell the tale

 

838-02493432

by Dr. Lauren Robinson

It’s the nightmare scenario: you look back at an old bit of code and realize you’ve made a mistake and, to make matters worse, the paper has already been published. This year I lived that nightmare scenario. I had shared my code only to discover that a variable that should have been reverse scored (which boils down to multiplying the number by -1), wasn’t. It was a minor oversight that I’d made as a 1st year PhD student learning new statistics, I hadn’t caught the mistake until now, and, worse still, the code had been used in two papers I wrote simultaneously. I considered changing my name and hiding but as I had a postdoc and my mother claims to like me, I figured it was better to keep my current identity.

‘…the right decisions don’t come without risk….’

Reaching out to the senior author we knew there was only one solution: We had to redo the statistics and submit corrections. As an early career researcher, I was panicked. What if the results were drastically different, was a retraction (possibly two) in my future? Fear aside, a mistake was made, we had to own it, and if we were going to believe in scientific integrity then we had to show ours. It’s been my experience that the most difficult decisions, the ones that I’m truly afraid to make – those are the decisions I know to be right. But the right decisions don’t come without risk and I can’t pretend that I wasn’t, and continue to be, worried that not everyone would see this as a minor mistake. Science is competitive and the feeling of having to be flawless, particularly at this phase of my career, is a weight. As a woman in science I already have to fight to be taken seriously, to be seen as competent, and I had committed a sin, I had made an honest mistake that had been published, twice. Before I could find out the results of my mistake on my career, I had to find out their impact on my papers.

‘As a woman in science I already have to fight to be taken seriously, to be seen as competent…’

I somehow survived three painful hours while I waited to finish work at my postdoc and could get back to where I kept the study data. Upon sitting at my desk (liquid courage in hand) I redid the stats, anxious to find the results. Now look, I’m no slouch with numbers, I know what multiplying by -1 does to them, but panic overrode sense in that moment and I needed to see to believe. First paper: Flipped the direction of effect on a non-significant variable that remained that way. Okay, fairly minor, just requires that the journal update the tables. Second paper: Again, the only thing that changed was the direction of effect, though this variable had been and still was significant, means we had to adjust the numbers, a line in the abstract, and three sentences in the results. Not great, but as variables go it hadn’t even rated being mentioned in the discussion.

Okay, okay, okay (deep breaths, bit more whisky), this could be so much worse I told myself. I screwed up but hey, everyone makes mistakes, I was learning something new, I should’ve have caught it earlier, but it was caught now. Onto the next step, making the corrections, contacting coauthors, and letting the journals know. Time to really live by our ideals. But first! Another moment of panic while I wondered if I had made the same mistake in my two newest papers. Opening code, reading through, and…no, I hadn’t made the mistake again. Somewhere along the way I had clearly learned how to do these statistics correctly, I just hadn’t caught it while I was working on these two papers and had copy-pasted the code across them. Good news, I am in fact capable of doing things correctly.

‘I had lived my nightmare and it felt, as least in this moment…completely survivable…’

Writing the email to my coauthors wasn’t something that I was particularly looking forward to. “Oh hey fellow researchers that I respect and admire, I screwed up and am going to let the journals and the world know. PS, please don’t think less of me and hate me. Okay, thanks.” While that’s not what I wrote, that’s what it felt like. An admission of imperfection, shame, guilt, a desire to live under a rock. However, I’ve been blessed with caring and understanding collaborators, each of whom was extremely supportive. Next, I sent an email to the journals explaining the mistake and requesting corrections be published. Each journal was understanding and helped us write and publish corrections and that was it, it was done. I had lived my nightmare and it felt, as least in this moment…completely survivable. I had imagined anxiety and panic and battling my own shame and guilt. This…this was a feeling of stillness that I was not prepared for.

Prior to contacting the journals and writing this blog, I asked myself how much this would hurt my career. Would a small mistake cost me my reputation, respect, and future in the science I’d already sacrificed so much for? Would writing this blog and openly speaking to the fact that I had made a mistake only further the potential damage to career and respect? Would a single mistake, done at the beginning of my PhD and not since repeated, mean that others didn’t trust my science and statistics, not want to work with me? Would I trust my own skills, and more importantly, myself, again? There was so much uncertainty and so little information available on this experience, yet mistakes like this must happen more than we think, they just go unspoken.

‘…genuine mistakes? We have to make those acceptable to acknowledge, correct, even retract, and speak about, to learn and move on from.’

This, this is the crux of a problem in science, there are unknown consequences of acknowledging and speaking openly about our mistakes and, by failing to do so, we only further increase the chance that mistakes go uncorrected. Let’s hold those that perform purposeful scientific misconduct accountable, but genuine mistakes? We have to make those acceptable to acknowledge, correct, even retract, and speak about, to learn and move on from them. Those who don’t learn from their mistakes? Well, they may be doomed to face the consequences. As a note, if we’re going to move towards openness and transparency in science then we need to be particularly careful that those in underrepresented groups aren’t unfairly punished or scrutinized for admitting and speaking about mistakes as these groups are already under a microscope and face unique and frustrating challenges. We cannot allow openness and transparency to be used as one more excuse for someone to tell us no, not if science is to diversify and progress.

‘What kind of person and scientist do I want to be?’

Of all the questions I asked myself, deciding to write this post came down to one: What kind of person and scientist do I want to be? As an animal welfare scientist, I have long believed in being transparent and open in science, I realized that’s who I am as a person as well. Living by my ideals meant not only correcting my mistake but also talking openly and frankly about it. These choices, challenging as they may have been, are the right ones. To err is human and luckily for me I have divine friends, mentors, and colleagues that forgive me my mistakes and sins. I believe that we should all be so lucky and that mistakes should be openly and transparently discussed. For now, I live to science another day and look forward to the challenges, mistakes (which I intend to catch prior to publication), and learning that come with it.

For those interested in working with me (imperfections and all) when my current postdoc ends this January, feel free to get in touch via ResearchGate (https://www.researchgate.net/profile/Lauren_Robinson7) or Twitter (https://twitter.com/Laurenmrobin).

Links to published corrections:

http://psycnet.apa.org/buy/2016-39633-001

https://www.sciencedirect.com/science/article/pii/S016815911830193X

Read about Lauren’s fascinating research (with lots of monkey photos!) into animal welfare and animal behaviour here.

Advertisements

DNA Databases: Between Crime Fighting and Human Rights

by Lauren O’ Connell, University College Dublin.

DNA

Many of us remember learning about DNA from either science class during our school days, or perhaps, our favourite detective series or film. But what is DNA? How did it get to be used in the criminal justice system in the first place? Most importantly, how is it being incorporated, used and understood by the criminal justice system? This piece provides a short introduction to this area of law in Ireland.

DNA stands for ‘Deoxyribonucleic Acid.’ A sample of DNA can be extracted from our saliva, blood and bone for example. Each person’s DNA is structured differently, meaning that our DNA is unique to each of us alone.  DNA profiling was discovered in 1985 by Sir Alec Jeffries and his colleagues in Leicester, and allowed the development of a ‘DNA profile’ from a physical DNA sample. A DNA profile looks similar to a barcode and is a digital representation of a DNA sample. Following this discovery, DNA became a prominent feature in the investigation of crime.

DNA evidence is important in the context of a crime because it can allow for the identification of a specific person at a crime scene and can help to identify unknown bodies. If DNA is found at a scene and then matched with a suspect, it places the suspect at the scene. DNA evidence has been praised because it is often seen as objective, scientific evidence. This has been considered preferable to other forms of evidence such as witness statements which are often subjective and unreliable. Despite these benefits, a problem can arise if DNA is discovered at a crime scene but there are no suspects to test it against. This limits the ability of DNA to aid in an investigation as, although it was obtained from a crime scene, it cannot be compared with anyone. In light of this, the storing or banking of DNA profiles for comparison purposes became desirable for those investigation crimes. DNA storage allows a DNA profile generated from a crime scene sample to be tested against a range of profiles which have already been collected from a pool of people. This is where the central appeal of DNA databasing originated.

Forensic DNA databases organise and store DNA information for the purposes of criminal investigations, and to aid searches for missing or unidentified persons. Therefore, theyallow “rapid comparison” between profiles collected from crime scenes and profiles collected from people who are included in the database (Bieber, 2004: 29). Another frequently mooted (and often debated) benefit offered by DNA databases is the ability to deter people from committing crime, as criminals may have a heightened expectation of being caught.This claim has been disputed however, both because of the difficulties in actually measuring deterrence, but also because criminals may merely adapt to the new circumstances by becoming more forensically aware.The storage of DNA information, even limited information such as a profile, has attracted much debate, particularly in relation to human rights. For example, while databasing is efficient in terms of managing information, a database can also be used “to track, group and classify people with or without their acquiescence” (Jasanoff, 2010: xx). People who have their DNA profiles stored on a forensic DNA database lose privacy, freedom and autonomy, and may be reluctant to engage in active citizenship (such as in protests) given the ability to identify them (Jasanoff, 2010: xxii).

The Irish DNA Database System

Ireland recently incorporated the DNA Database System into law, under the Criminal Justice (Forensic Evidence and DNA Database System) Act 2014. The2014 Act is extensive, but the main purposes of the Actwere neatly summarised by Colm O’Briain (who also provides a wonderfully succinct synopsis of the 2014 Act) (2014: 1-2). The main purposes include an overhaul of the previous legislation and common law practices in the area of taking DNA samples (from several different groups of people such as offenders, suspects and volunteers), the establishment of the DNA Database System, along with providing management and oversight for the System, and the implementing of the Prüm Council Decision, which provides for the international exchange of DNA evidence. Part 8 of the 2014 Actspecifically addresses the DNA Database System, which is currently controlled by Forensic Science Ireland, an independent body based in the Garda Headquarters in Phoenix Park.

Given the potential of DNA databases, one of the central debates which follows is who (or what offences) should qualify for entry onto the database. Typically, sex offenders are mooted as one of the key categories which should be included on a database. However, most databases extend beyond this to include people who have already been convicted of other serious offences such as murder for example. In some jurisdictions, inclusion criteria are based on the length of the sentence which the offence might warrant (premised on the logic that the more serious the offence, the lengthier the punishment). However, it is not always restricted to people who have been convicted of an offence. A DNA database can also include ‘volunteers’, who are innocent people not convicted or suspected of committing an offence. This has led to discussion on the possibility of population-wide databases, although these are often dismissed as being impracticable both on the grounds of human rights and logistical concerns.

In the case of Irish DNA Database System, there are four main ways that a person’s DNA profile can lawfully appear on same (O’Briain, 2014: 9). These are as follows:

  1. If a person is detained for a relevant offence

A ‘relevant offence’ is an offence for which a person may be detained under Section 9 of the 2014 Act. Offences include those under the Offences Against the State Act 1939, along with drug-trafficking offences, murder, false imprisonment, and offences which may be punished by a term of five years imprisonment or more. O’Briain (2014: 8) neatly summarises that the minimum requirement is an offence with a maximum sentence of at least 5 years.

  1. If a person is an offender or former offender

Offenders are identified as those who have been convicted of a relevant offence and are either (1) serving a sentence, on temporary release or subject to a suspended sentence, (2) convicted before or after the commencement of the Act and sentenced to imprisonment, (3) serving a term of imprisonment on foot of a transfer of prisoners provision (so long as the offence involved corresponds to a relevant offence) or (4) subject to the requirements of Part 2 of theSex Offenders Act 2001at the time of the commencement or at any time thereafter.

  1. If a person volunteers to provide a sample and then allows the profile to be entered onto the System

The taking of DNA samples from volunteers is governed by Part 3 of the Act, with the entry of volunteer profiles onto the DNA Database System covered under Section 28.

 

  1. If a DNA profile was generated under the previous statutory regime, then it may be entered onto the System under transitional provisions. 

Prior to the 2014 Act, the Criminal Justice (Forensic Evidence) Act 1990governed the taking of DNA samples. This provision therefore accommodates the transition of samples collected under the previous legislation and allows such samples to be entered onto the System.

The next debate that follows relates to how long we need to retain this information. As a result, retention periods make up a large part of the discourse on the development of DNA databases around the world. One argument for retaining the information for longer periods of time is that it may mean that detection rates are improved. However, retention of such data has also been considered an invasion of privacy. For example, the UK’s DNA database was subject to “serious scrutiny” which culminated in the European Court of Human Rights (ECtHR) reprimanding the UK’s approach to retention of data in the case of S and Marper v United Kingdom (2008) (Kazemian et al. 2011: 49). England, Wales and Northern Ireland were the only countries in the Council of Europe which allowed for the indefinite retention of DNA data of people who were not convicted of a crime. The ECtHR held that this indefinite retention of data was a violation of Article 8 (the right to privacy) of the European Convention on Human Rights (see Prainsack, 2010: 15-16).

Under the 2014 Act, there are different retention regimes for DNA profiles and samples depending on the origin of the sample. For example, volunteers and those who work in the forensic science laboratory have different retention regimes. It is therefore beyond the scope of this piece to explain each of these different regimes. Instead, this piece specifically considers those who are arrested for a ‘relevant’ offence. In Ireland, the retention regime for this category of persons is quite interesting. Under Section 80 of the 2014 Act,if a person is detained for a relevant offence and their DNA profile is entered onto the System, it is only removed in the following situations:

  1. If proceedings against a person are not instituted within 12 months of taking that sample (unless the reason for the delay is because the person has absconded or cannot be found).
  2. In the case that the proceedings have been instituted, then removal will occur if the person is acquitted of the relevant offence, if the charge is dismissed, or the proceedings discontinued.
  3. If the person’s conviction was identified as a miscarriage of justice.
  4. If the person receives an order under the Probation of Offenders Act 1907 for the relevant offence and they have not been convicted of a relevant offence in the 3 years following that order.

This is subject to Section 81, which allows the Garda Commissioner to extend the retention period for 12 months. This power to extend can be done up to a maximum of 6 years (so extending retention by twelve months six times). The person can however appeal this decision to the District Court. However, there is also a provision under Section 93 which allows the Garda Commissioner to apply to the District Court to extend the retention period once there is a “good reason” to do so (see O’Briain, 2014: 16). This indicates that removal is restricted to certain instances, and that retention of the information appears to be preferred by the legislation.

To conclude, DNA forms an important part of investigations into criminal activity and missing persons. DNA evidence can be highly useful, but the potential is limited if there is no source with which to compare it. To combat this limitation, DNA database systems have been established in jurisdictions around the world. Ireland has now joined this group by enacting the Criminal Justice (Forensic Evidence and DNA Database System) Act 2014 which governs this area of law.

 

References

  • Bieber F. R., (2004) ‘Science and Technology of Forensic DNA Profiling: Current Use and Future Directions’, in DNA and The Criminal Justice System: The Technology of Justice, edited by Lazer D., The MIT Press, Cambridge, pp 23-62.
  • Jasanoff S., (2010) ‘Foreword’, in Genetic Suspects: Global Governance of Forensic DNA Profiling and Databasing, edited by Hindmarsh R. and Prainsack B., Cambridge University Press.
  • Kazemian L., Pease K., and Farrington D. P., (2011) ‘DNA Retention Policies: The Potential Contribution of Criminal Career Research’, European Journal of Criminology8 (1) 48.
  • O’Briain C., (2014) ‘Criminal Justice (Forensic Evidence and DNA Database System) Act 2014: An Overview from a Criminal Practitioner’s Perspective,’ paper presented at the 15th Annual Conference of the Director of Public Prosecutions, available online at:  https://www.dppireland.ie/filestore/documents/PAPER_-_Colm_OBriain_-_2014.10.18.pdf
  • Prainsack B. (2010) ‘Key issues in DNA profiling and databasing: implications for governance’ in Genetic Suspects: Global Governance of Forensic DNA Profiling and Databasing, edited by Hindmarsh R. and Prainsack B., Cambridge University Press, pp 15-39.