Scientific Missteps
This section examines common challenges and limitations that can affect scientific integrity and research reliability, including statistical errors, biases in data, analytical flaws, and ethical considerations related to data handling. It also explores the distinction between correlation and causation, highlighting how incorrect assumptions can lead to misleading conclusions. Recognising these potential pitfalls is essential for conducting rigorous and transparent research.
Beyond bar plots
Bar charts can be misleading when presenting data because they only show summary statistics such as means, masking variability, skewness and outliers. This oversimplification can distort interpretation, especially when distributions are non-normal. In contrast, box plots, especially when overlaid with individual data points, reveal key distribution features including medians, quartiles and potential outliers. Violin plots go a step further by visualising the full probability density, making it easier to spot multimodal distributions and asymmetries. By replacing bar plots with these richer visualisations, we ensure a more accurate, transparent and informative representation of the data.
Bar plots hide the truth. Box plots and violin plots tell the whole story.
The perils of correlation
Correlation is often mistaken for causation, leading to false conclusions in science and everyday reasoning. Just because two variables are statistically related does not mean that one causes the other. Spurious correlations, where unrelated factors appear to be related, can be misleading, and even statistically significant correlations may have little meaning if they are weak. A humorous example of this can be found on Tyler Vigen's Spurious Correlations website, which presents absurd but statistically significant relationships, such as the correlation between cheese consumption and deaths from tangled bedsheets. Without careful analysis and proper controls, correlation-based claims can distort reality and reinforce biases rather than reveal true causal relationships. Recognising these pitfalls is essential for sound scientific reasoning and decision making.
Correlation does not imply causation!
The illusion of precision
Numbers are often used to lend credibility to weak or unsubstantiated arguments, but without proper accuracy and context, they can be deeply misleading. Quantities can be miscounted, measurement procedures can be flawed, and assumptions can go unexamined, all of which can produce numbers that falsely support a weak claim.
Beyond simply questioning numbers, it is crucial to recognise how their presentation can distort reality. A figure reported with excessive decimal places or without relevant background information creates a false sense of accuracy, making data appear more reliable than it actually is.
For example, when measuring DNA concentration with a NanoDrop spectrophotometer, reporting a value of 53.2749 ng/µL may appear highly precise. However, given the instrument's inherent variability and sensitivity to contaminants like RNA or protein, such precision is misleading. A rounded value such as 53 ng/µL is more appropriate and honestly acknowledges the limitations of the measurement. True precision comes from understanding the accuracy of the method, not from reporting more decimal places.
More decimal places do not make a number more true. They just make the nonsense fancier.
No magic in the black box
No matter how sophisticated an analysis is, its conclusions are only as good as the data on which it is based. This principle, often summarised as Garbage In, Garbage Out (GIGO), highlights the fundamental truth that faulty, biased or poor quality input data will inevitably lead to unreliable or misleading results, no matter how advanced the methods applied.
Even the most powerful statistical models, machine learning algorithms or bioinformatics pipelines cannot fix fundamental problems in the data. If sequencing reads are contaminated, metadata is incorrect, or measurements are inconsistent, the results will be equally flawed. The illusion of rigour created by complex methods can sometimes mask these underlying problems, leading to false confidence in erroneous conclusions.
You can't plant weeds and expect roses.
The self-correcting nature of science
Science is a self-correcting process in which any scientific work can be wrong. Scientific knowledge evolves as new data, methods and perspectives emerge, so conclusions are always subject to revision. While peer review is an important filter, it is not infallible, and errors, whether due to statistical mistakes, misinterpretation or biological variability, are common. Some papers may even be retracted due to fraud or irreproducibility. But this openness to challenge is science's greatest strength. By constantly testing, questioning and refining ideas, science advances and our understanding of the world becomes more accurate over time.
In science, no finding is final. Every idea is subject to revision.
Publication bias
A well-known bias in scientific research is that positive results are more likely to be published, while negative or inconclusive results are often filed away. This publication bias can distort the scientific literature, giving the misleading impression that certain treatments or hypotheses are more successful than they actually are. This is a major problem for meta-analyses, which rely on the existing literature to draw conclusions. If a meta-analysis is based mainly on positive results, it can lead to overestimated effects and misinformed conclusions, potentially misguiding future research or policy. It is vital that the scientific community recognises and addresses this bias to ensure more accurate and comprehensive evidence synthesis.
Positive results get the spotlight. Negative ones are backstage, waiting for their turn.
The saviour complex in science
Many scientists, driven by passion, sometimes present themselves as the sole saviours of the world, curing diseases or solving global issues on their own. This mindset forgets that science is about collaboration, sharing knowledge, and building on collective efforts. When researchers focus more on self-promotion than on advancing shared goals, it can hinder progress. True scientific advancement comes from openness, teamwork and mutual respect, not individual glory.
Science is a team sport, not a solo mission to save the world.
AI, machine learning and deep learning are not magic
In today's world, terms like artificial intelligence (AI), machine learning (ML) and deep learning (DL) are often used to impress and inflate the perceived value of analytics. While these technologies are powerful, they are not inherently revolutionary in their predictive capabilities. They are simply advanced methods, some dating back decades, now supercharged by modern computing power and massive datasets.
The basics of predictive modelling, such as linear regression, decision trees and neural networks, have been used in medicine, finance and business since the 1950s. What has changed is not the existence of AI, but its scale, speed and accessibility.
These tools do not automatically deliver better results just because they sound impressive. They offer flexible, data-driven approaches, but only if applied correctly with the right data and context. Large datasets can lead to large mistakes, and high-dimensional data can lead to overfitting, where a model fits noise rather than true patterns. These methods are meant to enhance decision making, not replace it.
By understanding AI, ML and DL as evolutions of long-standing statistical methods, we can appreciate their true value without getting lost in the hype.
Impressive terminology is not a substitute for good science.
Sampling bias
Sampling bias occurs when the sample selected for a study does not accurately represent the wider population, leading to biased or misleading results. In biological research, this can seriously affect conclusions about a species or ecosystem. For example, if a study of the health of a population of daphnids only collects healthy individuals because the sick ones are outside the sampling zone, the results will overestimate the overall health of the population. This bias can lead to incorrect conclusions, such as underestimating the effects of environmental stressors or overlooking critical health issues. Proper, random sampling is essential to obtain reliable and representative data.
Don't write a review of a book by reading only the first chapter.
p-hacking and multiple testing
One of the most prevalent problems in modern biology is the manipulation of analyses, whether intentional or not, until a statistically significant result appears. This practice, known as p-hacking, includes trying multiple statistical tests, removing inconvenient data points, or stopping data collection as soon as p < 0.05 is reached. The result is a finding that looks significant but rarely replicates.
This problem is compounded in genomics, where thousands of hypotheses are tested simultaneously, for example across all genes in an RNA-Seq experiment or all SNPs in a population study. Without correction for multiple testing, a large number of false positives is virtually guaranteed by chance alone. Methods such as the Benjamini-Hochberg False Discovery Rate (FDR) correction and the Bonferroni correction exist precisely to address this, and their appropriate use is not optional in high-throughput analyses.
If you torture the data long enough, it will confess to anything.
HARKing: Hypothesising after results are known
HARKing occurs when a researcher presents a hypothesis that was formulated after seeing the results as if it had been the original hypothesis driving the study. This practice is problematic because it makes exploratory findings look like confirmatory ones, inflating the apparent strength of the evidence. HARKing is closely related to p-hacking and is a major contributor to the replication crisis in science. Pre-registration of hypotheses and analysis plans before data collection is one of the most effective ways to prevent it.
Predicting the past is not science. It is storytelling.
Batch effects
A batch effect occurs when technical variation introduced during sample processing, such as different sequencing runs, reagent lots, operators, or processing dates, creates systematic differences in the data that have nothing to do with the biology being studied. In genomics, batch effects can be large enough to completely obscure true biological signals, or worse, be mistaken for them. Recognising, measuring and correcting for batch effects using tools such as ComBat or by including batch as a covariate in statistical models is a critical step in any serious genomic analysis.
If your biggest source of variation is the day of the week, something has gone wrong.
Overfitting
Overfitting occurs when a statistical model is tuned so closely to the data it was trained on that it captures noise rather than true underlying patterns. The model performs well on the training data but fails to generalise to new data. This is a risk in any analysis involving many parameters relative to the number of observations, which is a common situation in biology where datasets are often small but measurements are numerous. Cross-validation, regularisation, and independent test sets are standard approaches to detect and guard against overfitting.
A model that explains everything predicts nothing.
Misuse of reference genomes
The choice of reference genome has a direct and often underestimated impact on the outcome of alignment, variant calling, and gene expression analyses. Using an outdated genome assembly, the wrong species, or a reference that does not represent the genetic diversity of the study population can introduce systematic errors that propagate through the entire analysis. Always verify that the reference genome version is appropriate for your organism, your research question, and is consistent across all steps of the pipeline.
Your analysis is only as good as the map you are navigating by.
Confirmation bias
Confirmation bias is the tendency to search for, interpret, and remember information in a way that confirms what we already believe. In science, this can manifest as focusing only on results that support a hypothesis while dismissing or under-reporting contradictory findings. It is one of the most deeply human cognitive biases and affects experimental design, data interpretation, and peer review alike. Awareness of this bias is the first step toward countering it.
We see what we expect to see. Rigorous science demands that we look for what we hope is not there.
The HiPPO effect
The HiPPO effect, short for Highest Paid Person's Opinion, describes the tendency for decisions to be driven by seniority or authority rather than by data and evidence. In a research context, this can mean that a principal investigator's intuition overrides the results of a careful analysis, or that a junior researcher does not feel empowered to challenge an incorrect interpretation. Good science requires a culture where evidence, not hierarchy, determines conclusions.
Data beats opinion, regardless of whose opinion it is.
