Skip to content
CannabisSeeds Research Library
Search
Advanced Genetic Testing & Markers

Why Does Multiple Testing Matter in Cannabis Genetics?

Multiple testing matters in cannabis genetics because searching many markers gives chance more opportunities to produce apparently significant results. A marker with a small p-value needs to be interpreted in the context of the whole search, its error-control method and its validation evidence. One promising result isn't automatically a reliable predictor.

Branching hypothesis paths with a few red terminal marks in a conceptual linocut about multiple genetic tests.

Multiple testing matters in cannabis genetics because searching many markers gives chance more opportunities to produce apparently significant results. A marker with a small p-value needs to be interpreted in the context of the whole search, its error-control method and its validation evidence. One promising result isn’t automatically a reliable predictor.

The useful question is what had to happen for that result to reach the report. Was it the only planned comparison, one of thousands of marker tests, or the most favorable result from several analytical choices? Those are different evidence situations even if the printed p-value is identical.

A small calculation explains the problem

Imagine 20 independent tests, all testing true null hypotheses (the reference claims being tested), each using a 0.05 false-positive threshold. The probability of at least one false positive is 1 minus 0.95 raised to the twentieth power, or about 64%. This is a hypothetical calculation under those assumptions, not an observed error rate in cannabis research.

Real marker tests aren’t usually independent. Nearby variants can be correlated through linkage disequilibrium, so you shouldn’t apply that 64% example directly to a genomic dataset. The example shows why the size and organization of the search matter; the actual dependence needs to be handled by an appropriate method.

Decide which error you want to control

Family-wise error control addresses the chance of making one or more false rejections in a defined family of tests. Bonferroni is a simple approach: for 1,000 tests and an overall 0.05 target, the per-test threshold is 0.00005. It can be conservative when tests are highly correlated. NIST’s Bonferroni explanation gives the underlying simultaneous-inference reasoning.

False discovery rate, or FDR, asks a different question: what expected proportion of the rejected hypotheses are false rejections? Under its applicable assumptions, an FDR procedure can support a discovery list while accepting a controlled expected fraction of false discoveries. A 5% target doesn’t promise that exactly 5% of your particular list is wrong, and it doesn’t give each marker a 95% probability of being real.

Benjamini and Hochberg’s original paper establishes the distinction. Choose the method according to the research aim and its assumptions, rather than selecting whichever correction leaves the most hits.

Read a small FDR example

For five hypothetical valid, independent tests, sort the p-values as 0.002, 0.013, 0.029, 0.11 and 0.80. At a 0.05 FDR target, the Benjamini-Hochberg step-up thresholds are rank x 0.05 / 5: 0.01, 0.02, 0.03, 0.04 and 0.05. The largest rank meeting its threshold is three, so the first three hypotheses are rejected. This is an illustration of the rule, not a probability assignment to those three findings.

For correlated genomic tests, have the analyst justify the procedure’s dependence assumptions or choose an appropriate alternative. Don’t reuse this small independent-test example as a ready-made genetic analysis.

The testing family includes the question you searched

For a planned genetic study, record the phenotype, marker set, model and family of comparisons before interpreting the results. If you examine several traits, subgroups or alternative filters, retain those analytical choices too. Correcting a selected final table doesn’t automatically account for every search that produced it.

This doesn’t mean exploratory work is illegitimate. It means discovery should be described as discovery. Keep the full analysis trail, identify which results were planned and reserve fresh data for confirmation. If the report only shows favorable findings, you can’t reconstruct the strength of evidence from those p-values alone.

Correction doesn’t remove confounding

An extremely small adjusted p-value can still reflect a problem in the comparison. If ancestry and the measured trait travel together, markers may identify ancestry rather than influence the trait. The site’s genome-wide association explanation shows why population structure, relatedness and measurement quality remain necessary even after multiple-testing correction.

Nor does correction establish causation or practical predictive value. A signal may tag a nearby region, describe only one sampled population or have an effect too small for the intended use. Those limits aren’t repaired by printing more decimal places.

Keep discovery and validation separate

Before using a hit, ask for the testing family, correction method, effect estimate, uncertainty and independent confirmation. A quantitative geneticist should review a large or complex marker search, especially when the result will guide a consequential decision.

Your next question is whether the finding survives in relevant new material under a locked assay and analysis. That’s the job of marker validation. Treating that as a separate step gives a promising result room to become useful without claiming it has already done so.

Join the discussion