Project Scope: What to Process Together

Combining more samples into one project isn't primarily a storage problem, it's a complexity and bias problem. Bigger isn't automatically better.

It's tempting to combine as many samples as possible into one project, more data feels like it can only help. In practice, the size of the resulting files is rarely the real cost. The complexity is.

More Samples Means More zOTUs, and Mapping Gets Slow

Every additional sample can add new zOTUs to the cluster set, and mapping reads back to a much larger set of clusters takes real time. See the Mapping Algorithms note on the PE workflow page: the heuristic USEARCH uses by default is already a speed/accuracy tradeoff, and a full dynamic-programming alignment (guaranteed optimal, much slower) becomes even less practical as the cluster set grows. There are ways to speed this up. Before reaching for them, it's worth asking a more basic question first.

Ask First: Do You Actually Need to Compare These Samples Directly?

Combining samples into one processing batch only pays off if you genuinely intend to compare them directly, against the same zOTUs, in the same count table. If you don't, you're paying the complexity and time cost for no analytical benefit. Prepare what you actually want to compare. Combining samples "because you have them" or because it's the default, obvious move isn't a reason on its own.

The Bigger Problem: Heterogeneous Samples Force a Compromise

This matters more than processing speed. Samples from very different sources, different sample types, different expected diversity, different quality profiles, don't behave the same way during filtering and clustering. Processing them together means finding one shared set of parameters that works for all of them, and often, no single setting is actually right for every sample in the batch. A quality or length filter tuned well for one sample type can be too strict or too lenient for another processed alongside it in the same run.

A Compromise Is a Source of Bias, Not Just Inefficiency

When one shared parameter set has to fit samples that don't really belong together, the result isn't neutral. Some samples get slightly over-filtered, others slightly under-filtered, relative to what would have been correct for them individually. That difference can systematically affect some samples more than others, which means it can masquerade as a real biological difference between groups when it's actually a processing artefact. This is exactly the kind of problem the data loss philosophy on the PE workflow page is about: understanding why a step behaved the way it did, not just accepting the number. A compromise parameter setting is one of the harder cases to catch, because it doesn't show up as an obvious error, it shows up as a subtly biased result.

Comparing Across Separate Projects Is Still Possible, Just Not at the zOTU Level

If two datasets were processed separately, in different batches, on different runs, with different parameters, you can still compare them meaningfully. What you can't do is compare them at the zOTU level.

zOTU identity is specific to the clustering run that produced it. Clustering isn't fully deterministic even within one run (see the note on OTU clustering being a greedy algorithm with several equally valid ways to draw cluster boundaries), and across two separately processed datasets, the same underlying biological sequence has no guarantee of landing in a comparable cluster, let alone getting the same zOTU number. Treating zOTU IDs from two different processing runs as directly comparable is not a safe assumption.

Taxonomic annotation is different. It's assigned against a shared external reference (see Choosing a Reference Database), independently of how clustering happened in either dataset. Two separately processed projects can be meaningfully compared at the genus or species level, where the reference provides the common currency, even though their zOTU tables have nothing to say to each other directly.

Tree Building Gets Harder Too

The same complexity problem hits tree building specifically, and often worse. Building a tree starts with a multiple sequence alignment (MSA) across all your zOTUs, and alignment quality degrades both as the number of sequences grows and as those sequences become more phylogenetically divergent from each other. A large, diverse cluster set is close to a worst case for MSA: too many sequences to align efficiently, and too much divergence between them for the alignment itself to stay reliable. The resulting tree inherits those problems, a gappy or unstable alignment produces unreliable topology and branch lengths, on top of the approximation already inherent in the tree-building approach we use. If you only need a tree for part of your dataset, building it from that subset rather than the full combined cluster set gives the aligner an easier, more homogeneous job and a more trustworthy result.

In Short

Scope your processing batch to what you actually intend to compare directly, not to everything you happen to have. If a comparison can wait until the taxonomic level, it doesn't need to happen inside the same zOTU table, or the same tree, at all.