Skip to content

Data Prep Long-Reads

The PacBio CCS/HiFi workflow: no merging, but orientation, deconcatenation, and a few platform-specific quirks of its own.

This describes our processing workflow for PacBio CCS/HiFi long-read amplicon data. See the PacBio CCS Reads section of the Overview Workflows page for the background on how this data is generated and delivered, and the Long-Read vs. Short-Read Sequencing page if you're still deciding whether long reads are the right choice for your project in the first place.

As with the short-read workflows, our pipeline is modular, not fixed (see the note near the top of Data Prep Short-Reads (PE)), and the same caution about data loss applies here too: what matters is understanding why a step lost data, not the raw percentage.

PE:         QC → Merge → Primer trim → Filter → Cluster → Taxonomy

Long-Reads: QC → (Deconcatenate) → Demultiplex → Orient → Primer trim → Filter → Cluster → Taxonomy
                      ↑
              only if Kinnex

Data Processing Steps - Explained

Step A - Quality Control, Data and Parameter Evaluation

Same purpose as Step A of the PE workflow, understanding the data before processing it, but the report itself looks different for PacBio. The basecaller report is central here (see Data Prep Output): number of passes per CCS read, read quality distributions, and adapter/barcode statistics all inform how the later steps are parameterised. This is also where we confirm whether a dataset is standard CCS or a Kinnex array that needs deconcatenation first.

Step B - Deconcatenation and Demultiplexing

If your data is a Kinnex library, this step comes first: the concatenated array is segmented into individual amplicon reads before anything else can happen (see the Kinnex Libraries Are Different note on the Overview Workflows page). For the standard 16S-Kinnex kit this is well established; for other marker genes or custom array designs, it's trickier and worth talking to us about early.

We prefer to demultiplex from the undemultiplexed BAM file ourselves, even when demultiplexed FASTQ files are offered, since it gives us direct visibility into run performance, including reads carrying a barcode combination that wasn't actually used, a useful signal for misassignment or contamination. See Our Preference: Do the Demultiplexing Ourselves for the full reasoning.

We also apply a low-complexity filter at this stage, the same as in the short-read workflows. PhiX filtering doesn't apply here, PhiX is an Illumina-specific spike-in control and isn't part of PacBio runs.

Step C - Orientation and Primer Site Trimming

PacBio CCS reads aren't oriented the way Illumina reads are (see PacBio Sequence Orientation), so the first task is aligning every read to a consistent strand. Once oriented, both primer sites are trimmed, forward and reverse, since a single CCS read spans the entire amplicon rather than arriving as a pair that gets merged. The matching criteria (mismatch tolerance, wobble base handling) follow the same principles as Step C of the PE workflow.

If your amplicon is shorter than expected, or the array included flanking non-target regions, you may also see primer-site read-through, the same issue discussed for short single-end reads in Data Prep Short-Reads (SE), Step C, and it's screened for here too.

Step D - Filtering

Quality, size, and composition filtering follow the same logic as Step D of the PE workflow, applied to the oriented, trimmed CCS read directly rather than a merged sequence. Because a CCS read's accuracy comes from multiple internal passes around the same circular molecule rather than from a second read cross-checking a first (see the discussion of this tradeoff on the Overview Workflows page), there's no merge-quality step here, but the same coding-vs-non-coding length distribution logic applies: a full-length COI amplicon should show a tight length peak, an rRNA gene like full-length 16S a broader one (see Step D of the PE workflow for why).

Heteroduplexes on Revio

Forward/reverse strand mismatches (heteroduplexes) can produce a single, misleading consensus read on Revio instruments rather than being flagged as ambiguous, masking real allele-level variation. See Insights & Solutions for the mechanism and what reduces the impact. This is a platform property, not something this filtering step alone can catch, worth being aware of especially where precise variant-level resolution matters.

Step E - Clustering

Identical to Step E of the PE workflow: the same UPARSE and UNOISE approaches, implemented in USEARCH, apply regardless of whether the input was a merged short-read pair or an oriented long read.

Step F - Taxonomic Assignment

Identical process to Step F of the PE workflow: SINTAX with an appropriate reference database. One long-read-specific caveat worth knowing: for full-length 16S in particular, reference databases are still predominantly built from short-read fragments, which can paradoxically reduce classification confidence for a longer, more informative query rather than improving it. See the Reference Database Coverage section of the Long-Read vs. Short-Read page for why.

Step G - Extra(s)

The same additional, marker-specific steps described in Step G of the PE workflow apply here too: ITSx for ITS amplicons, and ORF checking for protein-coding loci such as full-length COI, run on the zOTU representative sequences after clustering rather than on raw reads.

GDC Report Files, by Step

This section is specific to how we organise processing at the GDC and won't be relevant if you're running your own pipeline elsewhere.

Step What happens Report
A Quality control, data and parameter evaluation y_help/A_*.report
B Deconcatenation (if Kinnex) and demultiplexing y_help/B_*.report
C Orientation and primer site trimming y_help/C_*.report
D Filtering y_help/D_*.report
E Clustering (OTU/zOTU) y_help/E*.report
F Taxonomic prediction y_help/F*.report
G Extras (e.g. ITSx, ORF checking) y_help/G_*.report

The * in each filename is a placeholder for your project and sample identifiers.

Report Filenames Not Yet Confirmed

Unlike the PE and SE pages, these exact report filename patterns (the B/C/D suffixes in particular) haven't been verified against a real long-read project's y_help output. Please check these against an actual run before treating them as accurate, and let me know what to correct.