![]()
Regular expressions (RegEx) are patterns that describe sets of strings. They are indispensable in bioinformatics for parsing sequence headers, filtering file lists, extracting metadata, and cleaning up data at every stage of a pipeline.
The core syntax — literals, character classes, quantifiers, anchors, and groups — is consistent across virtually all regex flavors. Differences between tools mostly concern advanced features such as lookarounds, shorthand classes (\d, \w, \s), and named capture groups. Learning the basics once gives you a foundation that transfers to grep, sed, awk, R, Python, and most other tools. A good place to practice with real data and immediate feedback is regex101.com, which lets you switch between PCRE2, ERE, and Python flavors and explains each match step by step.
Quick reference
A compact cheat sheet is included at the bottom of this page.
RegEx is not a parser
RegEx excels at finding and extracting text patterns, but it cannot perform numerical comparisons or biological calculations. Filtering sequences by size, computing GC content thresholds, or comparing alignment scores should be done with awk, R, Python, or dedicated bioinformatics tools. Trying to express size >= 100 as a regex pattern is a common beginner mistake.
Regex flavors
One of the most common sources of confusion is that different tools use different regex engines with slightly different syntax and capabilities.
| Tool | Flavor |
|---|---|
grep |
BRE (basic) |
grep -E |
ERE (extended) |
grep -P |
PCRE |
sed |
BRE (default) |
sed -E |
ERE |
awk |
ERE |
| base R | TRE |
base R (perl = TRUE) |
PCRE |
stringr |
ICU |
Python re |
Python regex |
Note
Not every feature is available in every flavor. Lookarounds and named capture groups generally require PCRE, ICU, or Python-style regex. When in doubt, check which flavor your tool uses.
Core syntax
Literal characters and metacharacters
Most characters match themselves. A handful of characters have special meaning and must be escaped with \ when you want to match them literally.
| Metacharacter | Meaning |
|---|---|
. |
Any single character (except newline by default) |
^ |
Start of string (or start of line in multiline mode) |
$ |
End of string (or end of line in multiline mode) |
* |
0 or more repetitions |
+ |
1 or more repetitions |
? |
0 or 1 repetition (also makes quantifiers lazy) |
{n,m} |
Between n and m repetitions |
[...] |
Character class |
[^...] |
Negated character class |
| |
Alternation (OR) |
(...) |
Capturing group |
(?:...) |
Non-capturing group |
\ |
Escape next character |
Alternation in BRE
In POSIX BRE (used by grep without -E and sed without -E), alternation must be written as \|. In all other flavors (ERE, PCRE, ICU, Python) the unescaped pipe | is used.
Groups
(...) capturing group — match is stored and reusable as \1, \2, …
(?:...) non-capturing group — groups the pattern without storing the match
Use non-capturing groups when you only need to group for quantifiers or alternation and do not need to refer back to the matched text. This is slightly more efficient and keeps backreference numbering cleaner.
Named capture groups:
Some PCRE implementations also accept (?P<name>...).
Character classes
[ACGT] one of A, C, G, or T
[^N] any character except N
[A-Za-z] any ASCII letter
[0-9] any digit
\w word character [A-Za-z0-9_] (PCRE/ICU/Python)
\s whitespace (PCRE/ICU/Python)
\d digit (PCRE/ICU/Python)
POSIX character classes (portable in grep, sed, awk):
[[:digit:]] digits (equivalent to [0-9])
[[:alpha:]] letters
[[:alnum:]] letters and digits
[[:space:]] whitespace
[[:upper:]] uppercase letters
[[:lower:]] lowercase letters
POSIX classes are often more portable than \d, \w, and \s, which are not available in BRE and are not standard in ERE.
Anchors
^ start of string (or start of line in multiline mode)
$ end of string (or end of line in multiline mode)
\b word boundary
Quantifiers
Quantifiers are greedy by default: they match as much text as possible.
A* zero or more A (greedy)
A+ one or more A (greedy)
A? zero or one A
A{3} exactly 3 A
A{2,5} 2 to 5 A
A{3,} 3 or more A
This matters in practice. The pattern ".*" will match from the first " to the last " on the line, not the nearest one. Append ? to make any quantifier lazy (match as few characters as possible):
Lookarounds
Lookarounds match a position in the string without consuming characters. They require PCRE, ICU (stringr), or Python regex.
(?=...) positive lookahead — position is followed by ...
(?!...) negative lookahead — position is NOT followed by ...
(?<=...) positive lookbehind — position is preceded by ...
(?<!...) negative lookbehind — position is NOT preceded by ...
Example: extract the value of size= without including the key name:
RegEx in bash
grep
# -E activates extended regex (ERE); -P activates Perl-compatible regex (PCRE)
grep -E '^>' sequences.fasta # all header lines
grep -c '^>' sequences.fasta # count sequences
grep -v '^>' sequences.fasta # sequence lines only
grep -E 'OTU[0-9]{4}' otu_table.txt # OTU IDs with exactly 4 digits
grep -oP '(?<=sample=)\w+' headers.txt # extract value after "sample=" (PCRE)
sed
sed uses BRE by default; use -E for ERE.
# Rename FASTA headers: keep only the first field (split on space)
sed -E 's/(>[^ ]+).*/\1/' sequences.fasta
# Remove trailing whitespace
sed -E 's/[[:space:]]+$//' file.txt
# Replace all occurrences of U with T (RNA to DNA)
sed 's/U/T/g' rna.fasta
# Extract sample name from Illumina filename
# e.g. Sample1_S1_L001_R1_001.fastq.gz -> Sample1
echo "Sample1_S1_L001_R1_001.fastq.gz" \
| sed -E 's/_S[0-9]+_L[0-9]+_R[12]_[0-9]+\.fastq\.gz//'
awk
awk uses ERE for its regex matching. Note that \d, \w, and \s are not standard ERE — use POSIX classes or explicit character classes instead.
# Print all header lines
awk '/^>/' sequences.fasta
# Print sequence lines only (negate the header pattern)
awk '!/^>/' sequences.fasta
# Extract rows where the first field matches a sample ID pattern
awk '$1 ~ /^Sample[0-9]+/' metadata.txt
# Match lines containing a primer sequence
awk '/GTGYCAGCMGCCGCGGTAA/' sequences.fasta
RegEx in R
R offers two regex flavors: TRE (default, perl = FALSE) and PCRE (perl = TRUE). The stringr package (part of the tidyverse) uses the ICU regex engine (via the stringi package), which supports lookaheads, lookbehinds, Unicode matching, and named capture groups.
Escaping backslashes in R strings
R processes backslashes in strings before the regex engine sees them. This means most regex backslashes must be doubled:
"\\d+" # regex engine receives: \d+
"\\|" # regex engine receives: \|
"\\." # regex engine receives: \. (literal dot)
Writing "\d+" alone will cause a warning or error because \d is not a recognised R escape sequence.
Base R
x <- c(">seq1 sample=A length=350", ">seq2 sample=B length=120", "ACGTACGT")
# Detect
grepl("^>", x) # TRUE TRUE FALSE
# Extract (with PCRE lookbehind)
regmatches(x, regexpr("(?<=sample=)\\w+", x, perl = TRUE))
# Replace
gsub("length=\\d+", "", x) # remove length field
# Split on pipe character (escaped in the pattern)
strsplit(">seq1|sample=A|length=350", "\\|")
Enabling PCRE in base R
Add perl = TRUE to grep, grepl, gsub, sub, regexpr, and gregexpr to unlock PCRE features such as lookarounds and \d / \w shorthand classes.
stringr
stringr functions use the ICU regex engine and support lookarounds by default — no extra argument needed.
library(stringr)
headers <- c(">seq1 sample=A length=350", ">seq2 sample=B length=120")
str_detect(headers, "^>") # TRUE TRUE
str_extract(headers, "(?<=sample=)\\w+") # "A" "B"
str_replace(headers, "(?<=length=)\\d+", "???") # mask length values
str_match(headers, ">(\\S+) sample=(\\w+)") # capture groups as matrix
Bioinformatics-specific patterns
FASTA headers
A typical FASTA header might look like:
# Extract OTU ID
grep -oP '^>OTU_\d+' file.fasta
# Extract size annotation
grep -oP '(?<=size=)\d+' file.fasta
# \K resets the start of the reported match (PCRE); often used as an alternative to lookbehind
grep -oP 'size=\K\d+' file.fasta
library(stringr)
headers <- c(">OTU_0042 size=1503;sample=Soil_A", ">OTU_0001 size=3;sample=Water_B")
str_extract(headers, "OTU_\\d+") # "OTU_0042" "OTU_0001"
str_extract(headers, "(?<=size=)\\d+") # "1503" "3"
str_extract(headers, "(?<=sample=)[^;]+") # "Soil_A" "Water_B"
# Numerical filtering must be done after extraction
as.integer(str_extract(headers, "(?<=size=)\\d+")) >= 100
DNA/RNA sequences
# Find sequence lines (not headers) containing a run of ambiguous bases
grep -v '^>' sequences.fasta | grep -E 'N{3,}'
# Identify lines with non-IUPAC characters (excluding headers)
grep -v '^>' sequences.fasta \
| grep -vP '^[ACGTURYSWKMBDHVNacgturyswkmbdhvn]+$'
seqs <- c("ACGTACGT", "ACGTNNNACGT", "ACGT---ACGT")
str_detect(seqs, "N{3,}") # runs of ambiguous bases
str_detect(seqs, "-") # alignment gaps present
str_count(seqs, "[GC]") / str_length(seqs) # GC content
Illumina file naming
Standard Illumina FASTQ filenames follow the pattern:
# List all R1 files
ls *_R1_001.fastq.gz
# Extract sample names from filenames
ls *.fastq.gz | sed -E 's/_S[0-9]+_L[0-9]+_R[12]_[0-9]+\.fastq\.gz//'
# Rename: replace spaces with underscores in sample names
for f in *.fastq.gz; do mv "$f" "${f// /_}"; done
files <- c("Sample_A_S1_L001_R1_001.fastq.gz",
"Sample_B_S2_L001_R2_001.fastq.gz")
# Remove the Illumina suffix to recover the sample name
str_remove(files, "_S\\d+_L\\d+_R[12]_\\d+\\.fastq\\.gz$")
# "Sample_A" "Sample_B"
str_detect(files, "_R1_") # is R1?
Metadata tables and sample IDs
# Find sample IDs with unexpected characters (should be alphanumeric + underscore only)
grep -vP '^[A-Za-z0-9_]+$' sample_ids.txt
# Extract run ID from a path like /data/runs/RUN20240315/sample.bam
echo "/data/runs/RUN20240315/sample.bam" | grep -oP 'RUN\d+'
Exercises
Challenge 1 — Count sequences
You have a FASTA file sequences.fasta. Write a one-liner using grep to count the number of sequences it contains.
Challenge 2 — Extract sample names
Given a list of Illumina filenames:
Write a sed command that prints only the sample names (SoilA, SoilB, WaterC).
Challenge 3 — GC content in R
Given this vector of sequences:
Use stringr to compute the GC content (proportion of G and C) for each sequence.
Challenge 4 — Validate sequence characters
Write a grep command that prints all lines from sequences.fasta (excluding headers) that contain characters other than A, C, G, T, or N.
Challenge 5 — Parse a custom FASTA header
Headers in your file look like:
In R, extract the OTU ID, size value, and the full taxonomy string into separate columns of a data frame.
Quick reference cheat sheet
| Pattern | Matches | Notes |
|---|---|---|
. |
Any character | except newline by default |
^ |
Start of string / line | multiline mode changes behavior |
$ |
End of string / line | multiline mode changes behavior |
\d |
Digit [0-9] |
PCRE/ICU/Python only |
\w |
Word character [A-Za-z0-9_] |
PCRE/ICU/Python only |
\s |
Whitespace | PCRE/ICU/Python only |
[[:digit:]] |
Digit (POSIX) | portable in grep/sed/awk |
[[:alpha:]] |
Letter (POSIX) | portable in grep/sed/awk |
[ACGT] |
One of A, C, G, T | |
[^N] |
Anything except N | |
A+ |
One or more A | greedy |
A* |
Zero or more A | greedy |
A+? |
One or more A | lazy |
A? |
Zero or one A | |
A{3,5} |
3 to 5 A | |
(seq) |
Capturing group | |
(?:seq) |
Non-capturing group | |
\1 |
Backreference to group 1 | |
a\|b (BRE) / a|b |
a or b | use \| in BRE only |
(?<=size=)\d+ |
Lookbehind: digits after "size=" | PCRE/ICU |
(?=_R1) |
Lookahead: position before "_R1" | PCRE/ICU |
Links
- regex101.com — interactive tester with real-time explanations; supports PCRE2, ERE, and Python flavors
- RegEx Tutorial — step-by-step exercises for building pattern intuition (uses JavaScript regex flavor)
- Cheat Sheet — compact reference for syntax across flavors
- stringr cheatsheet — R-specific string and regex functions
For command-line tools, the built-in manual pages are the authoritative reference: man grep, man sed, man awk. In R, ?base::regex documents the regex syntax accepted by base R functions.