nf-core/oncoanalyser

Edit

Introduction

The oncoanalyser pipeline typically takes FASTQ, BAM, or CRAM files as input. These can be generated by Illumina, Ultima Genomics, or Roche SBX (sequencing by expansion) sequencing technologies. The pipeline accepts most GRCh37 and GRCh38 human reference genome builds, and provides UMI (unique molecular identifier) processing for DNA sequencing data.

oncoanalyser has two main pipeline modes:

• wgts: whole genome and/or transcriptome sequencing
• targeted: targeted/panel sequencing

Both modes accept various combinations of sequencing type (DNA, RNA) and samples (tumor-only, matched tumor/normal with optional donor sample):

• DNA-only:
  • DNA(tumor)
  • DNA(tumor+normal)
  • DNA(tumor+normal+donor)
• DNA and RNA
  • RNA(tumor) + DNA(tumor)
  • RNA(tumor) + DNA(tumor+normal)
  • RNA(tumor) + DNA(tumor+normal+donor)
• RNA-only:
  • RNA(tumor)

Other available pipeline modes are:

• purity_estimate: tumor fraction estimation in longitudinal samples (e.g. for MRD)
• panel_resource_creation: creating reference data for custom panels
• prepare_reference: staging genomes and WiGiTS tool reference data

Running the pipeline

If you intend to run oncoanalyser on more than one sample, we recommend first staging and configuring reference data (genome and tool specific data). Otherwise, reference data is automatically staged every run resulting in unnecessary disk/network usage.

A typical command for running oncoanalyser is shown below:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -profile docker \
  -config reference_data.config \ # Optional but recommended
  --mode wgts \
  --genome GRCh38_hmf \
  --input samplesheet.csv \
  --outdir output/ \

Below is a brief description of each argument:

• -revision: oncoanalyser version to run (can be a git tag, branch, or commit hash)
• -profile: configuration presets for different compute environments
• -config: one or more comma separated configuration files for customising e.g. genome/tool reference data, custom panels reference, compute resources, or other configuration
• --mode: pipeline mode
• --genome: genome version, typically GRCh38_hmf or GRCh37_hmf
• --input: the samplesheet containing sample details and corresponding files to be analysed
• --outdir: output directory

If you have data from Roche SBX or Ultima sequencing technologies, you will need to specify argument --sequencing_platform (see Sequencing technologies).

Outputs

Running oncoanalyser will create the following files in your working directory:

work           # Directory containing the nextflow working files
<OUTDIR>       # Finished results in specified location (defined with --outdir)
.nextflow.log  # Log file from Nextflow
# Other nextflow hidden files, e.g. history of pipeline runs and old logs.

Descriptions of each output file in <OUTDIR> are provided in the output documentation.

Reusing CLI arguments

To use the same CLI arguments across multiple runs, you can specify these in a yaml or json file via -params-file <file>. The above command would have the equivalent yaml file:

mode: 'wgts'
genome: 'GRCh38_hmf'
input: 'samplesheet.csv'
outdir: 'output/'
<...>

and be run using this command:

nextflow run nf-core/oncoanalyser -revision 3.0.0 -profile docker -params-file params.yaml

You can also generate such YAML/JSON files via nf-core/launch.

Updating the pipeline

When you run the above command, Nextflow automatically pulls the pipeline code from GitHub and stores it as a cached version. When running the pipeline after this, it will always use the cached version if available - even if the pipeline has been updated since. To make sure that you’re running the latest version of the pipeline, make sure that you regularly update the cached version of the pipeline:

nextflow pull nf-core/oncoanalyser

Reproducibility

It is a good idea to specify the pipeline version when running the pipeline on your data. This ensures that a specific version of the pipeline code and software are used when you run your pipeline. If you keep using the same tag, you’ll be running the same version of the pipeline, even if there have been changes to the code since.

First, go to the nf-core/oncoanalyser releases page and find the latest pipeline version - numeric only (eg. 1.3.1). Then specify this when running the pipeline with -r (one hyphen)

• eg. -r 1.3.1. Of course, you can switch to another version by changing the number after the -r flag.

This version number will be logged in reports when you run the pipeline, so that you’ll know what you used when you look back in the future. For example, at the bottom of the MultiQC reports.

To further assist in reproducibility, you can use share and reuse parameter files to repeat pipeline runs with the same settings without having to write out a command with every single parameter.

Sequencing technologies

All pipeline modes of oncoanalyser support sequencing data from the following technologies by providing one of the below values to --sequencing_platform:

• illumina: Illumina (default)
• sbx: Roche SBX
• ultima: Ultima Genomics

For Illumina sequencing, --sequencing_platform does not need to be specified. For SBX and Ultima sequencing, you will need to explicitly specify --sequencing_platform like so:

nextflow run nf-core/oncoanalyser \
-revision 3.0.0 \
-config reference_data.config \
-profile docker \
--sequencing_platform sbx \
--mode wgts \
--genome GRCh38_hmf \
--input samplesheet.csv \
--outdir output/

Note that BAM (or CRAM) files are expected to:

• Be aligned to the reference genomes bundled with oncoanalyser
• Have soft-clipped supplementary reads (e.g. BWA-MEM2 run with -Y argument)
• Have the mate CIGAR attribute for any BAM records with paired reads. If not, BAMs may be rectified with e.g. Picard FixMateInformation
• Have certain sequencing technology specific tags, e.g. YC for SBX, TP and T0 for Ultima

For Ultima sequencing, BAM files generated by the on-board alignment pipeline will meet the above requirements.

For SBX sequencing, oncoanalyser has been validated with BAMs generated with the on-board “BWA-based alignment v1.1.0” protocol with the below settings:

• “Unmapped Reads”: Include
• “Supplementary Reads”: Include
• “Duplicate Marking”: Enable
• “Penalty for Mismatch”: 4 (default value)
• “Aligned reads threshold (in millions)”: 6000

In other cases, converting from BAM back to FASTQ may be required to run oncoanalyser.

Samplesheet

The samplesheet contains information in CSV format for each sample to be analysed by oncoanalyser, and uses a header row as the first line with the below columns:

Output file paths are constructed based on group_id and sample_id:

• group_id: top-level output directory for analysis files e.g. output/PATIENT1/
• tumor sample_id: output prefix for most filenames e.g. PATIENT1-T.purple.somatic.vcf.gz
• normal sample_id: output prefix for some filenames e.g. PATIENT1-N.cobalt.ratio.pcf

The read_group_overrides info field can be used for FASTQ inputs to override and set read group tags, which are then propagated through alignment into the respective output BAM / CRAM file. Tags are specified as <tag>=<value> with multiple tag and value entries separated by |, for example read_group_overrides:ID=id-1|SM=sample-1|LB=lib-1.

Analysis starting points

An oncoanalyser run will typically start from either FASTQ or alignments (BAM / CRAM, REDUX BAM / CRAM) inputs. However, the pipeline can also start from arbitrary entry points as long as the required inputs are provided in the samplesheet.

FASTQ

To run from FASTQ:

• specify fastq in the filetype field,
• set sequencing library and lane information in the info field separated by ;, and
• provide the forward (‘R1’) and reverse (‘R2’) FASTQ files in the filepath field separated by ;

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,info,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,fastq,library_id:S1;lane:001,/path/to/PATIENT1-T_S1_L001_R1_001.fastq.gz;/path/to/PATIENT1-T_S1_L001_R2_001.fastq.gz
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,fastq,library_id:S1;lane:002,/path/to/PATIENT1-T_S1_L002_R1_001.fastq.gz;/path/to/PATIENT1-T_S1_L002_R2_001.fastq.gz

BAM

To run from BAM, specify bam in the filetype field:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam,/path/to/PATIENT1-T.dna.bam

BAM indexes (.bai files) are expected to be in the same location as the BAM files with matching filenames suffixed with .bai. Where this is not the case, you can also explicitly provide the BAM index location by specifying bai in the filetype field:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam,/path/to/PATIENT1-T.dna.bam
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bai,/other/dir/PATIENT1-T.dna.bam.bai

CRAM

To run from CRAM, use cram and crai in the filetype field. crai only needs to be provided if the CRAM index is not in the same directory as the CRAM file:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,cram,/path/to/PATIENT1-T.dna.cram
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,crai,/other/dir/PATIENT1-T.dna.cram.crai

CRAMs should be of version ≤3.0, have the reference genome embedded, and have NM tags preserved. You can use the below template command to convert a BAM to a CRAM with the appropriate format:

samtools view \
  --cram \
  --output-fmt-option version=3.0 \
  --output-fmt-option reference=GRCh38_masked_exclusions_alts_hlas.fasta \
  --output-fmt-option store_nm=1 \
  --output sample.cram \
  --threads 4 \
  --write-index \
  sample.bam

REDUX alignments

The most time and resource-intensive pipeline steps are read alignment by BWA-MEM2 and read post-processing by REDUX. You may therefore want to start from existing REDUX outputs.

REDUX a BAM file with deduplicated reads and reads in problematic regions unmapped:

• <sample_id>.redux.bam
• <sample_id>.redux.bam.bai

REDUX also produces TSV files in the same directory as the BAM file:

• <sample_id>.redux.bqr.tsv: base quality recalibration per trinucleotide mutation context
• <sample_id>.redux.jitter_params.tsv: rates of microsatellite errors used for jitter modeling
• <sample_id>.redux.ms_table.tsv.gz: aggregated repeat units and repeat counts, also used for jitter modeling
• <sample_id>.redux.duplicate_freq.tsv: only required for QC metrics

When starting the pipeline from REDUX outputs, provide the path containing the REDUX BAM and TSVs to redux_dir (note: the provided sample_id must match the output file prefix, i.e. <sample_id> shown above):

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,redux_dir,/path/to/redux_PATIENT1-T/

The redux_dir may, instead of a REDUX BAM, contain a REDUX CRAM generated with the same samtools command as for regular CRAMs:

• <sample_id>.redux.cram
• <sample_id>.redux.cram.crai

When storing REDUX outputs, both the BAM/CRAM and TSVs should be kept. The following section describes re-generating REDUX TSVs if they are missing or out of date.

REDUX alignments but missing REDUX TSV files

You may have existing REDUX BAMs but one or more REDUX TSV files or missing or out of date, e.g. due having run an older oncoanalyser version. These TSVs can be quickly re-generated without having to full re-process the alignments with REDUX by setting generate_redux_tsvs_only in the samplesheet info field for the corresponding REDUX alignment:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath,info
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam_redux,/path/to/PATIENT1-T.dna.redux.bam,generate_redux_tsvs_only

You may explicitly provide the path to the BAM index with bai in the filetype field if it is not in the same directory as the BAM file:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath,info
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam_redux,/path/to/PATIENT1-T.dna.redux.bam,generate_redux_tsvs_only
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bai,/other/dir/PATIENT1-T.redux.bam.bai

Then, run oncoanalyser with as normal:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -config reference_data.config \
  -profile docker \
  --mode wgts \
  --genome GRCh38_hmf \
  --input samplesheet.csv \
  --outdir output/ \
  --redux_generate_tsvs_only

REDUX will run in a special mode to only generate the TSV files, skipping the computationally expensive read processing steps. The rest of the pipeline will then be run.

Starting from existing outputs

Similar to starting from existing REDUX alignments files, any of the downstream outputs of oncoanalyser can be used as inputs so that the associated downstream processes can be skipped. This could be considered an explicit form of -resume which does not require the Nextflow work/ directory to be preserved.

For example, you may already have tumor/normal sample output for small variant calling from SAGE and structural variant calling from ESVEE, and want to run the rest of the pipeline.

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-N,normal,dna,redux_dir,/path/to/redux_PATIENT1-N/
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,redux_dir,/path/to/redux_PATIENT1-T/
PATIENT1,PATIENT1,PATIENT1-N,normal,dna,sage_dir,/path/to/sage/germline/
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,sage_dir,/path/to/sage/somatic/
PATIENT1,PATIENT1,PATIENT1-T,tumor_normal,dna,esvee_dir,/path/to/esvee/

All valid combinations of sample_type, sequence_type and filetype can be found here.

This approach can be used in combination with manual process selection to run specific downstream components of the pipeline.

Sample setups

Providing sample_type and sequence_type in different combinations allows oncoanalyser to run in different sample setups. The below samplesheet examples use BAM files but different sample setups can also be specified for FASTQ or CRAM files.

Paired tumor and normal DNA

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-N,normal,dna,bam,/path/to/PATIENT1-N.dna.bam
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam,/path/to/PATIENT1-T.dna.bam

Tumor-only DNA

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam,/path/to/PATIENT1-T.dna.bam

Tumor-only RNA

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-T-RNA,tumor,rna,bam,/path/to/PATIENT1-T.rna.bam

Paired tumor and normal DNA with tumor-only RNA

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-N,normal,dna,bam,/path/to/PATIENT1-N.dna.bam
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam,/path/to/PATIENT1-T.dna.bam
PATIENT1,PATIENT1,PATIENT1-T-RNA,tumor,rna,bam,/path/to/PATIENT1-T.rna.bam

Paired tumor and normal DNA with donor sample

Including a donor sample in some types of analyses can be beneficial (e.g. bone marrow transplant) as this allows for germline variant subtraction using both the patient’s normal sample and the bone marrow donor’s normal sample.

To include a donor sample in an analysis, specify donor in the sample_type field with a unique sample identifier in the sample_id field:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-N,normal,dna,bam,/path/to/PATIENT1-N.dna.bam
PATIENT1,PATIENT1,PATIENT1-D,donor,dna,bam,/path/to/PATIENT1-D.dna.bam
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam,/path/to/PATIENT1-T.dna.bam

Multiple samples

To run with multiple samples, specify a different group_id and subject_id for each desired grouping:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-N,normal,dna,bam,/path/to/PATIENT1-N.dna.bam
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam,/path/to/PATIENT1-T.dna.bam
PATIENT2,PATIENT2,PATIENT2-N,normal,dna,bam,/path/to/PATIENT2-N.dna.bam
PATIENT2,PATIENT2,PATIENT2-T,tumor,dna,bam,/path/to/PATIENT2-T.dna.bam

Reference data

The reference data used in oncoanalyser corresponds to and includes reference genomes and their indexes, WiGiTS resources files, and panel specific resource files. Descriptions for each file can be found on the WiGiTS resource file documentation page.

We use the Cloudflare R2 egress-free object store to distribute reference data, and serve files through the https://data.oncoanalyser.com/r2/ domain path.

Staging reference data

By default oncoanalyser will download the required pre-configured reference data (based on the provided samplesheet and CLI arguments) to the Nextflow work directory during every run before proceeding with the analysis.

However, strongly recommended to first stage and configure reference data to avoid repeated retrieval when performing multiple oncoanalyser analyses. See the below for instructions.

Automatic staging

The reference data required for running oncoanalyser can be staged automatically using --mode prepare_reference and specifying --ref_data_types. The below example command will stage the required GRCh38_hmf genome (and indexes) and WiGiTS resources files for WGS analysis from alignments.

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -profile docker \
  --mode prepare_reference \
  --ref_data_types wgs \
  --genome GRCh38_hmf \
  --outdir output/

Once the above commands complete, the stated reference data can be found in <outdir>/reference_data/3.0.0. You will then need to provide a config file that points to these reference files (see Configuring reference data) which can be used for subsequent oncoanalyser runs. The Nextflow work directory can also be removed to free up disk space.

The below table shows the possible values for --ref_data_types. Note that multiple can be provided as comma separated list, e.g. --ref_data_types wgs,dna_alignment

Manual staging

Where automatic staging cannot be used, reference data files can be downloaded manually from the links provided in Reference data URLs. Any tarball should be extracted after download (using tar -xzvf <file>.tar.gz).

To use locally staged reference data, see Configuring reference data.

Configuring reference data

For oncoanalyser to use locally staged (or custom) reference data, the relevant settings can be defined in a configuration file:

params {
    genomes {
        GRCh38_hmf {
            fasta         = "/path/to/GRCh38_masked_exclusions_alts_hlas.fasta"
            fai           = "/path/to/GRCh38_masked_exclusions_alts_hlas.fasta.fai"
            dict          = "/path/to/GRCh38_masked_exclusions_alts_hlas.fasta.dict"
            img           = "/path/to/GRCh38_masked_exclusions_alts_hlas.fasta.img"
            bwamem2_index = "/path/to/bwa-mem2_index/"
            gridss_index  = "/path/to/gridss_index/"
            star_index    = "/path/to/star_index/"
        }
    }
    ref_data_hmf_data_path   = "/path/to/hmftools_data/"
    ref_data_panel_data_path = "/path/to/panel_data/"
}

The configuration file can then be supplied to oncoanalyser via the -config <file> argument:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -config reference_data.config  \
  <...>

Custom genomes

It is strongly recommended to use a Hartwig-distributed reference genome for alignments and subsequent analysis (GRCh37_hmf or GRCh38_hmf). Where it is not feasible to do so, a custom genome can instead be used by providing the relevant FASTA file in a configuration file:

params {
    genomes {
        CustomGenome {
            fasta = "/path/to/custom_genome.fa"
        }
    }
}

Each index can then be created in by using --mode prepare_reference and --ref_data_types (see section staging reference data). The below example command would create the indexes for WGS analysis:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -config genome.custom.config \
  -profile docker \
  --mode prepare_reference \
  --ref_data_types wgs,bwamem2_index,gridss_index
  --genome CustomGenome \
  --genome_version <37|38> \
  --genome_type <alt|no_alt> \
  --force_genome \
  --outdir output/

If aligning FASTQs from RNA seq data for WTS analysis, you should also provide star_index to --ref_data_types. Creating the STAR index also requires transcript annotations; please provide either of the following GTF files via the --ref_data_genome_gtf option after decompressing:

• GRCh37: GENCODE v37 (Ensembl v74) annotations
• GRCh38: GENCODE v38 (Ensembl v104) annotations

When creating indexes for reference genomes with alternative haplotypes, an ALT file must be given with --ref_data_genome_alt. Importantly, a STAR index will not be generated for reference genomes with alternative haplotypes since this requires careful processing and is hence left to the user.

Reference data URLs

GRCh37 genome (Hartwig): GRCh37_hmf

GRCh38 genome (Hartwig): GRCh38_hmf

Pipeline modes

Whole genome / transcriptome sequencing (WGTS)

--mode wgts is used for analysing of whole genome (WGS) and/or whole transcriptome (WTS) sequencing data, and can be run like so:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -config reference_data.config \
  -profile docker \
  --mode wgts \
  --genome GRCh38_hmf \
  --input samplesheet.csv \
  --outdir output/

Targeted sequencing

--mode targeted together with --panel <panel_name> is used for analysing targeted or panel sequencing samples. Currently, only the TSO500 (GRCh37) panel has built-in support.

A typical run command for TSO500 would be:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -config reference_data.config \
  -profile docker \
  --mode targeted \
  --panel tso500 \
  --genome GRCh37_hmf \
  --input samplesheet.csv \
  --outdir output/

Custom panels (i.e. those without built-in support) require custom reference data to be created, as well as additional arguments provided. Custom panels may require manual UMI processing configuration.

Purity estimate

--mode purity_estimate uses WISP to estimate the tumor fraction (aka purity) for a longitudinal sample (e.g. ctDNA sample) guided by variants identified in a primary sample of the same patient (e.g. primary tissue biopsy). This can be used for example for detecting minimal residual disease (MRD).

The primary sample must first have been run in either WGTS or targeted mode.

Purity estimate of the longitudinal sample can likewise be run in two modes:

• --purity_estimate_mode wgts: relies on SNVs, CNVs, and optionally LOH
• --purity_estimate_mode targeted: relies on SNVs only

The minimal inputs required are:

• Alignments or REDUX alignments from the longitudinal tumor sample
• PURPLE directory from the primary tumor sample

In the samplesheet the longitudinal sample must have longitudinal_sample set in the info field, for example a minimal samplesheet starting from a primary sample PURPLE directory and longitudinal sample REDUX directory:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,info,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,purple_dir,,/path/to/PATIENT1-T/purple/
PATIENT1,PATIENT1,PATIENT1-L,tumor,dna,redux_dir,longitudinal_sample,/path/to/redux_PATIENT1-L/

Similarly, a minimal samplesheet instead starting from FASTQ for the longitudinal sample:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,info,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,purple_dir,,/path/to/PATIENT1-T/purple/
PATIENT1,PATIENT1,PATIENT1-L,tumor,dna,redux_dir,longitudinal_sample;library_id:PATIENT1-L_library;lane:001,/path/to/PATIENT1-L.R1.fastq.gz;/path/to/PATIENT1-L.R2.fastq.gz

In --purity_estimate_mode wgts, to use LOH for purity estimation, you can optionally provide:

• AMBER directory from the primary tumor sample AND
• REDUX alignments from the primary normal sample

For example:

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,info,filepath
PATIENT1,PATIENT1,PATIENT1-N,normal,dna,redux_dir,,/path/to/redux_PATIENT1-N/
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,purple_dir,,/path/to/PATIENT1-T/purple/
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,amber_dir,,/path/to/PATIENT1-T/amber/
PATIENT1,PATIENT1,PATIENT1-L,tumor,dna,redux_dir,longitudinal_sample,/path/to/redux_PATIENT1-L/

Then run oncoanalyser providing --mode purity_estimate and --purity_estimate_mode <wgts|targeted>:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -config reference_data.config \
  -profile docker \
  --mode purity_estimate \
  --purity_estimate_mode targeted \
  --genome GRCh38_hmf \
  --input samplesheet.purity_estimate.csv \
  --outdir output/

Prepare reference data

--mode prepare_reference assists with staging all the reference data required to run oncoanalyser. Please see: Staging reference data: Automatic staging

Custom panels

Running custom panels with oncoanalyser involves 3 mains steps:

• Manually create some panel resource files for running --mode panel_resource_creation
• Run --mode panel_resource_creation to create the remaining panel resource files
• Run --mode targeted with the created panel resource files

The panel resource files are used to fit and normalise the biases inherent to the custom panel.

Panel resource files

These files need to be manually created to run --mode panel_resource_creation (instructions in WiGiTS targeted analysis readme):

• driver_gene_panel: Configuration for which driver gene events are to be reported
• target_region_bed: Bed file defining panel regions
• isofox_gene_ids: [RNA] List of gene names and IDs
• isofox_counts: [RNA] Expected fragment counts per transcript and gene. Optional: If not provided, defaults to read_151_exp_counts.<ref_genome_version>.csv from WiGiTS reference data
• isofox_gc_ratios: [RNA] Expected GC ratios per transcript. Optional: If not provided, defaults to read_100_exp_gc_ratios.<ref_genome_version>.csv from WiGiTS reference data

These files are generated after running --mode panel_resource_creation:

• target_region_normalisation: For normalising panel copy number levels to whole genome levels
• pon_artefacts: For variant filtering
• isofox_tpm_norm: [RNA] For normalising panel TPM levels to whole transcriptome levels

Files marked as ‘[RNA]’ are only required if your panel supports RNA-seq data.

Panel resource creation

Once your manually created files are ready, create a samplesheet with a representative set of panel sequencing samples (≥20 recommended). The below example samplesheet provides BAM files, but FASTQ files can also be provided. RNA samples are only required if your panel supports RNA-seq data.

group_id,subject_id,sample_id,sample_type,sequence_type,filetype,filepath
PATIENT1,PATIENT1,PATIENT1-T,tumor,dna,bam,/path/to/PATIENT1-T.dna.bam
PATIENT2,PATIENT2,PATIENT2-T,tumor,dna,bam,/path/to/PATIENT2-T.dna.bam
PATIENT1,PATIENT1,PATIENT1-T-RNA,tumor,rna,bam,/path/to/PATIENT1-T.rna.bam
PATIENT2,PATIENT2,PATIENT2-T-RNA,tumor,rna,bam,/path/to/PATIENT2-T.rna.bam

Then, run oncoanalyser with --mode panel_resource_creation providing the samplesheet, as well as the relevant manually created files to --driver_gene_panel and --target_regions_bed:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -config reference_data.config \
  -profile docker \
  --mode panel_resource_creation \
  --genome GRCh38_hmf \
  --input samplesheet.panel_resource_creation.csv \
  --outdir output/ \
  --driver_gene_panel DriverGenePanel.38.tsv \
  --target_regions_bed target_regions_definition.38.bed.gz \

If your panel supports RNA-seq, also specify --isofox_gene_ids and optionally --isofox_counts and --isofox_gc_ratios:

  --isofox_gene_ids rna_gene_ids.csv \
  --isofox_counts read_151_exp_counts.38.csv \
  --isofox_gc_ratios read_100_exp_gc_ratios.38.csv \

If your panel is for solid tumors, we recommend setting the below arg, which enables calculation of copy number normalisation factors based on per-region copy number percentiles derived from the Hartwig WGS solid tumor cohort:

  --enable_cn_norm_with_wgs_pct

This prevents commonly amplified regions/genes from being normalised to too low copy numbers in your panel samples, and commonly deleted regions/genes from being normalised to too high copy numbers.

Running targeted mode with custom panel

Place all the custom panel reference data files in a directory. In a config file:

• Define the reference data directory with ref_data_panel_data_path and file names with panel_data_paths
• Set panel = <name> and force_panel = true.
• You may also need to configure other parameters such as for UMI processing.

params {

    ref_data_panel_data_path = '/directory/containing/my_custom_panel_resources/'
    panel_data_paths {
        my_custom_panel {
            '38' { // Genome version: '37' or '38'
                // These are relative paths within the dir provided by `ref_data_panel_data_path` above

                driver_gene_panel           = 'driver_genes.38.tsv'
                pon_artefacts               = 'pon_artefacts.38.tsv.gz'
                target_region_bed           = 'panel_definition.38.bed.gz'
                target_region_normalisation = 'cobalt_normalisation.38.tsv'

                // These are not required and left unset by providing an empty list `[]`
                msi_model_error_rates       = [] // Currently defaults to TS0500 panel error rates for all custom panels
                known_umis                  = [] // Only required for MSK-IMPACT panel

                // Only for panels with RNA-seq. Provide e.g. `isofox_counts = []` for panels without RNA
                isofox_tpm_norm             = 'isofox.gene_normalisation.38.csv'
                isofox_counts               = 'read_151_exp_counts.38.csv'
                isofox_gc_ratios            = 'read_151_exp_gc_ratios.38.csv'
            }
        }
    }

    // Same as CLI arg `--panel <name>`. Should match name defined in `panel_data_paths`
    panel = 'my_custom_panel'

    // Same as CLI arg `--force_panel`. Enable non-built-in panels
    force_panel = true
}

Run oncoanalyser with --mode targeted and -config panel.config:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -config reference_data.config \
  -config panel.config \
  -profile docker \
  --mode targeted \
  --genome GRCh38_hmf \
  --input samplesheet.csv \
  --outdir output/

UMI processing

Unique molecular identifiers (UMI) allow for read deduplication and error correction. UMI processing is performed by fastp or fastq-tools for FASTQ files, and REDUX for alignment files.

We recommend using the --umi_type argument to automatically configure the above tools with the correct arguments. Currently, the supported --umi_type values are tso500, twist, kapa, or msk.

When running oncoanalyser in targeted mode with --panel tso500, --umi_type tso500 will automatically be set (you won’t need to set this).

Providing --umi_type twist for example is equivalent to the below config. You can also use the below config as a template for configuring custom UMIs.

params {
    fastp_umi_enabled = true        // Enable UMI stripping by fastp
    fastp_umi_location = "per_read" // --umi_loc fastp arg
    fastp_umi_length = 7            // --umi_len fastp arg
    fastp_umi_skip = 0              // --umi_skip fastp arg

    fastq_tools_umi_enabled = false // Enable UMI stripping by fastq-tools
    fastq_tools_umi_delim = ""      // UMI delimiter

    redux_umi_enabled = true        // Enable UMI processing by REDUX
    redux_umi_duplex_delim = "_"    // Duplex UMI delimiter
}

Below is an R1 FASTQ entry before and after TWIST UMI stripping:

Before:

@LH00144:313:22YY5KLT3:3:1101:3043:1016 1:N:0:GCGACCGATT+NCAAGTGCAA
GNTAACTCAGAATCATAGAGTATGCACCATTTTTTNTNAATCTGGGTGGTAGTTACACAGGTGTGTCTATACAGA
+
I#IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII#I#IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

After:

• Stripped first 7 bases GNTAACT from R1 (and NGCATCT from R2; not shown here)
• UMI appended to header as GNTAACT_NGCATCT

@LH00144:313:22YY5KLT3:3:1101:3043:1016:GNTAACT_NGCATCT 1:N:0:GCGACCGATT+NCAAGTGCAA
CAGAATCATAGAGTATGCACCATTTTTTNTNAATCTGGGTGGTAGTTACACAGGTGTGTCTATACAGA
+
IIIIIIIIIIIIIIIIIIIIIIIIIIII#I#IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

Process selection

It is possible to exclude or manually select specific processes when running oncoanalyser. The full list of processes that can be selected is available here.

Excluding processes

Most of the major components in oncoanalyser can be skipped using the --processes_exclude argument. You may want to skip resource intensive processes like Virusbreakend, or ORANGE because you do not require the report, for example:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -profile docker \
  --mode wgts \
  --processes_exclude virusinterpreter,orange \
  --genome GRCh38_hmf \
  --input samplesheet.csv \
  --outdir output/

Manual process selection

The --processes_manual argument can be used to select the exact processes that onconalyser will run. For example, you may only want to run alignment and SNV/indel, SV and CNV calling from DNA FASTQs, like so:

nextflow run nf-core/oncoanalyser \
  -revision 3.0.0 \
  -profile docker \
  --mode wgts \
  --processes_manual alignment,redux,amber,cobalt,esvee,sage,pave,purple \
  --genome GRCh38_hmf \
  --input samplesheet.csv \
  --outdir output/

Core Nextflow arguments

-profile

Use this parameter to choose a configuration profile. Profiles can give configuration presets for different compute environments.

Several generic profiles are bundled with the pipeline which instruct the pipeline to use software packaged using different methods (Docker, Singularity, Podman, Shifter, Charliecloud, Apptainer, Conda) - see below.

The pipeline also dynamically loads configurations from https://github.com/nf-core/configs when it runs, making multiple config profiles for various institutional clusters available at run time. For more information and to check if your system is supported, please see the nf-core/configs documentation.

Note that multiple profiles can be loaded, for example: -profile test,docker - the order of arguments is important! They are loaded in sequence, so later profiles can overwrite earlier profiles.

If -profile is not specified, the pipeline will run locally and expect all software to be installed and available on the PATH. This is not recommended, since it can lead to different results on different machines dependent on the computer environment.

• test
  • A profile with a complete configuration for automated testing
  • Includes links to test data so needs no other parameters
• docker
  • A generic configuration profile to be used with Docker
• singularity
  • A generic configuration profile to be used with Singularity
• podman
  • A generic configuration profile to be used with Podman
• shifter
  • A generic configuration profile to be used with Shifter
• charliecloud
  • A generic configuration profile to be used with Charliecloud
• apptainer
  • A generic configuration profile to be used with Apptainer
• wave
  • A generic configuration profile to enable Wave containers. Use together with one of the above (requires Nextflow 24.03.0-edge or later).
• conda
  • A generic configuration profile to be used with Conda. Please only use Conda as a last resort i.e. when it’s not possible to run the pipeline with Docker, Singularity, Podman, Shifter, Charliecloud, or Apptainer.

-resume

Specify this when restarting a pipeline. Nextflow will use cached results from any pipeline steps where the inputs are the same, continuing from where it got to previously. For input to be considered the same, not only the names must be identical but the files’ contents as well. For more info about this parameter, see this blog post.

You can also supply a run name to resume a specific run: -resume [run-name]. Use the nextflow log command to show previous run names.

-c

Specify the path to a specific config file (this is a core Nextflow command). See the nf-core website documentation for more information.

Custom configuration

Custom configuration can be provided to oncoanalyser by providing a config file to the CLI argument -config <file> or -c <file>. Syntax and examples of config items are described in the Nextflow documentation and nf-core documentation. Below subsections describe common use cases for custom configuration.

Compute resources

Compute resources (e.g. CPUs, RAM, disk space) can be configured in oncoanalyser if the defaults are not sufficient for one or more processes. Please see Usage: Compute resources for recommendations on compute resource configuration.

To change the resource requests, please see the max resources and customise process resources section of the nf-core website.

Container images

Custom containers

In some cases, you may wish to change the container or conda environment used by a pipeline steps for a particular tool. By default, nf-core pipelines use containers and software from the biocontainers or bioconda projects. However, in some cases the pipeline specified version maybe out of date.

To use a different container from the default container or conda environment specified in a pipeline, please see the updating tool versions section of the nf-core website.

Default containers

By default, nf-core pipelines use containers and software from the biocontainers or bioconda projects.

To use a different container from the default container or conda environment specified in a pipeline, please see the updating tool versions section of the nf-core website.

Below are links to these default images should you want to download images manually (e.g. to run oncoanalyser offline).

Docker (Bioconda)

• Host: quay.io
• Repo URL example: https://quay.io/repository/biocontainers/hmftools-redux?tab=tags
• Image URI example: quay.io/biocontainers/hmftools-redux:1.1--hdfd78af_1

Singularity (Bioconda)

• Host: Galaxy Project
• Image URI example: https://depot.galaxyproject.org/singularity/hmftools-redux:1.1--hdfd78af_1

Bioconda recipes for the above containers are found here:

• Host: bioconda/bioconda-recipes
• Recipe example: https://github.com/bioconda/bioconda-recipes/tree/master/recipes/hmftools-redux

Docker images built by Hartwig’s CI/CD infrastructure are also available, intended for beta releases and not used by default in oncoanalyser

• Host: Dockerhub
• Repo URL example: https://hub.docker.com/r/hartwigmedicalfoundation/redux/tags
• Image URI example: docker.io/hartwigmedicalfoundation/redux:1.1

Container configuration

All configuration options for containers can be found in the Nextflow configuration documentation. A typical config might look like this:

singularity {
    enabled = true
    cacheDir = '/path/to/cache_dir/'
    autoMounts = true
    runOptions = "-B </path/to/desired/mounted/volume/>"
    pullTimeout = '2h'
}

Executors

The executor is a Nextflow component that allows to submission of jobs for example via SLURM (typically on an HPC), AWS Batch, or Google Batch.

To enable SLURM for example, you would provide the below config:

process {
    executor = "slurm"
}

Additional options for the enabled executor can be provided to the executor directive as shown below. See the Config: Executor Nextflow documentation for all options.

executor {
    queueSize         = 100
    queueStatInterval = '10 sec'
    pollInterval      = '10 sec'
    submitRateLimit   = '10 sec'
}

Custom tool arguments

A pipeline might not always support every possible argument or option of a particular tool used in pipeline. Fortunately, nf-core pipelines provide some freedom to users to insert additional parameters that the pipeline does not include by default.

To learn how to provide additional arguments to a particular tool of the pipeline, please see the customising tool arguments section of the nf-core website.

nf-core/configs

In most cases, you will only need to create a custom config as a one-off but if you and others within your organisation are likely to be running nf-core pipelines regularly and need to use the same settings regularly it may be a good idea to request that your custom config file is uploaded to the nf-core/configs git repository. Before you do this please can you test that the config file works with your pipeline of choice using the -c parameter. You can then create a pull request to the nf-core/configs repository with the addition of your config file, associated documentation file (see examples in nf-core/configs/docs), and amending nfcore_custom.config to include your custom profile.

See the main Nextflow documentation for more information about creating your own configuration files.

If you have any questions or issues please send us a message on Slack on the #configs channel.

Running in the background

Nextflow handles job submissions and supervises the running jobs. The Nextflow process must run until the pipeline is finished.

The Nextflow -bg flag launches Nextflow in the background, detached from your terminal so that the workflow does not stop if you log out of your session. The logs are saved to a file.

Alternatively, you can use screen / tmux or similar tool to create a detached session which you can log back into at a later time. Some HPC setups also allow you to run nextflow within a cluster job submitted your job scheduler (from where it submits more jobs).

Nextflow memory requirements

In some cases, the Nextflow Java virtual machines can start to request a large amount of memory. We recommend adding the following line to your environment to limit this (typically in ~/.bashrc or ~/.bash_profile):

NXF_OPTS='-Xms1g -Xmx4g'