nf-core/stableexpression
This pipeline is designed to identify the most stable genes in one or more expression datasets (RNA-seq / Microarray). The ideal use case is for determining the most suitable RT-qPCR reference genes for a given species.
Introduction
nf-core/stableexpression is a highly scalable bioinformatics pipeline that identifies the most stable genes in one or more expression datasets. These datasets can be downloaded from public databases such as EBI Expression Atlas and NCBI GEO Datasets, or provided directly by the user. Both RNA-seq and Microarray count datasets can be used.
nf-core/stableexpression is available in the Galaxy Toolshed. Ask your Galaxy administrator to install it.
Selection of reference (housekeeping) genes for RT-qPCR / digital PCR
The selection of the most suitable reference (housekeeping) genes for RT-qPCR / digital PCR normalisation is a critical step in the analysis of gene expression data. However, as the expression of all genes is controlled by external factors such as tissue type, developmental stage, stress and age, the concept of a perfect reference gene is unrealistic, and the selection of reference genes must be closely adapted to the user’s experiment.
The nf-core/stableexpression pipeline is ideally suited to preselecting high-confidence candidates for use as RT-qPCR / digital PCR housekeeping genes, as it integrates normalised counts from a specific set of RNA-seq / Microarray data and selects the genes displaying the highest stability under such conditions.
Please read the guidelines for selecting RT-qPCR reference gene candidates before running the pipeline for your reference gene selection.
Pipeline overview
The main inputs are:
- a species name (mandatory)
- keywords for Expression Atlas / GEO search (optional)
- a
YAML/CSVinput file listing your own raw / normalised count datasets (optional).
The pipeline is built using Nextflow and processes data using the following steps:
1. Get accessions from public databases
- Fetch Expression Atlas dataset accessions corresponding to the provided species (and optionally keywords). This step is run by default but is optional. Set
--skip_fetch_eatlas_accessionsto skip it. - Fetch NBCI GEO dataset accessions corresponding to the provided species (and optionally keywords).
This step is optional and not run by default. Set
--fetch_geo_accessionsto run it. ⚠️ Due to the heterogeneous formats of NCBI GEO datasets, the retrieval of expression datasets from NCBI GEO is considered experimental. ⚠️
2. Download data (see usage)
- Download Expression Atlas data if any
- Download NBCI GEO data if any
🔌 At this stage, the datasets that have been downloaded from public databases are merged with the user datasets provided using the --datasets parameter. See usage for more information about user datasets.
3. ID Mapping (see usage)
- Gene IDs are cleaned.
- Map gene IDS to Ensembl IDs using
g:Profilerto standardise gene IDs across datasets (run by default; optional). - Rare gene IDs are filtered out.
4. Sample filtering
Samples showing too many zeros or missing values are removed from the analysis.
5. Normalisation of RNA-seq datasets
- Whenever possible, RNA-seq data should always be provided in their raw form.
- If using normalised data is unavoidable, choose data that have not been log2-transformed.
Raw RNA-seq data
- Unless provided by the user: download a reference annotation from the Ensembl FTPs (Ensembl Genomes or Ensembl) or using the NCBI Dataset API.
- Compute the length of the longest transcript for each gene.
- Normalise the raw RNA-seq data using GeTMM.
RNA-seq data already normalised (not recommended)
Already normalised RNA-seq data are converted to TPM (Transcripts Per Million).
🧮 Following normalisation, log2 is computed on all RNA-seq counts.
6. Data merging & gene filtering
- All the datasets are merged into a single dataframe for each platform (RNA-seq / Microarray).
- Genes with a too high proportion of zeros across samples on a specific platform are filtered out.
7. Imputation of missing values
Missing values are replaced by imputed values using KNN Imputation.
8. Scaling / Correction of batch effects
Pathway A: ReCombat (default)
Correction of batch effect is performed for each platform separately using reComBat.
Pathway B: Quantile normalisation
In case reComBat encounters memory issues, quantile normalisation is proposed as a more scalable alternative.
Please note that quantile normalisation erases the internal structure of the count distributions. Consequently, reCombat is largely preferred over quantile normalisation whenever data fit in memory.
9. General statistics for each gene
General statistics are computed for each gene and for each platform separately (RNAseq and microarray).
10. Division in sections
The full list of genes is divided into multiple sections based on their expression levels. The genes with the highest expression levels are in section 1, and so on.
11. Scoring
- Run optimised, scalable version of Normfinder.
- Run optimised, scalable version of Genorm.
- Compute stability score for each gene and for each platform.
- Compute a global stability score for each gene considering all platforms.
12. Reporting
- Result aggregation
- Make
MultiQCreport - Prepare Dash Plotly app for further investigation of gene / sample counts
Test pipeline
You can test the execution of the pipeline locally with:
nextflow run nf-core/stableexpression -profile test,<docker/apptainer/conda/micromamba/...>Basic usage
If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.
To search the most stable genes in a species considering all public datasets, simply run:
nextflow run nf-core/stableexpression \ -profile <PROFILE (docker / apptainer / singularity / conda / micromamba / ...)> \ --species <SPECIES (examples: arabidopsis_thaliana / "drosophila melanogaster")> \ --outdir <OUTDIR (example: results)> \ -resumePlease provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.
General usage
For more specific scenarios, such as:
- fetching datasets related to specific conditions only
- using your own expression dataset(s)
please refer to the usage documentation.
Resource allocation
For setting pipeline CPU / memory usage, see here.
Profiles
See here for more information about profiles.
Pipeline output
To see the results of an example test run with a full size dataset refer to the results tab on the nf-core website pipeline page. For more details about the output files and reports, please refer to the output documentation.
Troubleshooting
In case of issues with the pipeline, please check the troubleshooting page or report a new issue.
Support us
If you like nf-core/stableexpression, share it to the community and give it a star on GitHub!
Credits
nf-core/stableexpression was originally written and is currently developed by Olivier Coen, first as a personal project and then as a postdoctoral researcher at the EGCE (Université Paris-Saclay, CNRS).
We thank the following people for their assistance in the development of this pipeline:
- Rémy Costa
- Gaspard Ichas
- Shaheen Acheche
- Janine Soares
We also thank the following members of the nf-core organisation for their assistance and reviews:
- Simon Pearce
- Quentin Blampey
- Matthias Hörtenhuber
- Phil Ewels
- Maxime U Garcia
- James A. Fellows Yates
Contributions and Support
If you would like to contribute to this pipeline, please see the contributing guidelines.
For further information or help, don’t hesitate to get in touch on the Slack #stableexpression channel (you can join with this invite).
Citations
If you use nf-core/stableexpression for your analysis, please cite it using the following doi: 10.5281/zenodo.22895924
An extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.
You can cite the nf-core publication as follows:
The nf-core framework for community-curated bioinformatics pipelines.
Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.