Introduction
This document describes the output produced by the pipeline.
The directories listed below will be created in the results directory after the pipeline has finished. All paths are relative to the top-level results directory.
Pipeline overview
The pipeline is built using Nextflow and processes data using the following steps:
- Assembly - Metagenomic assembly of raw PacBio HiFi reads.
- Assembly QC - QC of metagenome assemblies including statistics and rRNA identification.
- Read mapping - Mapping of PacBio HiFi reads and Illumina Hi-C reads to the assembly for coverage estimation and contact map generation.
- Binning - Binning of total metagenome assemblies into genome bins.
- Bin refinement - Refining of genome bins by assessing single-copy gene content.
- Bin QC - QC of genome bins including basic statistics, rRNA content assessment, and tRNA annotation.
- Bin taxonomy - Taxonomic classification of bins with GTDB-Tk and conversion of these classifications to NCBI names.
- Pipeline summary - Summarising key information into a final table, scoring and classification of bins into quality categories according to completeness, contamination, tRNA and rRNA content.
- Pipeline information - Report metrics generated during the workflow execution.
Output directory structure
If a single sample is assembled, the specified output directory will contain the results for that single sample. However, if multiple samples are assembled, the output directory will contain a subdirectory for each sample, named according to the sample ID specified in the input samplesheet. Each sample subdirectory will contain the same structure as described below.
Assembly
Assembly of raw input reads using a long-read assembler, either metaMDBG or myloasm myloasm.
Output files
assembly/[sampleid].contigs.fasta.gz: If metaMDBG is used as the assembler, the output contigs.[sampleid].assembly_primary.fa.gz: If myloasm is used as the assembler, the output contigs.metamdbg/[sampleid].metaMDBG.log: log file detailing metaMDBG assembly process.myloasm/: full output of myloasm, including assembly graphs.
Assembly analyis
Analyses on the primary genome assembly, including basic statistics (GFAStats), ncRNA annotation (tRNAScan-SE, Infernal for rRNA genes), Genomad classification of contigs as plasmids or viruses, and Tiara domain classifications of contigs.
Output files
assembly/[sampleid].{contigs,assembly_primary}.{fasta,fa}.assembly_summary: GFAStats ssummary of assemblytrnascanse/[sampleid].trna,{gff,log,stats,tsv}: outputs of tRNAScan-SE including a GFF of tRNA annotations.rrna/[sampleid].gff: GFF file of rRNA annotationsgenomad/[sampleid]/: Outputs from Genomadtiara/[sampleid].txt: Tiara domain classifications by contigtiara/log_[sampleid].txt: Tiara classification log file
Read mapping
Mapping of long reads to the assembly using minimap2, and Hi-C reads to the assembly using bwa-mem2. Mean coverage estimation of contigs using CoverM. BAM files are only published if --save_bam is set.
Output files
assembly/mapping/[sampleid].[readid].[sequencing_platform].bam: Alignment BAM of reads from sample [readid] to assembly [sampleid].[sampleid].all.depth.tsv: TSV of per-contig mean coverages estimated using CoverM.[sampleid].hic.bam: Alignment BAM of HiFi reads to the assembly.
Binning
Binning of assembled contigs using MetaBat2, MaxBin2, and Metator (Hi-C binning).
Output files
binning/bins/[binner]/fasta/[sampleid].[binner]_{n}.*.fa.gz: Bins in gzipped fasta format output by the given binner.[binner]/*: Log files and other output from each binner.
Bin refinement
Refinement of genome bins using DAS_Tool and Binette.
Output files
binning/bins/[binner]/fasta/[sampleid].[binner]_{n}.*.fa.gz: Bins in gzipped fasta format output by the given binner.[binner]/*: Log files and other output from each binner.
Bin QC
QC of genome bins, including summary statistics using Seqkit, completeness/contamination assessment using CheckM2, rRNA identification using the assembly rRNA annotations, and tRNA annotation using tRNAscan-SE.
Output files
binning/[sampleid]_checkm2_report.tsv: TSV of single-copy-gene checking results for all bins from CheckM2.[sampleid].coverm.genome.tsv: TSV of mean coverage of each bin in each sample.
Bin Taxonomy
Taxonomic classification of bins with GTDB-TK and conversion of GTDB taxonomy classifications to NCBI classifications using TaxonKit.
Output files
binning/gtdbtk/[sampleid]/[sampleid].summary.tsv: GTDB-Tk summary TSV with classifications for each bin.gtdbtk/[sampleid]/[sampleid]_ncbi.tsv: TSV file containing the GTDB-Tk to NCBI classification translation.gtdbtk/[sampleid]/[sampleid].classify.tree.gz: Reference tree in Newick format containing query genomes placed with pplacer.gtdbtk/[sampleid]/[sampleid].markers_summary.tsv: A summary of unique, duplicated, and missing markers within the 120 bacterial marker set, or the 53 archaeal marker set for each submitted genome.gtdbtk/[sampleid]/[sampleid].*msa.fasta.gz: FASTA files containing MSA of submitted and reference genomes.gtdbtk/[sampleid]/[sampleid].filtered.tsv: A list of genomes with an insufficient number of amino acids in MSA.gtdbtk/[sampleid]/[sampleid].failed_genomes.tsv: TSV of genomes which failed classification by GTDB-TK.gtdbtk/[sampleid]/[sampleid].log: The console output of GTDB-Tk saved to disk.gtdbtk/[sampleid]/[sampleid].warnings.log: The verbose output of any GTDB-Tk warnings which were encountered.[sampleid].gtdb_summary.csv: Combined summary from GTDB-Tk (archaea and bacteria) with added NCBI classifications.
Bin summary
Summarising key information into a final table, scoring and classification of bins into quality categories according to completeness, contamination, tRNA and rRNA content.
Output files
bins/[sampleid].bins_summary.tsv: Bin level summary with statistics, completeness/contamination checks, ncRNA content, and taxonomic classifications.[sampleid].groups_summary.tsv: Aggregated summary for each assembly:binner combination showing the counts of bins in each MiMAG quality category.
Pipeline information
Output files
pipeline_info/
- Reports generated by Nextflow:
execution_report.html, execution_timeline.html, execution_trace.txt and pipeline_dag.dot/pipeline_dag.svg.
- Reports generated by the pipeline:
pipeline_report.html, pipeline_report.txt and software_versions.yml. The pipeline_report* files will only be present if the --email / --email_on_fail parameter's are used when running the pipeline.
- Reformatted samplesheet files used as input to the pipeline:
samplesheet.valid.csv.
- Parameters used by the pipeline run:
params.json.
Output files
pipeline_info/- Reports generated by Nextflow:
execution_report.html,execution_timeline.html,execution_trace.txtandpipeline_dag.dot/pipeline_dag.svg. - Reports generated by the pipeline:
pipeline_report.html,pipeline_report.txtandsoftware_versions.yml. Thepipeline_report*files will only be present if the--email/--email_on_failparameter's are used when running the pipeline. - Reformatted samplesheet files used as input to the pipeline:
samplesheet.valid.csv. - Parameters used by the pipeline run:
params.json.
- Reports generated by Nextflow:
Nextflow provides excellent functionality for generating various reports relevant to the running and execution of the pipeline. This will allow you to troubleshoot errors with the running of the pipeline, and also provide you with other information such as launch commands, run times and resource usage.