Introduction

This document describes the output produced by the pipeline.

The directories listed below will be created in the results directory after the pipeline has finished. All paths are relative to the top-level results directory.

The directories comply with Tree of Life's canonical directory structure.

Pipeline overview

The pipeline is built using Nextflow and processes data using the following steps:

• Preprocessing
  • Filtering – Filtering PacBio data before alignment
• Alignment and Mark duplicates
  • Short reads – Aligning HiC and Illumina reads using BWAMEM2
  • Oxford Nanopore reads – Aligning ONT reads using MINIMAP2
  • PacBio reads – Aligning PacBio CLR and CCS filtered reads using MINIMAP2
• Alignment post-processing
  • Statistics – Alignment statistics
• Workflow reporting and genomes
  • Reference genome files - Reference genome indices/files
  • Pipeline information - Report metrics generated during the workflow execution

Preprocessing

Filtering

PacBio reads generated using both CLR and CCS technology are filtered using BLAST_BLASTN against a database of adapter sequences. The collated FASTQ of the filtered reads is required by the downstream alignment step. The results from the PacBio filtering subworkflow are currently not set to output.

Alignment and Mark duplicates

Short reads

Short read data from HiC and Illumina technologies is aligned with BWAMEM2_MEM. The sorted and merged alignment files are processed using the SAMTOOLS mark-duplicate workflow. The mark duplicate alignments is output in the CRAM format, along with the index.

Oxford Nanopore reads

Reads generated using Oxford Nanopore technology are aligned with MINIMAP2_ALIGN. The sorted and merged alignment is output in the CRAM format, along with the index.

PacBio reads

The filtered PacBio reads are aligned with MINIMAP2_ALIGN. The sorted and merged alignment is output in the CRAM format, along with the index.

Alignment post-processing

Statistics

The output alignments, along with the index, are used to calculate mapping statistics. Output files are generated using SAMTOOLS_STATS, SAMTOOLS_FLAGSTAT and SAMTOOLS_IDXSTATS.

Workflow reporting and genomes

Reference genome files

A number of genome-specific files are generated by the pipeline because they are required for the downstream processing of the results. These include an unmasked version of the genome by process UNMASK and an index by BWAMEM2_INDEX. They are currently not set to output.

Pipeline information

Nextflow provides excellent functionality for generating various reports relevant to the running and execution of the pipeline. This will allow you to troubleshoot errors with the running of the pipeline, and also provide you with other information such as launch commands, run times and resource usage.