-
Notifications
You must be signed in to change notification settings - Fork 15
Add WDL implementation for Salmon #326
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
8a69a0e
Update Salmon container image to a more current version
PriyankaaXD d89ca54
Add Sprocket unit tests for Salmon WDL tasks
PriyankaaXD 58108ff
Fix fldMean/fldSD argument quoting and suppress SC2086 false-positive
PriyankaaXD 23aee50
Move test file to tools/test/ and rename to .yaml extension
PriyankaaXD 61ca77b
Fix single-end quant mode (-r flag), rebuild test fixtures from share…
PriyankaaXD 211befd
Require non-empty array for read_one_fastqs_gz
PriyankaaXD 7764bc0
Expose validateMappings as a Boolean input
PriyankaaXD b58e2f7
Expose decoy-aware indexing as an optional input on build_salmon_index
PriyankaaXD 13b1847
Make memory allocation dynamic for both tasks, based on input size
PriyankaaXD 882635b
Add CHANGELOG entry for Salmon WDL implementation
PriyankaaXD fd6f872
Document how Salmon test fixtures were generated
PriyankaaXD 4b73fad
Rename gentrome variable to fasta, remove 'recommended' wording from …
PriyankaaXD 6d7bc7d
Use existing repo reference FASTAs instead of synthetic transcripts, …
PriyankaaXD 8aa822a
Update to Salmon v2.6.0, rebuild index fixture with new format
PriyankaaXD a5e46e8
Revert fixture to v1 index pending maintainer decision on v2 fixture …
PriyankaaXD 6bc4b66
Use BCR_ABL1 transcriptome fixture for index tests
PriyankaaXD 59d07bc
Support gzipped FASTA input for transcripts_fasta/decoys_fasta, clean…
PriyankaaXD ce0a847
Output raw quant.sf file in addition to tarballed output
PriyankaaXD 1d2910e
Rename task to index, fix decoy conditional logic, add gzip symlink s…
PriyankaaXD c8f5157
chore: add ext help link
adthrasher 389eb05
nit: docs and test tags
a-frantz f798d2b
fix sprocket TOML
a-frantz 53e768d
nit: idiomatic bash
a-frantz 5755914
chore: remove salmon-ignored flag
a-frantz dc4f83d
chore: update to 1.3
adthrasher 32ac3ed
chore: bump memory based on experimental results
adthrasher ec51438
chore: switch to hardcoded memory values for index
adthrasher File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,4 @@ | ||
| [check] | ||
| all_lint_rules = true | ||
| except = ["ContainerUri", "TodoComment", "UnusedInput"] | ||
| deny_notes = true | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Salmon test fixtures | ||
|
|
||
| `salmon_index.tar.gz` - built with Salmon 2.6.0 by running the `index` task against the existing `reference/gencode.v50.BCR_ABL1.transcripts.fa.gz` fixture (a small protein-coding transcriptome subset covering the BCR and ABL1 genes). |
Git LFS file not shown
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,292 @@ | ||
| version 1.3 | ||
|
|
||
| task index { | ||
| meta { | ||
| description: "Builds a Salmon index from a transcriptome FASTA file, for use in quantification" | ||
| outputs: { | ||
| index_tar_gz: "A gzipped TAR file containing the Salmon index files." | ||
| } | ||
| } | ||
|
|
||
| parameter_meta { | ||
| transcripts_fasta: "FASTA format file containing the reference transcriptome to index. Can be gzipped." | ||
| decoys_fasta: { | ||
| description: "Optional FASTA file containing decoy genome sequences to improve mapping specificity. Can be gzipped.", | ||
| group: "Common", | ||
| } | ||
| index_name: { | ||
| description: "Name for the output index, in compressed archive format. The suffix `.tar.gz` will be added.", | ||
| group: "Common", | ||
| } | ||
| use_all_cores: { | ||
| description: "Use all cores? Recommended for cloud environments.", | ||
| group: "Resources", | ||
| } | ||
| ncpu: { | ||
| description: "Number of cores to allocate for task", | ||
| group: "Resources", | ||
| } | ||
| modify_disk_size_gb: { | ||
| description: "Add to or subtract from dynamic disk space allocation. Default disk size is determined by the size of the inputs. Specified in GB.", | ||
| group: "Resources", | ||
| } | ||
| modify_memory_gb: { | ||
| description: "Add to or subtract from dynamic memory allocation. Default memory is determined by the size of the inputs. Specified in GB.", | ||
| group: "Resources", | ||
| } | ||
| } | ||
|
|
||
| input { | ||
| File transcripts_fasta | ||
| File? decoys_fasta | ||
| String index_name = "salmon_index" | ||
| Boolean use_all_cores = false | ||
| Int ncpu = 4 | ||
| Int modify_disk_size_gb = 0 | ||
| Int modify_memory_gb = 0 | ||
| } | ||
|
|
||
| String salmon_index_filename = index_name + ".tar.gz" | ||
|
|
||
| Float transcripts_fasta_size = size(transcripts_fasta, "GB") | ||
| Float decoys_fasta_size = size(decoys_fasta, "GB") | ||
| Int disk_size_gb = ceil(transcripts_fasta_size * 4) + ceil(decoys_fasta_size * 4) + 10 + modify_disk_size_gb | ||
|
|
||
| command <<< | ||
| set -euo pipefail | ||
|
|
||
| n_cores=~{ncpu} | ||
| if ~{use_all_cores}; then | ||
| n_cores=$(nproc) | ||
| fi | ||
|
|
||
| transcripts_name="~{basename(transcripts_fasta, ".gz")}" | ||
| gunzip -c "~{transcripts_fasta}" > "$transcripts_name" || ln -sf "~{transcripts_fasta}" "$transcripts_name" | ||
| fasta="$transcripts_name" | ||
|
|
||
| decoys_name="~{if defined(decoys_fasta) then basename(select_first([decoys_fasta]), ".gz") else ""}" | ||
| if [ -n "$decoys_name" ]; then | ||
| gunzip -c "~{decoys_fasta}" > "$decoys_name" || ln -sf "~{decoys_fasta}" "$decoys_name" | ||
| grep "^>" "$decoys_name" | cut -d " " -f1 | sed "s/^>//" > decoys.txt | ||
| cat "$transcripts_name" "$decoys_name" > combined.fasta | ||
| fasta=combined.fasta | ||
| fi | ||
|
|
||
| salmon index \ | ||
| -t "$fasta" \ | ||
| -i "~{index_name}" \ | ||
| ~{if defined(decoys_fasta) then "-d decoys.txt" else ""} \ | ||
| -p "$n_cores" | ||
|
|
||
| tar -czf "~{salmon_index_filename}" "~{index_name}" | ||
|
|
||
| rm -f "$transcripts_name" "$decoys_name" combined.fasta | ||
| >>> | ||
|
a-frantz marked this conversation as resolved.
|
||
|
|
||
| output { | ||
| File index_tar_gz = salmon_index_filename | ||
| } | ||
|
|
||
| requirements { | ||
| cpu: ncpu | ||
| # Based on n of 1 test with GRCh38 and GENCODE v50. | ||
| memory: "~{if defined(decoys_fasta) then 52 else 5} + 4 + modify_memory_gb} GB" | ||
| disks: "~{disk_size_gb} GB" | ||
| container: "quay.io/biocontainers/salmon:2.6.0--hfa8f182_0" | ||
| maxRetries: 1 | ||
| } | ||
| } | ||
|
|
||
| task quant { | ||
| meta { | ||
| description: "Runs `salmon quant` in mapping-based mode to quantify transcript-level expression from RNA-Seq reads, using a pre-built Salmon index." | ||
| outputs: { | ||
| quant_results_tar_gz: "A gzipped TAR file containing the Salmon quantification output directory, including `quant.sf`.", | ||
| quant_sf: "The raw `quant.sf` file, renamed to `<prefix>.quant.sf`, provided alongside the tarballed output." | ||
| } | ||
| } | ||
|
|
||
| parameter_meta { | ||
| index_tar_gz: "A gzipped TAR file containing the Salmon index files. Suitable as the output of the `index` task." | ||
| read_one_fastqs_gz: "An array of gzipped FASTQ files containing read one information" | ||
| read_two_fastqs_gz: { | ||
| description: "An array of gzipped FASTQ files containing read two information. Omit for single-end reads.", | ||
| group: "Common", | ||
| } | ||
| lib_type: { | ||
| description: "Salmon library type describing the relative orientation and strandedness of paired reads.", | ||
| help: "Use `A` to let Salmon auto-detect the library type - recommended for most users.", | ||
| external_help: "https://combine-lab.github.io/salmon/guides/library-types/", | ||
| group: "Common", | ||
| } | ||
| prefix: { | ||
| description: "Prefix for the Salmon quantification output. The extension `.tar.gz` will be added.", | ||
| group: "Common", | ||
| } | ||
| num_bootstraps: { | ||
| description: "Compute bootstrapped abundance estimates.", | ||
| help: "This is done by resampling (with replacement) from the counts assigned to the fragment equivalence classes, and then re-running the optimization procedure for each such sample.", | ||
| group: "Salmon Options", | ||
| } | ||
| incompat_prior: { | ||
| description: "This parameter governs the a priori probability that a fragment mapping is nonetheless the correct mapping.", | ||
| help: "Specifically, this is for a fragment mapping or aligning to the reference in a manner incompatible with the prescribed library type.", | ||
| group: "Salmon Options", | ||
| } | ||
| range_factorization_bins: { | ||
| description: "The range-factorization feature allows using a data-driven likelihood factorization.", | ||
| help: "This can improve quantification accuracy on certain classes of difficult transcripts.", | ||
| group: "Salmon Options", | ||
| } | ||
| fld_mean: { | ||
| description: "Allows the user to set the expected mean fragment length of the sequencing library.", | ||
| help: "Since the empirical fragment length distribution cannot be estimated from the mappings of single-end reads, this is only important when running Salmon with single-end reads.", | ||
| group: "Salmon Options", | ||
| } | ||
| fld_sd: { | ||
| description: "Allows the user to set the expected standard deviation of the fragment length distribution.", | ||
| help: "Since the empirical fragment length distribution cannot be estimated from the mappings of single-end reads, this is only important when running Salmon with single-end reads.", | ||
| group: "Salmon Options", | ||
| } | ||
| seq_bias: { | ||
| description: "Passing this flag will enable it to learn and correct for sequence-specific biases in the input data.", | ||
| group: "Salmon Options", | ||
| } | ||
| gc_bias: { | ||
| description: "Passing this flag will enable it to learn and correct for fragment-level GC biases in the input data.", | ||
| group: "Salmon Options", | ||
| } | ||
| pos_bias: { | ||
| description: "Passing this flag will enable modeling of a position-specific fragment start distribution.", | ||
| group: "Salmon Options", | ||
| } | ||
| use_em: { | ||
| description: "Use the \"standard\" EM algorithm to optimize abundance estimates instead of the variational Bayesian EM algorithm.", | ||
| group: "Salmon Options", | ||
| } | ||
| recover_orphans: { | ||
| description: "This flag enables orphan \"rescue\" for reads.", | ||
| group: "Salmon Options", | ||
| } | ||
| hard_filter: { | ||
| description: "This flag turns off soft filtering and range-factorized equivalence classes.", | ||
| help: "Removes all but the equally highest scoring mappings from the equivalence class label for each fragment.", | ||
| group: "Salmon Options", | ||
| } | ||
| allow_dovetail: { | ||
| description: "Dovetailing mappings and alignments are considered discordant and discarded by default.", | ||
| help: "If you wish to consider dovetailing mappings as concordant, you can do so by passing this flag.", | ||
| group: "Salmon Options", | ||
| } | ||
| dump_eq: { | ||
| description: "If passed, Salmon will write a file in the auxiliary directory, called eq_classes.txt.", | ||
| help: "Contains the equivalence classes and corresponding counts that were computed during quasi-mapping.", | ||
| group: "Salmon Options", | ||
| } | ||
| write_unmapped_names: { | ||
| description: "Passing this flag will tell Salmon to write out the names of reads (or mates in paired-end reads) that do not map to the transcriptome.", | ||
| group: "Salmon Options", | ||
| } | ||
| use_all_cores: { | ||
| description: "Use all cores? Recommended for cloud environments.", | ||
| group: "Resources", | ||
| } | ||
| ncpu: { | ||
| description: "Number of cores to allocate for task", | ||
| group: "Resources", | ||
| } | ||
| modify_disk_size_gb: { | ||
| description: "Add to or subtract from dynamic disk space allocation. Default disk size is determined by the size of the inputs. Specified in GB.", | ||
| group: "Resources", | ||
| } | ||
| modify_memory_gb: { | ||
| description: "Add to or subtract from dynamic memory allocation. Default memory is determined by the size of the inputs. Specified in GB.", | ||
| group: "Resources", | ||
| } | ||
| } | ||
|
|
||
| input { | ||
| File index_tar_gz | ||
| Array[File]+ read_one_fastqs_gz | ||
| Array[File]? read_two_fastqs_gz | ||
| String lib_type = "A" | ||
| String prefix = sub(basename(read_one_fastqs_gz[0]), "(([_.][rR](?:ead)?[12])((?:[_.-][^_.-]*?)*?))?\\.(fastq|fq)(\\.gz)?$", "") | ||
| Int num_bootstraps = 0 | ||
| Float incompat_prior = 0.0 | ||
| Int range_factorization_bins = 4 | ||
| Int fld_mean = 250 | ||
| Int fld_sd = 25 | ||
| Boolean seq_bias = false | ||
| Boolean gc_bias = false | ||
| Boolean pos_bias = false | ||
| Boolean use_em = false | ||
| Boolean recover_orphans = false | ||
| Boolean hard_filter = false | ||
| Boolean allow_dovetail = false | ||
| Boolean dump_eq = false | ||
| Boolean write_unmapped_names = false | ||
| Boolean use_all_cores = false | ||
| Int ncpu = 4 | ||
| Int modify_disk_size_gb = 0 | ||
| Int modify_memory_gb = 0 | ||
| } | ||
|
|
||
| Array[File] read_twos = select_first([read_two_fastqs_gz, []]) | ||
|
|
||
| Float read_one_size = size(read_one_fastqs_gz, "GB") | ||
| Float read_two_size = size(read_twos, "GB") | ||
| Float index_size = size(index_tar_gz, "GB") | ||
| Int disk_size_gb = ceil((read_one_size + read_two_size + index_size) * 3) + 10 + modify_disk_size_gb | ||
| Int memory_gb = ceil(index_size * 4) + 8 + modify_memory_gb | ||
|
|
||
| command <<< | ||
| set -euo pipefail | ||
|
|
||
| n_cores=~{ncpu} | ||
| if ~{use_all_cores}; then | ||
| n_cores=$(nproc) | ||
| fi | ||
|
|
||
| mkdir salmon_index | ||
| tar -xzf "~{index_tar_gz}" -C salmon_index --strip-components 1 | ||
|
|
||
| # shellcheck disable=SC2086 | ||
| salmon quant \ | ||
| -i salmon_index \ | ||
| -l "~{lib_type}" \ | ||
| ~{if length(read_twos) > 0 then "-1 " + sep(" ", squote(read_one_fastqs_gz)) + " -2 " + sep(" ", squote(read_twos)) else "-r " + sep(" ", squote(read_one_fastqs_gz))} \ | ||
| -p "$n_cores" \ | ||
| --numBootstraps ~{num_bootstraps} \ | ||
| --incompatPrior ~{incompat_prior} \ | ||
| --rangeFactorizationBins ~{range_factorization_bins} \ | ||
| ~{if length(read_twos) == 0 then "--fldMean " + fld_mean else ""} \ | ||
| ~{if length(read_twos) == 0 then "--fldSD " + fld_sd else ""} \ | ||
| ~{if seq_bias then "--seqBias" else ""} \ | ||
| ~{if gc_bias then "--gcBias" else ""} \ | ||
| ~{if pos_bias then "--posBias" else ""} \ | ||
| ~{if use_em then "--useEM" else ""} \ | ||
| ~{if recover_orphans then "--recoverOrphans" else ""} \ | ||
| ~{if hard_filter then "--hardFilter" else ""} \ | ||
| ~{if allow_dovetail then "--allowDovetail" else ""} \ | ||
| ~{if dump_eq then "--dumpEq" else ""} \ | ||
| ~{if write_unmapped_names then "--writeUnmappedNames" else ""} \ | ||
| -o "~{prefix}" | ||
|
|
||
| cp "~{prefix}/quant.sf" "~{prefix}.quant.sf" | ||
|
|
||
| tar -czf "~{prefix}.tar.gz" "~{prefix}" | ||
| >>> | ||
|
|
||
| output { | ||
|
a-frantz marked this conversation as resolved.
|
||
| File quant_results_tar_gz = prefix + ".tar.gz" | ||
| File quant_sf = prefix + ".quant.sf" | ||
| } | ||
|
|
||
| requirements { | ||
| cpu: ncpu | ||
| memory: "~{memory_gb} GB" | ||
| disks: "~{disk_size_gb} GB" | ||
| container: "quay.io/biocontainers/salmon:2.6.0--hfa8f182_0" | ||
| maxRetries: 1 | ||
| } | ||
| } | ||
|
a-frantz marked this conversation as resolved.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| index: | ||
| - name: builds_index_successfully | ||
|
a-frantz marked this conversation as resolved.
|
||
| tags: | ||
| - reference | ||
| inputs: | ||
| transcripts_fasta: | ||
| - reference/gencode.v50.BCR_ABL1.transcripts.fa.gz | ||
| assertions: | ||
| outputs: | ||
| index_tar_gz: | ||
| - Name: salmon_index.tar.gz | ||
| - name: builds_index_with_decoys | ||
| tags: | ||
| - reference | ||
| - slow | ||
| inputs: | ||
| transcripts_fasta: | ||
| - reference/gencode.v50.BCR_ABL1.transcripts.fa.gz | ||
| decoys_fasta: | ||
| - reference/GRCh38.chrY_chrM.fa | ||
| assertions: | ||
| outputs: | ||
| index_tar_gz: | ||
| - Name: salmon_index.tar.gz | ||
|
|
||
| quant: | ||
| - name: quantifies_paired_end_reads | ||
|
a-frantz marked this conversation as resolved.
|
||
| inputs: | ||
| index_tar_gz: | ||
| - salmon/salmon_index.tar.gz | ||
| read_one_fastqs_gz: | ||
| - - fastqs/test_R1.fq.gz | ||
| read_two_fastqs_gz: | ||
| - - fastqs/test_R2.fq.gz | ||
| assertions: | ||
| outputs: | ||
| quant_results_tar_gz: | ||
| - Name: test.tar.gz | ||
| - name: quantifies_single_end_reads | ||
| inputs: | ||
| index_tar_gz: | ||
| - salmon/salmon_index.tar.gz | ||
| read_one_fastqs_gz: | ||
| - - fastqs/test_R1.fq.gz | ||
| assertions: | ||
| outputs: | ||
| quant_results_tar_gz: | ||
| - Name: test.tar.gz | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.