Notebook

Notebook for getting count matrix for individual crab RNAseq from crab transcriptome v 3.1¶

Note: The individual crab RNAseq was not used in the assembly of crab transcriptome v 3.1¶

I will just be using samples for an individual crab over time: ambient and infected. Library IDs: 178, 359, and 463

Notebook based on this from Steven Roberts: https://github.com/sr320/nb-2020/blob/master/C_bairdi/20-kallisto.ipynb ¶

In [1]:

!/Applications/bioinfo/kallisto/kallisto

kallisto 0.46.1

Usage: kallisto <CMD> [arguments] ..

Where <CMD> can be one of:

    index         Builds a kallisto index 
    quant         Runs the quantification algorithm 
    bus           Generate BUS files for single-cell data 
    pseudo        Runs the pseudoalignment step 
    merge         Merges several batch runs 
    h5dump        Converts HDF5-formatted results to plaintext
    inspect       Inspects and gives information about an index
    version       Prints version information
    cite          Prints citation information

Running kallisto <CMD> without arguments prints usage information for <CMD>

In [2]:

!pwd

/Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks

In [3]:

ls

cbai_transcriptome_31
cbai_transcriptome_v3.1.fasta
kallisto/
kallisto-4libraries.ipynb
kallisto-individual-crab.ipynb
readme.md
transcriptomev3.1-BLAST-to-GOslim.ipynb

In [4]:

!wget --no-check-certificate https://owl.fish.washington.edu/halfshell/genomic-databank/cbai_transcriptome_v3.1.fasta

--2020-08-15 12:57:37--  https://owl.fish.washington.edu/halfshell/genomic-databank/cbai_transcriptome_v3.1.fasta
Resolving owl.fish.washington.edu (owl.fish.washington.edu)... 128.95.149.83
Connecting to owl.fish.washington.edu (owl.fish.washington.edu)|128.95.149.83|:443... connected.
WARNING: cannot verify owl.fish.washington.edu's certificate, issued by ‘CN=InCommon RSA Server CA,OU=InCommon,O=Internet2,L=Ann Arbor,ST=MI,C=US’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 145648199 (139M)
Saving to: ‘cbai_transcriptome_v3.1.fasta.1’

cbai_transcriptome_ 100%[===================>] 138.90M  19.1MB/s    in 11s     

2020-08-15 12:57:48 (13.0 MB/s) - ‘cbai_transcriptome_v3.1.fasta.1’ saved [145648199/145648199]

In [5]:

!/Applications/bioinfo/kallisto/kallisto \
index -i cbai_transcriptome_31 \
cbai_transcriptome_v3.1.fasta

[build] loading fasta file cbai_transcriptome_v3.1.fasta
[build] k-mer length: 31
[build] warning: clipped off poly-A tail (longer than 10)
        from 12560 target sequences
[build] counting k-mers ... done.
[build] building target de Bruijn graph ...  done 
[build] creating equivalence classes ...  done
[build] target de Bruijn graph has 303833 contigs and contains 43881871 k-mers

In [8]:

#sequence files for library 178 (single crab: ambient infected - day 0)
!/Applications/bioinfo/kallisto/kallisto quant \
-i /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/cbai_transcriptome_31 \
-t 4 \
-o kallisto/178_ambient_infected_0/ \
/Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/178_R1_001.fastp-trim.202003181815.fq.gz /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/178_R2_001.fastp-trim.202003181815.fq.gz

[quant] fragment length distribution will be estimated from the data
[index] k-mer length: 31
[index] number of targets: 78,649
[index] number of k-mers: 43,881,871
[index] number of equivalence classes: 213,585
[quant] running in paired-end mode
[quant] will process pair 1: /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/178_R1_001.fastp-trim.202003181815.fq.gz
                             /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/178_R2_001.fastp-trim.202003181815.fq.gz
[quant] finding pseudoalignments for the reads ... done
[quant] processed 29,009,909 reads, 19,091,104 reads pseudoaligned
[quant] estimated average fragment length: 122.091
[   em] quantifying the abundances ... done
[   em] the Expectation-Maximization algorithm ran for 1,141 rounds

In [9]:

#sequence files for library 359 (single crab: ambient infected - day 2)
!/Applications/bioinfo/kallisto/kallisto quant \
-i /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/cbai_transcriptome_31 \
-t 4 \
-o kallisto/359_ambient_infected_2/ \
/Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/359_R1_001.fastp-trim.202003182247.fq.gz /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/359_R2_001.fastp-trim.202003182247.fq.gz

[quant] fragment length distribution will be estimated from the data
[index] k-mer length: 31
[index] number of targets: 78,649
[index] number of k-mers: 43,881,871
[index] number of equivalence classes: 213,585
[quant] running in paired-end mode
[quant] will process pair 1: /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/359_R1_001.fastp-trim.202003182247.fq.gz
                             /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/359_R2_001.fastp-trim.202003182247.fq.gz
[quant] finding pseudoalignments for the reads ... done
[quant] processed 27,021,012 reads, 16,787,891 reads pseudoaligned
[quant] estimated average fragment length: 124.407
[   em] quantifying the abundances ... done
[   em] the Expectation-Maximization algorithm ran for 1,223 rounds

In [10]:

#sequence files for library 463 (single crab: ambient infected - day 17)
!/Applications/bioinfo/kallisto/kallisto quant \
-i /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/cbai_transcriptome_31 \
-t 4 \
-o kallisto/463_ambient_infected_17/ \
/Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/463_R1_001.fastp-trim.202003185732.fq.gz /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/463_R2_001.fastp-trim.202003185732.fq.gz

[quant] fragment length distribution will be estimated from the data
[index] k-mer length: 31
[index] number of targets: 78,649
[index] number of k-mers: 43,881,871
[index] number of equivalence classes: 213,585
[quant] running in paired-end mode
[quant] will process pair 1: /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/463_R1_001.fastp-trim.202003185732.fq.gz
                             /Volumes/web/Atumefaciens/20200318_cbai_RNAseq_fastp_trimming/463_R2_001.fastp-trim.202003185732.fq.gz
[quant] finding pseudoalignments for the reads ... done
[quant] processed 19,061,045 reads, 8,884,132 reads pseudoaligned
[quant] estimated average fragment length: 77.2951
[   em] quantifying the abundances ... done
[   em] the Expectation-Maximization algorithm ran for 1,402 rounds

Create matrix¶

In [11]:

!perl /Applications/bioinfo/trinityrnaseq-v2.8.6/util/abundance_estimates_to_matrix.pl

####################################################################################
#
# Usage:  /Applications/bioinfo/trinityrnaseq-v2.8.6/util/abundance_estimates_to_matrix.pl --est_method <method>  sample1.results sample2.results ...
#
#      or  /Applications/bioinfo/trinityrnaseq-v2.8.6/util/abundance_estimates_to_matrix.pl --est_method <method> --quant_files file.listing_target_files.txt
#
#      Note, if only a single input file is given, it's expected to contain the paths to all the target abundance estimation files.
#
# Required:
#            
#  --est_method <string>           RSEM|eXpress|kallisto|salmon  (needs to know what format to expect)
#
#  --gene_trans_map <string>           the gene-to-transcript mapping file. (if you don't want gene estimates, indicate 'none'.
#
#
# Options:
#
#  --cross_sample_norm <string>         TMM|UpperQuartile|none   (default: TMM)
#
#  --name_sample_by_basedir             name sample column by dirname instead of filename
#      --basedir_index <int>            default(-2)
#
#  --out_prefix <string>                default: value for --est_method
#
#  --quant_files <string>              file containing a list of all the target files.
#
######################################################################################

In [12]:

!perl /Applications/bioinfo/trinityrnaseq-v2.8.6/util/abundance_estimates_to_matrix.pl \
--est_method kallisto \
    --gene_trans_map none \
    --out_prefix /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab \
    --name_sample_by_basedir \
     /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/kallisto/178_ambient_infected_0/abundance.tsv \
     /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/kallisto/359_ambient_infected_2/abundance.tsv \
     /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/kallisto/463_ambient_infected_17/abundance.tsv 

-reading file: /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/kallisto/178_ambient_infected_0/abundance.tsv
-reading file: /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/kallisto/359_ambient_infected_2/abundance.tsv
-reading file: /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/notebooks/kallisto/463_ambient_infected_17/abundance.tsv


* Outputting combined matrix.

/Applications/bioinfo/trinityrnaseq-v2.8.6/util/support_scripts/run_TMM_scale_matrix.pl --matrix /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab.isoform.TPM.not_cross_norm > /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab.isoform.TMM.EXPR.matrixCMD: R --no-save --no-restore --no-site-file --no-init-file -q < /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab.isoform.TPM.not_cross_norm.runTMM.R 1>&2 
dyld: Library not loaded: @rpath/libreadline.6.2.dylib
  Referenced from: /Users/graciecrandall/anaconda/lib/R/lib/libR.dylib
  Reason: image not found
sh: line 1:   808 Abort trap: 6           R --no-save --no-restore --no-site-file --no-init-file -q < /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab.isoform.TPM.not_cross_norm.runTMM.R 1>&2
Error, cmd: R --no-save --no-restore --no-site-file --no-init-file -q < /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab.isoform.TPM.not_cross_norm.runTMM.R 1>&2  died with ret (34304)  at /Applications/bioinfo/trinityrnaseq-v2.8.6/util/support_scripts/run_TMM_scale_matrix.pl line 105.
Error, CMD: /Applications/bioinfo/trinityrnaseq-v2.8.6/util/support_scripts/run_TMM_scale_matrix.pl --matrix /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab.isoform.TPM.not_cross_norm > /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab.isoform.TMM.EXPR.matrix died with ret 6400 at /Applications/bioinfo/trinityrnaseq-v2.8.6/util/abundance_estimates_to_matrix.pl line 385.

In [13]:

!head /Users/graciecrandall/Documents/GitHub/paper-tanner-crab/analyses/kallisto-single_crab_over_time/kallisto-single-crab.isoform.counts.matrix

	178_ambient_infected_0	359_ambient_infected_2	463_ambient_infected_17
TRINITY_DN10815_c0_g1_i1	0	0	0
TRINITY_DN1636_c2_g1_i11	0	20.5709	0
TRINITY_DN23821_c0_g1_i2	0	1.16885	0
TRINITY_DN446_c0_g1_i7	0	0	0
TRINITY_DN112563_c0_g1_i1	1	1	0
TRINITY_DN3547_c2_g1_i14	1.67035e-08	0	0
TRINITY_DN22245_c0_g1_i1	2.74956	0	9.72359
TRINITY_DN91454_c0_g2_i1	0	0	0
TRINITY_DN8799_c0_g1_i6	0	0	0

In [ ]:

Notebook for getting count matrix for individual crab RNAseq from crab transcriptome v 3.1¶

Note: The individual crab RNAseq was not used in the assembly of crab transcriptome v 3.1¶

Notebook based on this from Steven Roberts: https://github.com/sr320/nb-2020/blob/master/C_bairdi/20-kallisto.ipynb¶

Create matrix¶

Notebook based on this from Steven Roberts: https://github.com/sr320/nb-2020/blob/master/C_bairdi/20-kallisto.ipynb ¶