In [1]:

%%bash
date

Wed May 25 12:35:03 PDT 2016

In [2]:

%%bash
system_profiler SPSoftwareDataType

Software:

    System Software Overview:

      System Version: OS X 10.8.5 (12F45)
      Kernel Version: Darwin 12.6.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: genefish
      User Name: Sam (Sam)
      Secure Virtual Memory: Enabled
      Time since boot: 105 days 2:33

In [3]:

%%bash
#Uses grep to exclude lines that display serial number and hardware UUID
system_profiler SPHardwareDataType | grep -v [SH][ea]


      Model Name: Mac mini
      Model Identifier: Macmini6,2
      Processor Name: Intel Core i7
      Processor Speed: 2.3 GHz
      Number of Processors: 1
      Total Number of Cores: 4
      L2 Cache (per Core): 256 KB
      L3 Cache: 6 MB
      Memory: 16 GB
      Boot ROM Version: MM61.0106.B03
      SMC Version (system): 2.8f0

In [4]:

%%bash
time cp /Volumes/web/nightingales/O_lurida/20160223_gbs/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_*.gz /Users/Sam/oly_gbs_bgi/

real	14m59.735s
user	0m0.061s
sys	1m2.362s

In [5]:

pwd

Out[5]:

u'/Users/Sam/gitrepos/LabDocs/jupyter_nbs/sam'

In [1]:

cd

/Users/Sam

In [2]:

cd oly_gbs_bgi/

/Users/Sam/oly_gbs_bgi

In [8]:

ls

160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz
160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz
20160525_pyrad/
params.txt

Rename FASTQ files to match R1 and R2 requirements for pyrad demultiplexing¶

In [9]:

mv 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1.fq.gz

In [10]:

mv 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2.fq.gz

In [11]:

ls

160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1.fq.gz
160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2.fq.gz
20160525_pyrad/
params.txt

In [12]:

mv params.txt 20160525_pyrad/params.txt

In [3]:

cd 20160525_pyrad/

/Users/Sam/oly_gbs_bgi/20160525_pyrad

In [14]:

ls

analysis_pyrad/ params.txt

In [15]:

%%bash
cat params.txt

==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==
./                     ## 1. Working directory                                 (all)
/Users/Sam/oly_gbs_bgi/*.fq.gz             	       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)
/Volumes/web/Athaliana/oly_gbs.barcodes	               ## 3. Loc. of barcode file (if not line 18)             (s1)
/usr/local/bin/vsearch-1.11.1/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)
/usr/local/bin/muscle3.8.31_i86darwin64                    ## 5. command (or path) to call muscle                  (s3,s7)
CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)
16                     ## 7. N processors (parallel)                           (all)
6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)
4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)
.88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)
merged                 ## 11. Datatype: rad,gbs,pairgbs,pairddrad,(others:see docs)(all)
4                      ## 12. MinCov: min samples in a final locus             (s7)
3                      ## 13. MaxSH: max inds with shared hetero site          (s7)
oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)
==== optional params below this line ===================================  affected step ==
                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)
                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)
                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)
                       ## 18.opt.: loc. of de-multiplexed data                      (s2)
                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)
                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)
2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)
                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)
4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)
8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)
                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)
                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)
                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)
                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)
                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)
*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)
                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)
50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)
                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)
                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)
                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)
                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)
16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)
==== optional: list group/clade assignments below this line (see docs) ==================
1HL	5	1HL*
1NF	5	1NF*
1SN	5	1SN*

Edited line 11 (Datatype) to reflect paired-end GBS data¶

In [16]:

%%bash
cat params.txt

==** parameter inputs for pyRAD version 3.0.66  **======================== affected step ==
./                     ## 1. Working directory                                 (all)
/Users/Sam/oly_gbs_bgi/*.fq.gz             	       ## 2. Loc. of non-demultiplexed files (if not line 18)  (s1)
/Volumes/web/Athaliana/oly_gbs.barcodes	               ## 3. Loc. of barcode file (if not line 18)             (s1)
/usr/local/bin/vsearch-1.11.1/bin/vsearch                ## 4. command (or path) to call vsearch (or usearch)    (s3,s6)
/usr/local/bin/muscle3.8.31_i86darwin64                    ## 5. command (or path) to call muscle                  (s3,s7)
CWGC                   ## 6. Restriction overhang (e.g., C|TGCAG -> TGCAG)     (s1,s2)
16                     ## 7. N processors (parallel)                           (all)
6                      ## 8. Mindepth: min coverage for a cluster              (s4,s5)
4                      ## 9. NQual: max # sites with qual < 20 (or see line 20)(s2)
.88                    ## 10. Wclust: clustering threshold as a decimal        (s3,s6)
pairgbs                 ## 11. Datatype: rad,gbs,pairgbs,pairddrad,(others:see docs)(all)
4                      ## 12. MinCov: min samples in a final locus             (s7)
3                      ## 13. MaxSH: max inds with shared hetero site          (s7)
oly_gbs_pyrad          ## 14. Prefix name for final output (no spaces)         (s7)
==== optional params below this line ===================================  affected step ==
                       ## 15.opt.: select subset (prefix* only selector)            (s2-s7)
                       ## 16.opt.: add-on (outgroup) taxa (list or prefix*)         (s6,s7)
                       ## 17.opt.: exclude taxa (list or prefix*)                   (s7)
                       ## 18.opt.: loc. of de-multiplexed data                      (s2)
                       ## 19.opt.: maxM: N mismatches in barcodes (def= 1)          (s1)
                       ## 20.opt.: phred Qscore offset (def= 33)                    (s2)
2                      ## 21.opt.: filter: def=0=NQual 1=NQual+adapters. 2=strict   (s2)
                       ## 22.opt.: a priori E,H (def= 0.001,0.01, if not estimated) (s5)
4                      ## 23.opt.: maxN: max Ns in a cons seq (def=5)               (s5)
8                      ## 24.opt.: maxH: max heterozyg. sites in cons seq (def=5)   (s5)
                       ## 25.opt.: ploidy: max alleles in cons seq (def=2;see docs) (s4,s5)
                       ## 26.opt.: maxSNPs: (def=100). Paired (def=100,100)         (s7)
                       ## 27.opt.: maxIndels: within-clust,across-clust (def. 3,99) (s3,s7)
                       ## 28.opt.: random number seed (def. 112233)              (s3,s6,s7)
                       ## 29.opt.: trim overhang left,right on final loci, def(0,0) (s7)
*                      ## 30.opt.: output formats: p,n,a,s,v,u,t,m,k,g,* (see docs) (s7)
                       ## 31.opt.: maj. base call at depth>x<mindepth (def.x=mindepth) (s5)
50                     ## 32.opt.: keep trimmed reads (def=0). Enter min length.    (s2)
                       ## 33.opt.: max stack size (int), def= max(500,mean+2*SD)    (s3)
                       ## 34.opt.: minDerep: exclude dereps with <= N copies, def=1 (s3)
                       ## 35.opt.: use hierarchical clustering (def.=0, 1=yes)      (s6)
                       ## 36.opt.: repeat masking (def.=1='dust' method, 0=no)      (s3,s6)
16                     ## 37.opt.: vsearch max threads per job (def.=6; see docs)   (s3,s6)
==== optional: list group/clade assignments below this line (see docs) ==================
1HL	5	1HL*
1NF	5	1NF*
1SN	5	1SN*

Step 1: De-multiplex reads¶

In [17]:

%%bash
time pyrad -p params.txt -s 1

	First read files names must contain '_R1_'.


     ------------------------------------------------------------
      pyRAD : RADseq for phylogenetics & introgression analyses
     ------------------------------------------------------------


	step 1: sorting reads by barcode
	 
real	0m6.897s
user	0m0.462s
sys	0m0.335s

Rename fastq files again, based on error message above¶

In [18]:

%%bash
mv /Users/Sam/oly_gbs_bgi/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1.fq.gz \
/Users/Sam/oly_gbs_bgi/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz

In [19]:

%%bash
mv /Users/Sam/oly_gbs_bgi/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2.fq.gz \
/Users/Sam/oly_gbs_bgi/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz

In [20]:

ls ../

160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R1_.fq.gz
160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_R2_.fq.gz
20160525_pyrad/

In [5]:

%%bash
time pyrad -p params.txt -s 1

Process is terminated.

In [ ]: