docker run -p 8888:8888 -v /Users/sam/data/:/data -v /Users/sam/owl_home/:/owl_home -v /Users/sam/owl_web/:/owl_web -v /Users/sam/gitrepos:/gitrepos -it f99537d7e06a
The command allows access to Jupyter Notebook over port 8888 and makes my Jupyter Notebook GitHub repo and my data files on Owl/home and Owl/web accessible to the Docker container.
Once the container was started, started Jupyter Notebook with the following command inside the Docker container:
jupyter notebook
This is configured in the Docker container to launch a Jupyter Notebook without a browser on port 8888.
The Docker container is running on an image created from this Dockerfile (Git commit 443bc42)
%%bash
date
Tue Mar 14 21:22:24 UTC 2017
%%bash
hostname
0f2bca9c664b
%%bash
lscpu
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 1 Core(s) per socket: 8 Socket(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 26 Model name: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz Stepping: 5 CPU MHz: 2260.998 BogoMIPS: 4521.99 Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 8192K
cd /gitrepos/
/gitrepos
%%bash
git clone https://github.com/BGI-shenzhen/Reseqtools.git
Cloning into 'Reseqtools'...
%%bash
ls
2bRAD_GATK LabDocs OwlUploader Reseqtools paper_oly_gbs
%%bash
cd Reseqtools/
%%bash
ls
2bRAD_GATK LabDocs OwlUploader Reseqtools paper_oly_gbs
cd Reseqtools/
/gitrepos/Reseqtools
ls
LICENSE README.md iTools_Code20160530.tar.gz
Hi Sam,
We downloaded it and it seems fine when compiling. You can compile it with the below command under Linux system. tar -zxvf ReSeqTools_XXX.tar.gz ; cd iTools_Code; chmod 775 iTools ; ./ iTools -h
%%bash
tar -zxvf ReSeqTools_XXX.tar.gz ; cd iTools_Code; chmod 775 iTools ; ./ iTools -h
tar (child): ReSeqTools_XXX.tar.gz: Cannot open: No such file or directory tar (child): Error is not recoverable: exiting now tar: Child returned status 2 tar: Error is not recoverable: exiting now bash: line 1: cd: iTools_Code: No such file or directory chmod: cannot access 'iTools': No such file or directory bash: line 1: ./: Is a directory
%%bash
cd iTools_Code; chmod 775 iTools ; ./ iTools -h
bash: line 1: cd: iTools_Code: No such file or directory chmod: cannot access 'iTools': No such file or directory bash: line 1: ./: Is a directory
Ooops, my bad. Here we go again...
%%bash
tar -zxf iTools_Code20160530.tar.gz
cd iTools_Code20160530
chmod 775 iTools
./iTools -h
bash: line 2: cd: iTools_Code20160530: No such file or directory chmod: cannot access 'iTools': No such file or directory bash: line 4: ./iTools: No such file or directory
ls
LICENSE README.md iTools_Code/ iTools_Code20160530.tar.gz
%%bash
cd iTools_Code/
chmod 775 iTools
./iTools -h
Error opening terminal: xterm-color.
ls
LICENSE README.md iTools_Code/ iTools_Code20160530.tar.gz
cd iTools_Code/
/gitrepos/Reseqtools/iTools_Code
ls
Example/ ReSeqTools@ config.h.in depcomp@ install-sh@ Install.Readme ReSeqTools.Readme@ config.log document/ missing@ Makefile aclocal.m4 config.status* iTools* src/ Makefile.am autom4te.cache/ configure* iTools.Readme@ stamp-h1 Makefile.in bin/ configure.ac iTools.cpp NEW config.h configure.scan iTools.o
chmod 775 iTools
File "<ipython-input-19-e2a0a72d025a>", line 1 chmod 775 iTools ^ SyntaxError: invalid syntax
%%bash
chmod 775 iTools
./iTools -h
File "<ipython-input-21-32b17f07a71f>", line 1 ./iTools -h ^ SyntaxError: invalid syntax
%%bash
./iTools -h
Error opening terminal: xterm-color.
%%bash
./ iTools -h
bash: line 1: ./: Is a directory
%%bash
make
make all-am make[1]: Entering directory '/gitrepos/Reseqtools/iTools_Code' if g++ -DHAVE_CONFIG_H -I. -I. -I. -g -O2 -MT iTools.o -MD -MP -MF ".deps/iTools.Tpo" -c -o iTools.o iTools.cpp; \ then mv -f ".deps/iTools.Tpo" ".deps/iTools.Po"; else rm -f ".deps/iTools.Tpo"; exit 1; fi Makefile:264: recipe for target 'iTools.o' failed make[1]: Leaving directory '/gitrepos/Reseqtools/iTools_Code' Makefile:174: recipe for target 'all' failed
In file included from /usr/include/c++/4.9/ext/new_allocator.h:33:0, from /usr/include/x86_64-linux-gnu/c++/4.9/bits/c++allocator.h:33, from /usr/include/c++/4.9/bits/allocator.h:46, from /usr/include/c++/4.9/string:41, from /usr/include/c++/4.9/bits/locale_classes.h:40, from /usr/include/c++/4.9/bits/ios_base.h:41, from /usr/include/c++/4.9/ios:42, from /usr/include/c++/4.9/ostream:38, from /usr/include/c++/4.9/iostream:39, from iTools.cpp:1: ./new:2:1: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:3: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:5: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:7: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:9: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:11: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:13: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:15: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:29: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:31: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:33: error: stray '#' in program ################ 2016-05-20 ##### ^ ./new:6:1: error: stray '##' in program ################2014-09-19############## ^ ./new:6:3: error: stray '##' in program ################2014-09-19############## ^ ./new:6:5: error: stray '##' in program ################2014-09-19############## ^ ./new:6:7: error: stray '##' in program ################2014-09-19############## ^ ./new:6:9: error: stray '##' in program ################2014-09-19############## ^ ./new:6:11: error: stray '##' in program ################2014-09-19############## ^ ./new:6:13: error: stray '##' in program ################2014-09-19############## ^ ./new:6:15: error: stray '##' in program ################2014-09-19############## ^ ./new:6:22: error: invalid digit "9" in octal constant ################2014-09-19############## ^ ./new:6:27: error: stray '##' in program ################2014-09-19############## ^ ./new:6:29: error: stray '##' in program ################2014-09-19############## ^ ./new:6:31: error: stray '##' in program ################2014-09-19############## ^ ./new:6:33: error: stray '##' in program ################2014-09-19############## ^ ./new:6:35: error: stray '##' in program ################2014-09-19############## ^ ./new:6:37: error: stray '##' in program ################2014-09-19############## ^ ./new:6:39: error: stray '##' in program ################2014-09-19############## ^ ./new:10:1: error: stray '##' in program ################2012-06-21############## ^ ./new:10:3: error: stray '##' in program ################2012-06-21############## ^ ./new:10:5: error: stray '##' in program ################2012-06-21############## ^ ./new:10:7: error: stray '##' in program ################2012-06-21############## ^ ./new:10:9: error: stray '##' in program ################2012-06-21############## ^ ./new:10:11: error: stray '##' in program ################2012-06-21############## ^ ./new:10:13: error: stray '##' in program ################2012-06-21############## ^ ./new:10:15: error: stray '##' in program ################2012-06-21############## ^ ./new:10:27: error: stray '##' in program ################2012-06-21############## ^ ./new:10:29: error: stray '##' in program ################2012-06-21############## ^ ./new:10:31: error: stray '##' in program ################2012-06-21############## ^ ./new:10:33: error: stray '##' in program ################2012-06-21############## ^ ./new:10:35: error: stray '##' in program ################2012-06-21############## ^ ./new:10:37: error: stray '##' in program ################2012-06-21############## ^ ./new:10:39: error: stray '##' in program ################2012-06-21############## ^ ./new:14:1: error: stray '##' in program ################2012-05-21############## ^ ./new:14:3: error: stray '##' in program ################2012-05-21############## ^ ./new:14:5: error: stray '##' in program ################2012-05-21############## ^ ./new:14:7: error: stray '##' in program ################2012-05-21############## ^ ./new:14:9: error: stray '##' in program ################2012-05-21############## ^ ./new:14:11: error: stray '##' in program ################2012-05-21############## ^ ./new:14:13: error: stray '##' in program ################2012-05-21############## ^ ./new:14:15: error: stray '##' in program ################2012-05-21############## ^ ./new:14:27: error: stray '##' in program ################2012-05-21############## ^ ./new:14:29: error: stray '##' in program ################2012-05-21############## ^ ./new:14:31: error: stray '##' in program ################2012-05-21############## ^ ./new:14:33: error: stray '##' in program ################2012-05-21############## ^ ./new:14:35: error: stray '##' in program ################2012-05-21############## ^ ./new:14:37: error: stray '##' in program ################2012-05-21############## ^ ./new:14:39: error: stray '##' in program ################2012-05-21############## ^ ./new:17:1: error: stray '##' in program ################2012-04-27############## ^ ./new:17:3: error: stray '##' in program ################2012-04-27############## ^ ./new:17:5: error: stray '##' in program ################2012-04-27############## ^ ./new:17:7: error: stray '##' in program ################2012-04-27############## ^ ./new:17:9: error: stray '##' in program ################2012-04-27############## ^ ./new:17:11: error: stray '##' in program ################2012-04-27############## ^ ./new:17:13: error: stray '##' in program ################2012-04-27############## ^ ./new:17:15: error: stray '##' in program ################2012-04-27############## ^ ./new:17:27: error: stray '##' in program ################2012-04-27############## ^ ./new:17:29: error: stray '##' in program ################2012-04-27############## ^ ./new:17:31: error: stray '##' in program ################2012-04-27############## ^ ./new:17:33: error: stray '##' in program ################2012-04-27############## ^ ./new:17:35: error: stray '##' in program ################2012-04-27############## ^ ./new:17:37: error: stray '##' in program ################2012-04-27############## ^ ./new:17:39: error: stray '##' in program ################2012-04-27############## ^ ./new:20:1: error: stray '##' in program #################2012-03-12############## ^ ./new:20:3: error: stray '##' in program #################2012-03-12############## ^ ./new:20:5: error: stray '##' in program #################2012-03-12############## ^ ./new:20:7: error: stray '##' in program #################2012-03-12############## ^ ./new:20:9: error: stray '##' in program #################2012-03-12############## ^ ./new:20:11: error: stray '##' in program #################2012-03-12############## ^ ./new:20:13: error: stray '##' in program #################2012-03-12############## ^ ./new:20:15: error: stray '##' in program #################2012-03-12############## ^ ./new:20:17: error: stray '#' in program #################2012-03-12############## ^ ./new:20:28: error: stray '##' in program #################2012-03-12############## ^ ./new:20:30: error: stray '##' in program #################2012-03-12############## ^ ./new:20:32: error: stray '##' in program #################2012-03-12############## ^ ./new:20:34: error: stray '##' in program #################2012-03-12############## ^ ./new:20:36: error: stray '##' in program #################2012-03-12############## ^ ./new:20:38: error: stray '##' in program #################2012-03-12############## ^ ./new:20:40: error: stray '##' in program #################2012-03-12############## ^ ./new:24:1: error: stray '##' in program #################2010-01-12############## ^ ./new:24:3: error: stray '##' in program #################2010-01-12############## ^ ./new:24:5: error: stray '##' in program #################2010-01-12############## ^ ./new:24:7: error: stray '##' in program #################2010-01-12############## ^ ./new:24:9: error: stray '##' in program #################2010-01-12############## ^ ./new:24:11: error: stray '##' in program #################2010-01-12############## ^ ./new:24:13: error: stray '##' in program #################2010-01-12############## ^ ./new:24:15: error: stray '##' in program #################2010-01-12############## ^ ./new:24:17: error: stray '#' in program #################2010-01-12############## ^ ./new:24:28: error: stray '##' in program #################2010-01-12############## ^ ./new:24:30: error: stray '##' in program #################2010-01-12############## ^ ./new:24:32: error: stray '##' in program #################2010-01-12############## ^ ./new:24:34: error: stray '##' in program #################2010-01-12############## ^ ./new:24:36: error: stray '##' in program #################2010-01-12############## ^ ./new:24:38: error: stray '##' in program #################2010-01-12############## ^ ./new:24:40: error: stray '##' in program #################2010-01-12############## ^ In file included from /usr/include/c++/4.9/bits/stl_construct.h:59:0, from /usr/include/c++/4.9/vector:62, from ./src/ALL/comm.h:8, from iTools.cpp:9: ./new:2:1: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:3: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:5: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:7: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:9: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:11: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:13: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:15: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:29: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:31: error: stray '##' in program ################ 2016-05-20 ##### ^ ./new:2:33: error: stray '#' in program ################ 2016-05-20 ##### ^ ./new:6:1: error: stray '##' in program ################2014-09-19############## ^ ./new:6:3: error: stray '##' in program ################2014-09-19############## ^ ./new:6:5: error: stray '##' in program ################2014-09-19############## ^ ./new:6:7: error: stray '##' in program ################2014-09-19############## ^ ./new:6:9: error: stray '##' in program ################2014-09-19############## ^ ./new:6:11: error: stray '##' in program ################2014-09-19############## ^ ./new:6:13: error: stray '##' in program ################2014-09-19############## ^ ./new:6:15: error: stray '##' in program ################2014-09-19############## ^ ./new:6:22: error: invalid digit "9" in octal constant ################2014-09-19############## ^ ./new:6:27: error: stray '##' in program ################2014-09-19############## ^ ./new:6:29: error: stray '##' in program ################2014-09-19############## ^ ./new:6:31: error: stray '##' in program ################2014-09-19############## ^ ./new:6:33: error: stray '##' in program ################2014-09-19############## ^ ./new:6:35: error: stray '##' in program ################2014-09-19############## ^ ./new:6:37: error: stray '##' in program ################2014-09-19############## ^ ./new:6:39: error: stray '##' in program ################2014-09-19############## ^ ./new:10:1: error: stray '##' in program ################2012-06-21############## ^ ./new:10:3: error: stray '##' in program ################2012-06-21############## ^ ./new:10:5: error: stray '##' in program ################2012-06-21############## ^ ./new:10:7: error: stray '##' in program ################2012-06-21############## ^ ./new:10:9: error: stray '##' in program ################2012-06-21############## ^ ./new:10:11: error: stray '##' in program ################2012-06-21############## ^ ./new:10:13: error: stray '##' in program ################2012-06-21############## ^ ./new:10:15: error: stray '##' in program ################2012-06-21############## ^ ./new:10:27: error: stray '##' in program ################2012-06-21############## ^ ./new:10:29: error: stray '##' in program ################2012-06-21############## ^ ./new:10:31: error: stray '##' in program ################2012-06-21############## ^ ./new:10:33: error: stray '##' in program ################2012-06-21############## ^ ./new:10:35: error: stray '##' in program ################2012-06-21############## ^ ./new:10:37: error: stray '##' in program ################2012-06-21############## ^ ./new:10:39: error: stray '##' in program ################2012-06-21############## ^ ./new:14:1: error: stray '##' in program ################2012-05-21############## ^ ./new:14:3: error: stray '##' in program ################2012-05-21############## ^ ./new:14:5: error: stray '##' in program ################2012-05-21############## ^ ./new:14:7: error: stray '##' in program ################2012-05-21############## ^ ./new:14:9: error: stray '##' in program ################2012-05-21############## ^ ./new:14:11: error: stray '##' in program ################2012-05-21############## ^ ./new:14:13: error: stray '##' in program ################2012-05-21############## ^ ./new:14:15: error: stray '##' in program ################2012-05-21############## ^ ./new:14:27: error: stray '##' in program ################2012-05-21############## ^ ./new:14:29: error: stray '##' in program ################2012-05-21############## ^ ./new:14:31: error: stray '##' in program ################2012-05-21############## ^ ./new:14:33: error: stray '##' in program ################2012-05-21############## ^ ./new:14:35: error: stray '##' in program ################2012-05-21############## ^ ./new:14:37: error: stray '##' in program ################2012-05-21############## ^ ./new:14:39: error: stray '##' in program ################2012-05-21############## ^ ./new:17:1: error: stray '##' in program ################2012-04-27############## ^ ./new:17:3: error: stray '##' in program ################2012-04-27############## ^ ./new:17:5: error: stray '##' in program ################2012-04-27############## ^ ./new:17:7: error: stray '##' in program ################2012-04-27############## ^ ./new:17:9: error: stray '##' in program ################2012-04-27############## ^ ./new:17:11: error: stray '##' in program ################2012-04-27############## ^ ./new:17:13: error: stray '##' in program ################2012-04-27############## ^ ./new:17:15: error: stray '##' in program ################2012-04-27############## ^ ./new:17:27: error: stray '##' in program ################2012-04-27############## ^ ./new:17:29: error: stray '##' in program ################2012-04-27############## ^ ./new:17:31: error: stray '##' in program ################2012-04-27############## ^ ./new:17:33: error: stray '##' in program ################2012-04-27############## ^ ./new:17:35: error: stray '##' in program ################2012-04-27############## ^ ./new:17:37: error: stray '##' in program ################2012-04-27############## ^ ./new:17:39: error: stray '##' in program ################2012-04-27############## ^ ./new:20:1: error: stray '##' in program #################2012-03-12############## ^ ./new:20:3: error: stray '##' in program #################2012-03-12############## ^ ./new:20:5: error: stray '##' in program #################2012-03-12############## ^ ./new:20:7: error: stray '##' in program #################2012-03-12############## ^ ./new:20:9: error: stray '##' in program #################2012-03-12############## ^ ./new:20:11: error: stray '##' in program #################2012-03-12############## ^ ./new:20:13: error: stray '##' in program #################2012-03-12############## ^ ./new:20:15: error: stray '##' in program #################2012-03-12############## ^ ./new:20:17: error: stray '#' in program #################2012-03-12############## ^ ./new:20:28: error: stray '##' in program #################2012-03-12############## ^ ./new:20:30: error: stray '##' in program #################2012-03-12############## ^ ./new:20:32: error: stray '##' in program #################2012-03-12############## ^ ./new:20:34: error: stray '##' in program #################2012-03-12############## ^ ./new:20:36: error: stray '##' in program #################2012-03-12############## ^ ./new:20:38: error: stray '##' in program #################2012-03-12############## ^ ./new:20:40: error: stray '##' in program #################2012-03-12############## ^ ./new:24:1: error: stray '##' in program #################2010-01-12############## ^ ./new:24:3: error: stray '##' in program #################2010-01-12############## ^ ./new:24:5: error: stray '##' in program #################2010-01-12############## ^ ./new:24:7: error: stray '##' in program #################2010-01-12############## ^ ./new:24:9: error: stray '##' in program #################2010-01-12############## ^ ./new:24:11: error: stray '##' in program #################2010-01-12############## ^ ./new:24:13: error: stray '##' in program #################2010-01-12############## ^ ./new:24:15: error: stray '##' in program #################2010-01-12############## ^ ./new:24:17: error: stray '#' in program #################2010-01-12############## ^ ./new:24:28: error: stray '##' in program #################2010-01-12############## ^ ./new:24:30: error: stray '##' in program #################2010-01-12############## ^ ./new:24:32: error: stray '##' in program #################2010-01-12############## ^ ./new:24:34: error: stray '##' in program #################2010-01-12############## ^ ./new:24:36: error: stray '##' in program #################2010-01-12############## ^ ./new:24:38: error: stray '##' in program #################2010-01-12############## ^ ./new:24:40: error: stray '##' in program #################2010-01-12############## ^ In file included from /usr/include/c++/4.9/ext/hash_map:60:0, from ./src/Soap/Soap_Split.h:15, from ./src/Soap/SOAPTools.h:19, from iTools.cpp:10: /usr/include/c++/4.9/backward/backward_warning.h:32:2: warning: #warning This file includes at least one deprecated or antiquated header which may be removed without further notice at a future date. Please use a non-deprecated interface with equivalent functionality instead. For a listing of replacement headers and interfaces, consult the file backward_warning.h. To disable this warning use -Wno-deprecated. [-Wcpp] #warning \ ^ In file included from ./src/CNSTool/CNSTools.h:11:0, from iTools.cpp:12: ./src/CNSTool/Addcn_All_V2.2.h:12:35: fatal error: boost/thread/thread.hpp: No such file or directory #include <boost/thread/thread.hpp> ^ compilation terminated. make[1]: *** [iTools.o] Error 1 make: *** [all] Error 2
%%bash
./iTools
Program: iTools (ReSeqtools) Version: 0.23 hewm2008@gmail.com May 30 2016 Usage: Fatools Tools For Fasta Fqtools Tools For Fastq SOAPtools Tools For SOAP Vartools Tools For SOAP Variant CNStools Tools For CNS Xamtools Tools For Sam/Bam Gfftools Tools For Gff Formtools Tools For Form convert Filetools Tools For Specified File Othertools Tools For Other Gametools Tools For Game Help Show help in detail
%%bash
./iTools Fqtools
FqTools Usage: stat quick stat fastq's info fqcheck fqchek Base Q Distribute filterV1 filter fastq for clean datas with trim filterV2 filter fastq for clean datas select trim rmAdapterPE index remove adapter of PE rmAdapterSE index remove adapter of SE splitpool Split pooling Fq to sample for RAD (GBS) cutIndex cut the Read Length in the Fq pooling pooling index library data filter bubble filter the N bubble site Read changQ chang Fq seq Quality (+/- 31) Help Show this help
%%bash
./iTools Fqtools splitpool
Usage: splitpool -InFq1 <In1.fq> -InFq2 <In2.fq> -Index <sample.info> -Flag <ferment.seq> -InFq1 <str> Input #_1.Fq to split RAD(GBS) -InFq2 <str> Input #_2.Fq to split RAD(GBS) -Index <str> Input File with (sample seq) -Flag <str> Input File with Flag(ferment) seq -OutDir <str> Output Dir for Split Files[PWD] -MisMatch Allow one misMatch on the sample seq -NoCheckS No Check Sample double,Allow one sample with multi seq but Read1 may be different length -help show this help
make
command failed? Well, let's try running the script...¶%%bash
mkdir /data/oly_gbs_raw
time cp /owl_web/nightingales/O_lurida/20160223_gbs/160123*.gz /data/oly_gbs_raw/
real 27m30.177s user 0m0.020s sys 5m34.770s
ls /data/oly_gbs_raw
160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz
%%bash
cp /owl_web/nightingales/O_lurida/20160223_gbs/*.[sl]*
ls /data/oly_gbs_raw
160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz
%%bash
head index.lst
head: cannot open 'index.lst' for reading: No such file or directory
%%bash
cp /owl_web/nightingales/O_lurida/20160223_gbs/*.[sl]* /data/oly_gbs_raw/
ls
Example/ ReSeqTools@ config.h.in depcomp@ install-sh@ Install.Readme ReSeqTools.Readme@ config.log document/ missing@ Makefile aclocal.m4 config.status* iTools* src/ Makefile.am autom4te.cache/ configure* iTools.Readme@ stamp-h1 Makefile.in bin/ configure.ac iTools.cpp NEW config.h configure.scan iTools.o
ls /data/oly_gbs_raw
160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz index.lst 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz split.sh
%%bash
head /data/oly_gbs_raw/index.lst
OYSzenG1AAD96FAAPEI-109 CTCC 1NF_1A OYSzenG1AAD96FAAPEI-109 TGCA 1NF_2A OYSzenG1AAD96FAAPEI-109 ACTA 1NF_4A OYSzenG1AAD96FAAPEI-109 CAGA 1NF_5A OYSzenG1AAD96FAAPEI-109 AACT 1NF_6A OYSzenG1AAD96FAAPEI-109 GCGT 1NF_7A OYSzenG1AAD96FAAPEI-109 CGAT 1NF_8A OYSzenG1AAD96FAAPEI-109 GTAA 1NF_9A OYSzenG1AAD96FAAPEI-109 AGGC 1NF_10A OYSzenG1AAD96FAAPEI-109 GATC 1NF_11A
%%bash
head /data/oly_gbs_raw/split.sh
OYSzenG1AAD96FAAPEI-109 CTCC 1NF_1A OYSzenG1AAD96FAAPEI-109 TGCA 1NF_2A OYSzenG1AAD96FAAPEI-109 ACTA 1NF_4A OYSzenG1AAD96FAAPEI-109 CAGA 1NF_5A OYSzenG1AAD96FAAPEI-109 AACT 1NF_6A OYSzenG1AAD96FAAPEI-109 GCGT 1NF_7A OYSzenG1AAD96FAAPEI-109 CGAT 1NF_8A OYSzenG1AAD96FAAPEI-109 GTAA 1NF_9A OYSzenG1AAD96FAAPEI-109 AGGC 1NF_10A OYSzenG1AAD96FAAPEI-109 GATC 1NF_11A
%%bash
head /data/oly_gbs_raw/split.sh
OYSzenG1AAD96FAAPEI-109 CTCC 1NF_1A OYSzenG1AAD96FAAPEI-109 TGCA 1NF_2A OYSzenG1AAD96FAAPEI-109 ACTA 1NF_4A OYSzenG1AAD96FAAPEI-109 CAGA 1NF_5A OYSzenG1AAD96FAAPEI-109 AACT 1NF_6A OYSzenG1AAD96FAAPEI-109 GCGT 1NF_7A OYSzenG1AAD96FAAPEI-109 CGAT 1NF_8A OYSzenG1AAD96FAAPEI-109 GTAA 1NF_9A OYSzenG1AAD96FAAPEI-109 AGGC 1NF_10A OYSzenG1AAD96FAAPEI-109 GATC 1NF_11A
split.sh
script when I screwed up the first copy command in line 30 above! Whoops! Good thing for backups!!¶%%bash
cp /gitrepos/paper_oly_gbs/data/split.sh /owl_web/nightingales/O_lurida/20160223_gbs/
%%bash
cp /gitrepos/paper_oly_gbs/data/split.sh /data/oly_gbs_raw/
%%bash
head /owl_web/nightingales/O_lurida/20160223_gbs/split.sh
iTools Fqtools splitpool -InFq1 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz -InFq2 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz -Index index.lst -Flag enzyme.txt -MisMatch -OutDir split
%%bash
head /data/oly_gbs_raw/split.sh
iTools Fqtools splitpool -InFq1 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz -InFq2 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz -Index index.lst -Flag enzyme.txt -MisMatch -OutDir split
enzyme.txt
file mentioned in the split.sh
script, sicne it wasn't supplied by BGI.¶%%bash
printf %"s\n" CAGC CTGC > /data/oly_gbs_raw/enzyme.txt
%%bash
cat /data/oly_gbs_raw/enzyme.txt
CAGC CTGC
%%bash
mkdir /data/oly_gbs_raw/split
%%bash
time ./iTools Fqtools splitpool \
-InFq1 /data/oly_gbs_raw/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz \
-InFq2 /data/oly_gbs_raw/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz \
-Index /data/oly_gbs_raw/index.lst \
-Flag /data/oly_gbs_raw/enzyme.txt \
-MisMatch \
-OutDir /data/oly_gbs_raw/split/
Warming : sample double in this INDEX Files. Sample ID: OYSzenG1AAD96FAAPEI-109; please renamed it diff real 0m0.181s user 0m0.000s sys 0m0.010s
%%bash
date
Wed Mar 15 17:19:02 UTC 2017
%%bash
head /data/oly_gbs_raw/split.sh
iTools Fqtools splitpool -InFq1 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz -InFq2 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz -Index index.lst -Flag enzyme.txt -MisMatch -OutDir split
%%bash
head /data/oly_gbs_raw/index.lst
OYSzenG1AAD96FAAPEI-109 CTCC 1NF_1A OYSzenG1AAD96FAAPEI-109 TGCA 1NF_2A OYSzenG1AAD96FAAPEI-109 ACTA 1NF_4A OYSzenG1AAD96FAAPEI-109 CAGA 1NF_5A OYSzenG1AAD96FAAPEI-109 AACT 1NF_6A OYSzenG1AAD96FAAPEI-109 GCGT 1NF_7A OYSzenG1AAD96FAAPEI-109 CGAT 1NF_8A OYSzenG1AAD96FAAPEI-109 GTAA 1NF_9A OYSzenG1AAD96FAAPEI-109 AGGC 1NF_10A OYSzenG1AAD96FAAPEI-109 GATC 1NF_11A
%%bash
awk {'printf ("%s\t%s\n", $2, $3)'} /data/oly_gbs_raw/index.lst > /data/oly_gbs_raw/index.tmp
%%bash
head /data/oly_gbs_raw/index.tmp
CTCC 1NF_1A TGCA 1NF_2A ACTA 1NF_4A CAGA 1NF_5A AACT 1NF_6A GCGT 1NF_7A CGAT 1NF_8A GTAA 1NF_9A AGGC 1NF_10A GATC 1NF_11A
Looks good! Let's try the script again...
%%bash
time ./iTools Fqtools splitpool \
-InFq1 /data/oly_gbs_raw/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz \
-InFq2 /data/oly_gbs_raw/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz \
-Index /data/oly_gbs_raw/index.tmp \
-Flag /data/oly_gbs_raw/enzyme.txt \
-MisMatch \
-OutDir /data/oly_gbs_raw/split/
bash: line 1: ./iTools: No such file or directory real 0m0.002s user 0m0.000s sys 0m0.000s
%%bash
cd /gitrepos/Reseqtools/iTools_Code/
cd /gitrepos/Reseqtools/iTools_Code/
/gitrepos/Reseqtools/iTools_Code
%%bash
time ./iTools Fqtools splitpool \
-InFq1 /data/oly_gbs_raw/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz \
-InFq2 /data/oly_gbs_raw/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz \
-Index /data/oly_gbs_raw/index.tmp \
-Flag /data/oly_gbs_raw/enzyme.txt \
-MisMatch \
-OutDir /data/oly_gbs_raw/split/
Process is interrupted.
I stopped the run because it was naming the output files incorrectly.
ls /data/oly_gbs_raw/split/
AAAAGTT_1.fq.gz CAGA_2.fq.gz GAATTCA_1.fq.gz TAATA_2.fq.gz AAAAGTT_2.fq.gz CATAAGT_1.fq.gz GAATTCA_2.fq.gz TACAT_1.fq.gz AACCGAGA_1.fq.gz CATAAGT_2.fq.gz GAGATA_1.fq.gz TACAT_2.fq.gz AACCGAGA_2.fq.gz CATCGT_1.fq.gz GAGATA_2.fq.gz TAGCATGC_1.fq.gz AACGCCT_1.fq.gz CATCGT_2.fq.gz GAGGA_1.fq.gz TAGCATGC_2.fq.gz AACGCCT_2.fq.gz CATCT_1.fq.gz GAGGA_2.fq.gz TAGCGGA_1.fq.gz AACT_1.fq.gz CATCT_2.fq.gz GATC_1.fq.gz TAGCGGA_2.fq.gz AACT_2.fq.gz CCACAA_1.fq.gz GATC_2.fq.gz TAGGAA_1.fq.gz AATATGC_1.fq.gz CCACAA_2.fq.gz GCCAGT_1.fq.gz TAGGAA_2.fq.gz AATATGC_2.fq.gz CCAGCT_1.fq.gz GCCAGT_2.fq.gz TAGGCCAT_1.fq.gz ACAAA_1.fq.gz CCAGCT_2.fq.gz GCGGAAT_1.fq.gz TAGGCCAT_2.fq.gz ACAAA_2.fq.gz CCATGGGT_1.fq.gz GCGGAAT_2.fq.gz TATCGGGA_1.fq.gz ACAGGGAA_1.fq.gz CCATGGGT_2.fq.gz GCGT_1.fq.gz TATCGGGA_2.fq.gz ACAGGGAA_2.fq.gz CCGGATAT_1.fq.gz GCGT_2.fq.gz TATTTTT_1.fq.gz ACCGT_1.fq.gz CCGGATAT_2.fq.gz GCTCTA_1.fq.gz TATTTTT_2.fq.gz ACCGT_2.fq.gz CCTAC_1.fq.gz GCTCTA_2.fq.gz TCACC_1.fq.gz ACCTAA_1.fq.gz CCTAC_2.fq.gz GCTGTGGA_1.fq.gz TCACC_2.fq.gz ACCTAA_2.fq.gz CGAT_1.fq.gz GCTGTGGA_2.fq.gz TCAC_1.fq.gz ACGACTAC_1.fq.gz CGAT_2.fq.gz GCTTA_1.fq.gz TCAC_2.fq.gz ACGACTAC_2.fq.gz CGCCTTAT_1.fq.gz GCTTA_2.fq.gz TCGAAGA_1.fq.gz ACGTGGTA_1.fq.gz CGCCTTAT_2.fq.gz GGAAC_1.fq.gz TCGAAGA_2.fq.gz ACGTGGTA_2.fq.gz CGCGGAGA_1.fq.gz GGAAC_2.fq.gz TCGTT_1.fq.gz ACGTGTT_1.fq.gz CGCGGAGA_2.fq.gz GGAAGA_1.fq.gz TCGTT_2.fq.gz ACGTGTT_2.fq.gz CGCGGT_1.fq.gz GGAAGA_2.fq.gz TCTCAGTC_1.fq.gz ACTA_1.fq.gz CGCGGT_2.fq.gz GGACCTA_1.fq.gz TCTCAGTC_2.fq.gz ACTA_2.fq.gz CGCTGAT_1.fq.gz GGACCTA_2.fq.gz TCTGTGA_1.fq.gz AGCCC_1.fq.gz CGCTGAT_2.fq.gz GGATTGGT_1.fq.gz TCTGTGA_2.fq.gz AGCCC_2.fq.gz CGCTT_1.fq.gz GGATTGGT_2.fq.gz TGCAAGGA_1.fq.gz AGGAT_1.fq.gz CGCTT_2.fq.gz GGTGT_1.fq.gz TGCAAGGA_2.fq.gz AGGAT_2.fq.gz CGGTAGA_1.fq.gz GGTGT_2.fq.gz TGCA_1.fq.gz AGGC_1.fq.gz CGGTAGA_2.fq.gz GGTTGT_1.fq.gz TGCA_2.fq.gz AGGC_2.fq.gz CGTGTGGT_1.fq.gz GGTTGT_2.fq.gz TGCGA_1.fq.gz AGTGGA_1.fq.gz CGTGTGGT_2.fq.gz GTAA_1.fq.gz TGCGA_2.fq.gz AGTGGA_2.fq.gz CTACGGA_1.fq.gz GTAA_2.fq.gz TGCTGGA_1.fq.gz ATATGT_1.fq.gz CTACGGA_2.fq.gz GTACTT_1.fq.gz TGCTGGA_2.fq.gz ATATGT_2.fq.gz CTAGC_1.fq.gz GTACTT_2.fq.gz TGGCTA_1.fq.gz ATCGTA_1.fq.gz CTAGC_2.fq.gz GTATT_1.fq.gz TGGCTA_2.fq.gz ATCGTA_2.fq.gz CTATTA_1.fq.gz GTATT_2.fq.gz TGGTACGT_1.fq.gz ATGAAAC_1.fq.gz CTATTA_2.fq.gz GTCAA_1.fq.gz TGGTACGT_2.fq.gz ATGAAAC_2.fq.gz CTCC_1.fq.gz GTCAA_2.fq.gz TTCAGA_1.fq.gz ATGCCT_1.fq.gz CTCC_2.fq.gz GTCGATT_1.fq.gz TTCAGA_2.fq.gz ATGCCT_2.fq.gz CTGTA_1.fq.gz GTCGATT_2.fq.gz TTCCTGGA_1.fq.gz ATTAATT_1.fq.gz CTGTA_2.fq.gz GTGAGGGT_1.fq.gz TTCCTGGA_2.fq.gz ATTAATT_2.fq.gz CTTCCA_1.fq.gz GTGAGGGT_2.fq.gz TTCTC_1.fq.gz ATTGA_1.fq.gz CTTCCA_2.fq.gz GTTGAA_1.fq.gz TTCTC_2.fq.gz ATTGA_2.fq.gz CTTGCTT_1.fq.gz GTTGAA_2.fq.gz UnKnow_1.fq.gz ATTGGAT_1.fq.gz CTTGCTT_2.fq.gz TAACGA_1.fq.gz UnKnow_2.fq.gz ATTGGAT_2.fq.gz GAACTTC_1.fq.gz TAACGA_2.fq.gz CAGA_1.fq.gz GAACTTC_2.fq.gz TAATA_1.fq.gz
%%bash
awk {'printf ("%s\t%s\n", $3, $2)'} /data/oly_gbs_raw/index.lst > /data/oly_gbs_raw/index.tmp
%%bash
head /data/oly_gbs_raw/index.tmp
1NF_1A CTCC 1NF_2A TGCA 1NF_4A ACTA 1NF_5A CAGA 1NF_6A AACT 1NF_7A GCGT 1NF_8A CGAT 1NF_9A GTAA 1NF_10A AGGC 1NF_11A GATC
for i in /data/oly_gbs_raw/split/*.gz
do
rm "$i"
done
File "<ipython-input-13-07828763ed1d>", line 1 for i in /data/oly_gbs_raw/split/*.gz ^ SyntaxError: invalid syntax
%%bash
for i in /data/oly_gbs_raw/split/*.gz
do
rm "$i"
done
ls /data/oly_gbs_raw/split/
%%bash
time ./iTools Fqtools splitpool \
-InFq1 /data/oly_gbs_raw/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1.fq.gz \
-InFq2 /data/oly_gbs_raw/160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2.fq.gz \
-Index /data/oly_gbs_raw/index.tmp \
-Flag /data/oly_gbs_raw/enzyme.txt \
-MisMatch \
-OutDir /data/oly_gbs_raw/split/
real 980m9.467s user 797m30.100s sys 156m26.850s
ls -lhr /data/oly_gbs_raw/split/ | head
total 36G -rw-r--r-- 1 srlab staff 507M Mar 16 10:05 UnKnow_2.fq.gz -rw-r--r-- 1 srlab staff 445M Mar 16 10:05 UnKnow_1.fq.gz -rw-r--r-- 1 srlab staff 229M Mar 16 10:05 1SN_9A_2.fq.gz -rw-r--r-- 1 srlab staff 188M Mar 16 10:05 1SN_9A_1.fq.gz -rw-r--r-- 1 srlab staff 240M Mar 16 10:05 1SN_8A_2.fq.gz -rw-r--r-- 1 srlab staff 197M Mar 16 10:05 1SN_8A_1.fq.gz -rw-r--r-- 1 srlab staff 216M Mar 16 10:05 1SN_7A_2.fq.gz -rw-r--r-- 1 srlab staff 175M Mar 16 10:05 1SN_7A_1.fq.gz -rw-r--r-- 1 srlab staff 242M Mar 16 10:05 1SN_6A_2.fq.gz ls: write error
Alright! Looks like those commands work! Will correct the index.lst
file and will add the enzyme.txt
file to the GBS paper repo.
ls /gitrepos/paper_oly_gbs/data/
160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc/ SNP.stat.xls 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_1_fastqc.html index.lst 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc/ readme.md 160123_I132_FCH3YHMBBXX_L4_OYSzenG1AAD96FAAPEI-109_2_fastqc.html split.sh Data.stat.xls
%%bash
head /gitrepos/paper_oly_gbs/data/index.lst
OYSzenG1AAD96FAAPEI-109 CTCC 1NF_1A OYSzenG1AAD96FAAPEI-109 TGCA 1NF_2A OYSzenG1AAD96FAAPEI-109 ACTA 1NF_4A OYSzenG1AAD96FAAPEI-109 CAGA 1NF_5A OYSzenG1AAD96FAAPEI-109 AACT 1NF_6A OYSzenG1AAD96FAAPEI-109 GCGT 1NF_7A OYSzenG1AAD96FAAPEI-109 CGAT 1NF_8A OYSzenG1AAD96FAAPEI-109 GTAA 1NF_9A OYSzenG1AAD96FAAPEI-109 AGGC 1NF_10A OYSzenG1AAD96FAAPEI-109 GATC 1NF_11A
%%bash
mv /data/oly_gbs_raw/index.tmp /gitrepos/paper_oly_gbs/data/index.lst
%%bash
head /gitrepos/paper_oly_gbs/data/index.lst
1NF_1A CTCC 1NF_2A TGCA 1NF_4A ACTA 1NF_5A CAGA 1NF_6A AACT 1NF_7A GCGT 1NF_8A CGAT 1NF_9A GTAA 1NF_10A AGGC 1NF_11A GATC
%%bash
git add /gitrepos/paper_oly_gbs/data/index.lst
fatal: /gitrepos/paper_oly_gbs/data/index.lst: '/gitrepos/paper_oly_gbs/data/index.lst' is outside repository
Guess I can't add/commit to the repo from this notebook. Will do so outside.
%%bash
mv /data/oly_gbs_raw/enzyme.txt /gitrepos/paper_oly_gbs/data/enzyme.txt
We'll just look at line counts and see how they compare.
%%bash
wc -l /owl_web/nightingales/O_lurida/20160223_gbs/1SN_9A_2.fq.gz
741674 /owl_web/nightingales/O_lurida/20160223_gbs/1SN_9A_2.fq.gz
%%bash
wc -l /data/oly_gbs_raw/1SN_9A_2.fq.gz
wc: /data/oly_gbs_raw/1SN_9A_2.fq.gz: No such file or directory
%%bash
wc -l /data/oly_gbs_raw/split/1SN_9A_2.fq.gz
1052299 /data/oly_gbs_raw/split/1SN_9A_2.fq.gz
Well, this is curious... A couple of issues that I notice off the bat:
Obviously, the line numbers differ.
The demultiplexed file I created with the BGI script is not evenly divisible by the number 4. This is important because each read in a FASTQ file is supposed to have four lines of information.
Let's look at another file.
%%bash
wc -l /data/oly_gbs_raw/1SN_9A_1.fq.gz
wc -l /data/oly_gbs_raw/split/1SN_9A_1.fq.gz
802505 /data/oly_gbs_raw/split/1SN_9A_1.fq.gz
wc: /data/oly_gbs_raw/1SN_9A_1.fq.gz: No such file or directory
%%bash
wc -l /owl_web/nightingales/O_lurida/20160223_gbs/1SN_9A_1.fq.gz
wc -l /data/oly_gbs_raw/split/1SN_9A_1.fq.gz
602771 /owl_web/nightingales/O_lurida/20160223_gbs/1SN_9A_1.fq.gz 802505 /data/oly_gbs_raw/split/1SN_9A_1.fq.gz
Well, this is beginning to seem problematic. Let's glance at a bit of these FASTQ files and see if we can determine what's going on (doubtful, though).
Let's try this again...
%%bash
time gzip -c /owl_web/nightingales/O_lurida/20160223_gbs/1SN_9A_1.fq.gz | wc -l
602994
real 0m10.640s user 0m5.110s sys 0m3.300s
%%bash
time gzip -c /data/oly_gbs_raw/split/1SN_9A_1.fq.gz | wc -l
802289
real 0m12.173s user 0m7.970s sys 0m2.620s
%%bash
gzip -c /data/oly_gbs_raw/split/1SN_9A_1.fq.gz | head -12
�Lc�X 1SN_9A_1.fq.gz 3��� Խˮ�<�,8�����0�wg<8<��l�7������1AYvZV�Ԫv}{��Z�$�C���9��ӿ�����������?������_�����K�M����u��?�:�K$FI1�W��I��)I�~�i�G����Կc �� f ��]\��٘i!T,�N����?+�L�t��ӄ��O���9��i�v�tZ�":��G] :黾�p�z�wGϼ� ��eܟ���~��p�i��pe����r�|��(���Gu�@½�1-�iA[t�M+�-Ә���z?4��RIF���K�7�����e7��Űo��v�G��!�Q�)�\�V��f?b68=`h��b�-q��!A��~��R�+�}����imu����7*]DEm�J���xv���MW0�G�p��{�ɟ��[9Pg�jQ�ֶ�;9��i2z��s�\��շ�^ ��~\R&`֒| s��z�n�X��l���w��Zv��X��k=J�1�Ď��S�4+�En�{V����sls�˲��\� ����۹��`rr��� '���R%�@jQ`]����x���!�8_� ��Ɇ�%$��S�$�S4��=�T�s�!>:E���NQ�^q�)M�h��ӌl��d�<��Fy���:`�۽��3��Pr�n� �b�;C�{�Cc������ )%ζ x4��!�W"������GT�؉Ȃ�#(��"k��~��b��j���.�ЊLY����읮+8f�eD�L��s��w.�vG Q@A�&�"�yGB���z��n猃�a��u_4�p��@����K�@�%p3�a���A���ei�x���m��Q��K5��6�Q�f�w����#\e� �^���ъk����K�1R���P���Sw��^�+?7�G+������z\j�&oV��~���t9��$��ŋ�u��ulxj���p�[��D�OZ�*��}�\w<-�}� D�;SY�@�����=� ���5������Z�B�Fd���-�b������� خI����9���^�Z���]���Xo����ѳG�E��d� �i,�Y7^d j�o&�w���\�㣲s��j�Ya�+� so���!�z����a\Be���2g�Z1S�c�����b8#��9(������u�ѥ;�K|�k���B��*�k� ��mјֶ_y������;����M�0HT#��:� ƺ/��-���PO=�m0��koV�� A�3JP�'��k��)�9"݊�sh>����'ܞ�� R��z�E" ��X�4RC����Q�el��H@W@��O�C,�b~ %�X�GW]�娕���lŇv엟�+1�}=��ָn� &�) T�����b�i0��$�W�V +�5g6��7�Y�L���F��`ܣm�Ae���9���F¤�х�0��U��dq���/�ê�s�Wq5�>��cM�%�)$*�����{�sO�72�a ��ǡ>�)z� �����%y{�r"���'�,�y#�3�oFk���U�j>� !x� ������DމHr8[�K#�%ȹ�{S�oX��7��),�V �[Mʍ��"�tv���,Yԁm�g�j2䫬ڸ�چw̫���
gzip: stdout: Broken pipe
%%bash
gzip -cd /data/oly_gbs_raw/split/1SN_9A_1.fq.gz | head -12
@K00132:90:H3YHMBBXX:4:1101:4706:1173 1:N:0:AAGGATGC CAGCATGTTGTTCTGCTTACCTTGGATTGATTGATTGATTGATTAGTTACTGTTTTACGTCCCACTCGAGAATATTTCACTCATATGGAGACGT + JJJJJJJJJJJJJJJJJJJJJFFJFFJJJJJJFJJJJJJJJJFFJJJJJJJJJJJJJJJJJJJJJJJJJJFJJJJJJJJJJJFFJJJJJJJJJA @K00132:90:H3YHMBBXX:4:1101:5741:1173 1:N:0:AAGGATGC CAGCACGTATCAATTTAATTCTCATTAATATCTGATTTAATTCTACTTAATATCTGTCGATTATTTTGTGTGAAGAAAATCTTTATGCGATGTA + JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJFJJJJJJJ @K00132:90:H3YHMBBXX:4:1101:6005:1173 1:N:0:AAGGATGC CAGCTGCGGCGTGACTCGATGGGGGCCGTTCGCGGTCCACGCTTGTCGTGCTGAGATCGGAAGAGCACACGTCTGAACTCCAGTCACAAGGATG + JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
gzip: stdout: Broken pipe
%%bash
gzip -cd /owl_web/nightingales/O_lurida/20160223_gbs/1SN_9A_1.fq.gz | wc -l
gzip -cd /data/oly_gbs_raw/split/1SN_9A_1.fq.gz | wc -l
12708820 16594884
OK! Now we've gotten somewhere! I've fixed the counting and head display issue. Need the -d
(decompress) argument for the gzip command (still need the -c
, though!). Counts for both files are evenly divisible by 4.
However, we see that the line numbers differ from each other for the same (theoretically) demultiplexed reads.
Maybe I'll re-run the BGI script and compare those files with the ones I created in this notebook entry?