In [1]:

!date

Mon Apr 11 07:47:32 PDT 2016

In [2]:

%%bash
system_profiler SPSoftwareDataType

Software:

    System Software Overview:

      System Version: OS X 10.9.5 (13F34)
      Kernel Version: Darwin 13.4.0
      Boot Volume: Hummingbird
      Boot Mode: Normal
      Computer Name: hummingbird
      User Name: Sam (Sam)
      Secure Virtual Memory: Enabled
      Time since boot: 142 days 16:38

In [128]:

cd /Volumes/nightingales/O_lurida/20160203_mbdseq/

/Volumes/nightingales/O_lurida/20160203_mbdseq

Concatenate entire set of FASTQ files for each individual¶

The commands below use a for loop to:¶

process each FASTQ corresponding to an individual (zr1394_n_*)
concatenate each file (cat $file) to the designated output file (${file/_s*_R1/})

The output file is named using bash parameter expansion. It takes the file name ($file) and replaces the designated matching text (_s*_R1) with whatever follows the last slash - in this case, it's replaced with an empty string (i.e. deleted).

In [129]:

%%bash

for file in zr1394_1_*; do
  cat $file >> ${file/_s*_R1/}
done

In [130]:

%%bash

for file in zr1394_2_*; do
  cat $file >> ${file/_s*_R1/}
done

In [131]:

%%bash

for file in zr1394_3_*; do
  cat $file >> ${file/_s*_R1/}
done

In [132]:

%%bash

for file in zr1394_4_*; do
  cat $file >> ${file/_s*_R1/}
done

In [133]:

%%bash

for file in zr1394_5_*; do
  cat $file >> ${file/_s*_R1/}
done

In [134]:

%%bash

for file in zr1394_6_*; do
  cat $file >> ${file/_s*_R1/}
done

In [135]:

%%bash

for file in zr1394_7_*; do
  cat $file >> ${file/_s*_R1/}
done

In [136]:

%%bash

for file in zr1394_8_*; do
  cat $file >> ${file/_s*_R1/}
done

In [137]:

%%bash

for file in zr1394_9_*; do
  cat $file >> ${file/_s*_R1/}
done

In [138]:

%%bash

for file in zr1394_10_*; do
  cat $file >> ${file/_s*_R1/}
done

In [139]:

%%bash

for file in zr1394_11_*; do
  cat $file >> ${file/_s*_R1/}
done

In [140]:

%%bash

for file in zr1394_12_*; do
  cat $file >> ${file/_s*_R1/}
done

In [141]:

%%bash

for file in zr1394_13_*; do
  cat $file >> ${file/_s*_R1/}
done

In [142]:

%%bash

for file in zr1394_14_*; do
  cat $file >> ${file/_s*_R1/}
done

In [143]:

%%bash

for file in zr1394_15_*; do
  cat $file >> ${file/_s*_R1/}
done

In [144]:

%%bash

for file in zr1394_16_*; do
  cat $file >> ${file/_s*_R1/}
done

In [145]:

%%bash

for file in zr1394_17_*; do
  cat $file >> ${file/_s*_R1/}
done

In [146]:

%%bash

for file in zr1394_18_*; do
  cat $file >> ${file/_s*_R1/}
done

In [147]:

%%bash

ls -lh zr1394_{1..18}.fastq.gz

-rw-rw-rw-  1 Sam  staff   2.4G Apr 11 14:33 zr1394_1.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.7G Apr 11 14:50 zr1394_10.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.8G Apr 11 14:52 zr1394_11.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.8G Apr 11 14:55 zr1394_12.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.0G Apr 11 14:59 zr1394_13.fastq.gz
-rw-rw-rw-  1 Sam  staff   4.6G Apr 11 15:03 zr1394_14.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.2G Apr 11 15:06 zr1394_15.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.5G Apr 11 15:09 zr1394_16.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.2G Apr 11 15:11 zr1394_17.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.5G Apr 11  2016 zr1394_18.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.4G Apr 11 14:35 zr1394_2.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.5G Apr 11 14:37 zr1394_3.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.1G Apr 11 14:38 zr1394_4.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.0G Apr 11 14:40 zr1394_5.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.3G Apr 11 14:42 zr1394_6.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.6G Apr 11 14:43 zr1394_7.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.0G Apr 11 14:45 zr1394_8.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.9G Apr 11 14:47 zr1394_9.fastq.gz

Concatenate sets of s4, s5, & s6 of FASTQ files for each individual¶

The commands below use a for loop to:¶

process the s4, s5, & s6 FASTQ corresponding to an individual (zr1394_1_s{4..6}*)
concatenate each file (cat $file) to the designated output file (${file/_s*_R1/_s456})

The output file is named using bash parameter expansion. It takes the file name ($file) and replaces the designated matching text (_s*_R1) with whatever follows the last slash - in this case, it's replaced with "_s456".

In [148]:

%%bash

for file in zr1394_1_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [149]:

%%bash

for file in zr1394_2_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [150]:

%%bash

for file in zr1394_3_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [151]:

%%bash

for file in zr1394_4_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [152]:

%%bash

for file in zr1394_5_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [153]:

%%bash

for file in zr1394_6_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [154]:

%%bash

for file in zr1394_7_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [155]:

%%bash

for file in zr1394_8_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [156]:

%%bash

for file in zr1394_9_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [157]:

%%bash

for file in zr1394_10_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [158]:

%%bash

for file in zr1394_11_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [159]:

%%bash

for file in zr1394_12_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [160]:

%%bash

for file in zr1394_13_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [161]:

%%bash

for file in zr1394_14_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [162]:

%%bash

for file in zr1394_15_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [163]:

%%bash

for file in zr1394_16_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [164]:

%%bash

for file in zr1394_17_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [165]:

%%bash

for file in zr1394_18_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done

In [166]:

ls -lh zr1394_*_s456.fastq.gz | sort

-rw-rw-rw-  1 Sam  staff   1.2G Apr 11 15:22 zr1394_8_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.4G Apr 11  2016 zr1394_17_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.4G Apr 11 15:18 zr1394_4_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:14 zr1394_1_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:16 zr1394_2_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:17 zr1394_3_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:19 zr1394_5_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:20 zr1394_6_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.6G Apr 11 15:22 zr1394_7_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.7G Apr 11  2016 zr1394_18_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.7G Apr 11 15:27 zr1394_11_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.9G Apr 11 15:29 zr1394_12_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.0G Apr 11 15:30 zr1394_13_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.2G Apr 11 15:35 zr1394_15_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.4G Apr 11 15:24 zr1394_9_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.4G Apr 11 15:37 zr1394_16_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.5G Apr 11 15:26 zr1394_10_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.8G Apr 11 15:32 zr1394_14_s456.fastq.gz