CIAO Workshop Exercises¶

This Jupyter notebook uses the bash_kernel. Users will need to have this kernel installed before they can run the commadns in this notebook.

CIAO 4.11 users with pip3 can use the following commands to install bash_kernel into their CIAO installtion:

pip3 install bash_kernel
python -m bash_kernel.install --sys-prefix
jupyter kernelspec list

after which they can then run

jupyter notebook

Introduction¶

These exercises are designed to familiarize students with Chandra data and some of the basics of X-ray data analysis using CIAO with Chandra data.

Students will use a variety of tools and scripts to perform typical Chandra data analysis tasks. These exercises are not intended to be a Cookbook for all Chandra data analysis, but rather are meant to expose students to basic X-ray analysis techniques.

Students should already have CIAO installed on their systems. The exercises here have been developed with CIAO 4.11, using the CIAO-4.11.1 contributed scripts package, with Chandra CALDB 4.8.2. Students should also have MARX installed. These exercises have been developed using MARX 5.4.0. The install_marx script in CIAO can be used to automate the installation process.

In [1]:

ciaover

CIAO 4.11 Wednesday, December  5, 2018
  bindir      : /export/ciao-4.11/bin
  CALDB       : 4.8.2

Getting to know Chandra data¶

In this section students will obtain Chandra data and learn some techniques to inspect the quality of the data and how to reprocess their dataset.

Download Dataset¶

Exercise 1¶

Obtain the standard data distribution for Chandra OBS_ID 13858.

Q: How did you obtain the data (Chaser, download_chandra_obsid, FTP)?

In [2]:

/bin/rm -rf 13858
download_chandra_obsid 13858

Downloading files for ObsId 13858, total size is 68 Mb.

  Type     Format      Size  0........H.........1  Download Time Average Rate
  ---------------------------------------------------------------------------
  readme   ascii      11 Kb  ####################          < 1 s  150.9 kb/s
  oif      fits       23 Kb  ####################          < 1 s  996.8 kb/s
  vv       pdf        35 Kb  ####################          < 1 s  315.6 kb/s
  cntr_img fits       77 Kb  ####################          < 1 s  2046.4 kb/s
  cntr_img jpg       371 Kb  ####################          < 1 s  10757.1 kb/s
  evt2     fits        5 Mb  ####################          < 1 s  22074.2 kb/s
  full_img fits       52 Kb  ####################          < 1 s  4736.1 kb/s
  full_img jpg        42 Kb  ####################          < 1 s  2490.1 kb/s
  bpix     fits       11 Kb  ####################          < 1 s  723.2 kb/s
  fov      fits        6 Kb  ####################          < 1 s  717.4 kb/s
  eph1     fits      308 Kb  ####################          < 1 s  3212.8 kb/s
  asol     fits        4 Mb  ####################          < 1 s  5839.1 kb/s
  evt1     fits       22 Mb  ####################            2 s  14225.9 kb/s
  flt      fits        6 Kb  ####################          < 1 s  129.6 kb/s
  msk      fits        5 Kb  ####################          < 1 s  36.8 kb/s
  mtl      fits      640 Kb  ####################          < 1 s  9206.5 kb/s
  stat     fits      360 Kb  ####################          < 1 s  5311.4 kb/s
  bias     fits      493 Kb  ####################          < 1 s  3922.4 kb/s
  bias     fits      450 Kb  ####################          < 1 s  6816.9 kb/s
  bias     fits      422 Kb  ####################          < 1 s  6443.3 kb/s
  bias     fits      436 Kb  ####################          < 1 s  6731.9 kb/s
  pbk      fits        4 Kb  ####################          < 1 s  273.2 kb/s
  vv       pdf        32 Mb  ####################            2 s  19850.3 kb/s
  eph1     fits        8 Kb  ####################          < 1 s  331.9 kb/s
  eph1     fits      303 Kb  ####################          < 1 s  2374.5 kb/s
  eph1     fits      283 Kb  ####################          < 1 s  2758.3 kb/s
  osol     fits      368 Kb  ####################          < 1 s  4895.8 kb/s
  aqual    fits      250 Kb  ####################          < 1 s  4394.2 kb/s
  osol     fits      358 Kb  ####################          < 1 s  4826.9 kb/s
  osol     fits      129 Kb  ####################          < 1 s  2249.6 kb/s
  osol     fits      368 Kb  ####################          < 1 s  3816.5 kb/s

      Total download size for ObsId 13858 = 68 Mb
      Total download time for ObsId 13858 = 6 s

Students may want to uncompress, gunzip, all of the files in the top-level directory, along with the primary and secondary directories at this point.

In [3]:

gunzip 13858/primary/*gz
gunzip 13858/secondary/*gz

In [4]:

download_obsid_caldb 13858 ./CALDB

download_obsid_caldb
          infile = 13858
          outdir = ./CALDB
      background = no
         missing = no
         clobber = no
         verbose = 1
            mode = ql

Retrieving files for CALDB_VER = 4.8.2
Retrieving CALDB index files
Processing infile=13858/primary/acisf13858N001_evt2.fits
Retrieving CALDB data files
    Filename:                               0------------------1
    telD1999-07-23skyN0002.fits             ####################
    telD1999-07-23aimptsN0002.fits          ####################
    telD1999-07-23tdetN0001.fits            ####################
    telD1999-07-23sgeomN0001.fits           ####################
    telD1999-07-23geomN0006.fits            ####################
    acisD2010-02-01qeuN0007.fits            ####################
    hrmaD1996-12-20vignetN0003.fits         ####################
    hrmaD1996-12-20axeffaN0008.fits         ####################
    acisD2000-11-28badpixN0003.fits         ####################
    acisD1999-08-13contamN0012.fits         ####################
    acisD1997-04-17qeN0006.fits             ####################
    acisD2000-01-29gain_ctiN0007.fits       ####################
    acisD2010-01-01ctiN0008.fits            ####################
    acisD2005-07-01evtspltN0002.fits        ####################
    acisD2012-05-01t_gainN0006.fits         ####################
    acisD1996-11-01gradeN0004.fits          ####################
    acisD2000-01-29grdimgN0001.fits         ####################
    acisD1999-07-22subpixN0001.fits         ####################
    acisD2000-01-29fef_pha_ctiN0004.fits    ####################
    acisD1999-09-16dead_areaN0001.fits      ####################
    hrmaD1996-12-20reefN0001.fits           ####################
    acisD2000-01-29p2_respN0007.fits        ####################
    hrmaD1996-11-01wpsfN0001.fits           ####################
    acisD2000-01-29osip_ctiN0006.fits       ####################
    acisD2002-11-15gtilimN0003.fits         ####################

Be sure to setup the following environment variables to
use these CALDB files.

(t)csh:
setenv CALDB /data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB
setenv CALDBCONFIG /data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB/software/tools/caldb.config
setenv CALDBALIAS none

bash:
export CALDB=/data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB
export CALDBCONFIG=/data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB/software/tools/caldb.config
export CALDBALIAS=none

In [5]:

export CALDB=`pwd`/CALDB
export CALDBCONFIG=`pwd`/CALDB/software/tools/caldb.config
export CALDBALIS=none

Review V&V Report¶

Exercise 2¶

Read the Chandra Verification and Validation report in the top level directory of the data distribution: axaff13858N001_VV001_vv2.pdf

In [6]:

acroread 13858/axaff13858N001_VV001_vv2.pdf  &

[1] 813

Q: Summarize the V&V comments: Joint Proposal: NRAO

Q: What is the target of this observation? SDSS J091449.05+085321.1

Q: What is the sequence number SEQ_NUM of this observation? 702584

In [7]:

echo

(acroread:813): Gtk-WARNING **: /usr/lib64/gtk-2.0/2.10.0/engines/libcrux-engine.so: wrong ELF class: ELFCLASS64

(acroread:813): Gtk-WARNING **: /usr/lib64/gtk-2.0/2.10.0/engines/libcrux-engine.so: wrong ELF class: ELFCLASS64
Gtk-Message: Failed to load module "canberra-gtk-module"

(acroread:813): Pango-WARNING **: /export/ciao-4.11/ots/lib/pango/1.6.0/modules/pango-basic-fc.so: wrong ELF class: ELFCLASS64

[1]+  Done                    acroread 13858/axaff13858N001_VV001_vv2.pdf

Display data in ds9¶

Locate the event files. There are two event files included in the standard data distribution. The Level 1 event file, evt1, is in the secondary/ directory, the Level 2 event file, evt2, is in the primary/ directory.

In [8]:

/bin/ls -1 13858/*/*evt*.fits

13858/primary/acisf13858N001_evt2.fits
13858/secondary/acisf13858_000N001_evt1.fits

Exercise 3¶

Display the Level 2 event file using ds9. Make sure to use Log scale.

In [9]:

ds9 13858/primary/acisf13858N001_evt2.fits -scale log &

[1] 961

Q: Using the cursor, record the coordinate of the bright source in the center of the image:

In [10]:

xpaset -p ds9 crosshair 4103 4115 physical
xpaset -p ds9 saveimage png ds9_01.png
display < ds9_01.png

In [11]:

xpaget ds9 crosshair wcs fk5 sexagesimal
xpaset -p ds9 mode region

9:14:49.1404 +8:53:20.741

Exercise 4¶

By default, ds9 only shows part of the Chandra dataset. Use the Bin menu to bin the data by a factor of 4 and then by a factor of 8.

Binning is different than Zooming. Zooming is done to the image (so after the event file has been binned). Set bin=1, and then zoom to ⅛.

In [12]:

xpaset -p ds9 bin factor 4
xpaset -p ds9 saveimage png ds9_exercise04_a.png
display < ds9_exercise04_a.png

In [13]:

xpaset -p ds9 bin factor 8
xpaset -p ds9 saveimage png ds9_exercise04_b.png
display < ds9_exercise04_b.png

In [14]:

xpaset -p ds9 bin factor 1
xpaset -p ds9 zoom 0.125
xpaset -p ds9 saveimage png ds9_exercise04_c.png
display < ds9_exercise04_c.png

Q: Describe why the Bin 8 image is different than the image Zoom'ed by ⅛:

Zooming takes the default 1024x1024 image and samples every 8th row and column to create the display. Binning rebins the original event file with 8x8 pixels.*

Exercise 5¶

ds9 bins all the events in the event file into an image. That includes all events for all time and with all energies.

Use the Bin → Binning Parameters menu to add an energy=500:7000 as a Bin Filter.

In [15]:

xpaset -p ds9 zoom to 1
xpaset -p ds9 bin filter 'energy=500:7000'
xpaset -p ds9 saveimage png ds9_exercise05_a.png
display < ds9_exercise05_a.png

Q: Describe the difference in the images compared to the unfiltered image: Less background noise, max pixel value is lower (see colorbar scale compared to above)

Q: What unit is energy in? eV

Extra Credit¶

Try different energy ranges

In [16]:

xpaset -p ds9 bin filter 'energy=2000:10000'
xpaset -p ds9 saveimage png ds9_exercise05_extra1.png
display < ds9_exercise05_extra1.png

Try different time ranges

In [17]:

dmlist 13858/primary/acisf13858N001_evt2.fits header | grep TST

0047 TSTART               456518974.4871399999 [s]       Real8        Observation start time (MET)
0059 TSTOP                456536836.1380900145 [s]       Real8        Observation end time (MET)
0085 STARTBEP             301280942                      Int4         BEP timer value at TSTART
0086 STOPBEP              1808980142                     Int4         BEP timer value at TSTOP
 --  HISTORY               PARM  :tstart=TSTART                                           ASC00210
 --  HISTORY               PARM  :tstop=TSTOP                                             ASC00211

In [18]:

xpaset -p ds9 bin filter 'time=456520000:456530000'
xpaset -p ds9 saveimage png ds9_exercise05_extra2.png
display < ds9_exercise05_extra2.png

In [19]:

xpaset -p ds9 quit

[1]+  Done                    ds9 13858/primary/acisf13858N001_evt2.fits -scale log

Inspect headers¶

Most Chandra data products are FITS files, specifically FITS binary tables, with extensive headers that fully describe the dataset. Students should become familiar with some of the basic keywords and tool used to examine those keywords

Exercise 7¶

Open the Level 2 event file in prism

The EVENTS extension is automatically selected. The Header Keywords are shown in the top, right pane.

In [20]:

prism 13858/primary/acisf13858N001_evt2.fits &

[1] 1274

In [21]:

import -window prism prism_exercise7.png
display < prism_exercise7.png