This Jupyter notebook uses the bash_kernel
. Users will need to have this kernel installed before they can run the commadns in this notebook.
CIAO 4.11 users with pip3
can use the following commands to install bash_kernel
into their CIAO installtion:
pip3 install bash_kernel
python -m bash_kernel.install --sys-prefix
jupyter kernelspec list
after which they can then run
jupyter notebook
These exercises are designed to familiarize students with Chandra data and some of the basics of X-ray data analysis using CIAO with Chandra data.
Students will use a variety of tools and scripts to perform typical Chandra data analysis tasks. These exercises are not intended to be a Cookbook for all Chandra data analysis, but rather are meant to expose students to basic X-ray analysis techniques.
Students should already have CIAO installed on their systems. The exercises here have been developed with CIAO 4.11, using the CIAO-4.11.1 contributed scripts package, with Chandra CALDB 4.8.2. Students should also have MARX installed. These exercises have been developed using MARX 5.4.0. The install_marx script in CIAO can be used to automate the installation process.
ciaover
In this section students will obtain Chandra data and learn some techniques to inspect the quality of the data and how to reprocess their dataset.
Obtain the standard data distribution for Chandra OBS_ID 13858.
Q: How did you obtain the data (Chaser, download_chandra_obsid, FTP)?
/bin/rm -rf 13858
download_chandra_obsid 13858
Students may want to uncompress, gunzip, all of the files in the top-level directory, along with the primary and secondary directories at this point.
gunzip 13858/primary/*gz
gunzip 13858/secondary/*gz
download_obsid_caldb 13858 ./CALDB
export CALDB=`pwd`/CALDB
export CALDBCONFIG=`pwd`/CALDB/software/tools/caldb.config
export CALDBALIS=none
Read the Chandra Verification and Validation report in the top level directory of the data distribution: axaff13858N001_VV001_vv2.pdf
acroread 13858/axaff13858N001_VV001_vv2.pdf &
Q: Summarize the V&V comments: Joint Proposal: NRAO
Q: What is the target of this observation? SDSS J091449.05+085321.1
Q: What is the sequence number SEQ_NUM of this observation? 702584
echo
Locate the event files. There are two event files included in the standard data distribution. The Level 1 event file, evt1, is in the secondary/ directory, the Level 2 event file, evt2, is in the primary/ directory.
/bin/ls -1 13858/*/*evt*.fits
Display the Level 2 event file using ds9. Make sure to use Log scale.
ds9 13858/primary/acisf13858N001_evt2.fits -scale log &
Q: Using the cursor, record the coordinate of the bright source in the center of the image:
xpaset -p ds9 crosshair 4103 4115
xpaset -p ds9 saveimage png ds9_01.png
display < ds9_01.png
xpaget ds9 crosshair wcs fk5 sexagesimal
xpaset -p ds9 mode region
By default, ds9 only shows part of the Chandra dataset. Use the Bin menu to bin the data by a factor of 4 and then by a factor of 8.
Binning is different than Zooming. Zooming is done to the image (so after the event file has been binned). Set bin=1, and then zoom to ⅛.
xpaset -p ds9 bin factor 4
xpaset -p ds9 saveimage png ds9_exercise04_a.png
display < ds9_exercise04_a.png
xpaset -p ds9 bin factor 8
xpaset -p ds9 saveimage png ds9_exercise04_b.png
display < ds9_exercise04_b.png
xpaset -p ds9 bin factor 1
xpaset -p ds9 zoom 0.125
xpaset -p ds9 saveimage png ds9_exercise04_c.png
display < ds9_exercise04_c.png
Q: Describe why the Bin 8 image is different than the image Zoom'ed by ⅛:
Zooming takes the default 1024x1024 image and samples every 8th row and column to create the display. Binning rebins the original event file with 8x8 pixels.*
ds9 bins all the events in the event file into an image. That includes all events for all time and with all energies.
Use the Bin → Binning Parameters menu to add an energy=500:7000 as a Bin Filter.
xpaset -p ds9 zoom to 1
xpaset -p ds9 bin filter 'energy=500:7000'
xpaset -p ds9 saveimage png ds9_exercise05_a.png
display < ds9_exercise05_a.png
Q: Describe the difference in the images compared to the unfiltered image: Less background noise, max pixel value is lower (see colorbar scale compared to above)
Q: What unit is energy in? eV
Try different energy ranges
xpaset -p ds9 bin filter 'energy=2000:10000'
xpaset -p ds9 saveimage png ds9_exercise05_extra1.png
display < ds9_exercise05_extra1.png
Try different time ranges
dmlist 13858/primary/acisf13858N001_evt2.fits header | grep TST
xpaset -p ds9 bin filter 'time=456520000:456530000'
xpaset -p ds9 saveimage png ds9_exercise05_extra2.png
display < ds9_exercise05_extra2.png
xpaset -p ds9 quit
Most Chandra data products are FITS files, specifically FITS binary tables, with extensive headers that fully describe the dataset. Students should become familiar with some of the basic keywords and tool used to examine those keywords
Open the Level 2 event file in prism
The EVENTS extension is automatically selected. The Header Keywords are shown in the top, right pane.
prism 13858/primary/acisf13858N001_evt2.fits &
import -window prism prism_exercise7.png
display < prism_exercise7.png
Q: Record the value for the following keywords:
MISSION
: AXAFTELESCOP
: CHANDRAINSTRUME
: ACISDETNAM
: ACIS-5678GRATING
: NONEDATAMODE
: FAINTREADMODE
: TIMEDDATE-OBS
: 2012-06-19T18:49:34OBSERVER
: Dr. Kayhan GultekinONTIME
: 15069.1001159Q: What units are the ACIS focal plane temperature, FP_TEMP, recorded in? K
xpaset -p prism quit
Use dmlist with the header option to display the header.
dmlist 13858/primary/acisf13858N001_evt2.fits header,clean | \
egrep '^._TARG|^.*_NOM|^.*_PNT|^SIM_|^DTCOR|^ASCDSVER|^CALDBVER'
Q: Record the value for the following header keywords: see above
Q: This is an "ACIS-S" observation. How can you tell this from the event file header information? **There is nothing in the header that says this is an
'ACIS-S' observation directly. The value of the SIM_Z
keyword indicates that
the aim point is located on the ACIS-S array**
dmkeypar 13858/primary/acisf13858N001_evt2.fits TIMEDEL echo+
dmmakepar 13858/primary/acisf13858N001_evt2.fits[events] \
dmmakepar_exercise08.par clob+
pdump dmmakepar_exercise08.par | \
egrep -i '^._TARG|^.*_NOM|^.*_PNT|^SIM_|^DTCOR|^ASCDSVER|^CALDBVER'
plist dmmakepar_exercise08.par | \
egrep -i '^._TARG|^.*_NOM|^.*_PNT|SIM_|DTCOR|ASCDSVER|CALDBVER'
pget dmmakepar_exercise08.par \
sim_x sim_y sim_z ra_pnt dec_pnt roll_pnt ra_nom dec_nom roll_nom \
dtcor caldbver ascdsver
dmhistory 13858/primary/acisf13858N001_evt2.fits acis_process_events
The Chandra calibration database (CALDB) is continually updated. The then most recent CALDB is used for observations as they are taken. Some calibrations, such as the time dependent gain calibrations, can only be definitively computed based on historical observations; thus the file in the current CALDB is always predicted. These calibrations are later updated to be definitive post facto.
The CIAO team strongly advises users to always reprocess data obtained from the archive.
Use chandra_repro to reprocess the dataset.
chandra_repro 13858 out= clob+
dmdiff 13858/primary/acisf13858N001_evt2.fits 13858/repro/acisf13858_repro_evt2.fits data- || echo
dmkeypar 13858/primary/acisf13858N001_evt2.fits CALDBVER echo+
dmkeypar 13858/repro/acisf13858_repro_evt2.fits CALDBVER echo+
check_ciao_caldb
dmlist 13858/primary/acisf13858N001_evt2.fits counts
dmlist 13858/repro/acisf13858_repro_evt2.fits counts
Q: Compare the header keyword values in the _repro_evt2.fits file with the evt2 file obtained from the archive. Discuss the differences: We see newer calibration files are used (
GAINFILE
,CTIFILE
, andTGAINFIL
). We see small diff inONTIME
.
Q: What version of the CALDB is installed? What is the value of the CALDBVER keyword? It is unchanged.
CALDBVER
is not updated by the tools; it's set in SDP.
Q: Compare the number of events in the reprocessed Level 2 event file with the number of events in the archived Level 2 event file. Why are they same (or not the same)?
Slightly different. New calibrations mean some good events go bad, some bad events go good (grade migration)
In this section students will exercise some of the basic CIAO tools and scripts needed to perform basic imaging tasks.
One of the most common analysis tasks is to detect sources.
Please keep in mind: CIAO detect tools only detect candidate sources and they are not photometric tools. The output from the detect tools should be used to guide further analysis.
See also:
In this exercise students should complete the following steps
Create image and exposure map. Run the fluximage script on the dataset. Students should be sure to set the binsize=1 and set bands=default or bands=broad.
Create the psfmap. Run mkpsfmap using the fluximage output image as input. Students should use energy=1.4967 ecf=0.393
Detect Sources. Run wavdetect on the fluximage output counts image, using the exposure map and psfmap. Students should select a set of wavelet scales to use that is appropriate for the dataset being analyzed.
fluximage 13858/repro/acisf13858_repro_evt2.fits out=acisf13858 bin=1 bands=broad clob+ mode=h
mkpsfmap acisf13858_broad_thresh.img acisf13858_broad.psfmap \
energy=1.4967 ecf=0.393 mode=h clob+
pset wavdetect \
infile=acisf13858_broad_thresh.img \
expfile=acisf13858_broad_thresh.expmap \
psffile=acisf13858_broad.psfmap \
scales='1.4 2 4 8 12 16 24 32' \
outfile=acisf13858_wav.src \
scellfile=acisf13858_wav.cell \
imagefile=acisf13858_wav.recon \
defnbkg=acisf13858_wav.nbkg \
clob+
wavdetect mode=h
ds9 acisf13858_broad.psfmap -scale linear -cmap standard -zoom to fit \
-saveimage png ds9_exercise10_a.png -quit
display < ds9_exercise10_a.png
ds9 acisf13858_broad_thresh.img -block 2 -scale log -region acisf13858_wav.src \
-scale limits 0 20 -smooth -saveimage png ds9_exercise10_b.png -quit
display < ds9_exercise10_b.png
Q: Display the PSF map in ds9. Why does it look the way it does? Chandra PSF increases in size the further away from optical axis.
Q: Record choice of wavelet scales and why that set was selected: above. Pseduo sqrt(2) used to detect srcs across field
Q: Open the broad-band image in ds9 and load the source list as a region file. Comment on the detected sources and any missed detections: src detects look good ; regions are a little small; maybe some faint stuff missed
Q: Are the ellipses the position error or the apparent size of the source?
They are a measure of the observed source size. They are not the intrinsic (deconvolved) source size, nor are they the position error.
Try differenct wavelet scales
wavdetect scales='1 4 16 64' mode=h clobber=yes outfile=fewscales.src
ds9 acisf13858_broad_thresh.img -block 2 -scale log -region fewscales.src \
-scale limits 0 20 -smooth -saveimage png ds9_exercise10_ec1.png -quit
display < ds9_exercise10_ec1.png
Try different significance thresholds, sigthresh.
wavdetect sigthresh=1e-7 mode=h clobber=yes outfile=lowersigthresh.src
ds9 acisf13858_broad_thresh.img -block 2 -scale log -region lowersigthresh.src \
-scale limits 0 20 -smooth -saveimage png ds9_exercise10_ec2.png -quit
display < ds9_exercise10_ec2.png
Try different ecf values when making the PSF map.
mkpsfmap acisf13858_broad_thresh.img ecf90.psfmap \
energy=1.4967 ecf=0.90 mode=h clob+
wavdetect mode=h clobber=yes outfile=ecf90.src psffile=ecf90.psfmap
ds9 acisf13858_broad_thresh.img -block 2 -scale log -region lowersigthresh.src \
-scale limits 0 20 -smooth -saveimage png ds9_exercise10_ec3.png -quit
display < ds9_exercise10_ec3.png
Try different energy bands.
mkpsfmap acisf13858_broad_thresh.img 5keV.psfmap \
energy=5.0 ecf=0.90 mode=h clob+
wavdetect mode=h clobber=yes outfile=5keV.src psffile=5keV.psfmap
ds9 acisf13858_broad_thresh.img -block 2 -scale log -region 5keV.src \
-scale limits 0 20 -smooth -saveimage png ds9_exercise10_ec4.png -quit
display < ds9_exercise10_ec4.png
Try using the celldetect tool.
celldetect acisf13858_broad_thresh.img \
expstk=acisf13858_broad_thresh.expmap \
psffile=ecf90.psfmap \
out=acisf13858_cell.src \
clob+ mode=h
ds9 acisf13858_broad_thresh.img -block 2 -scale log -region acisf13858_cell.src \
-scale limits 0 20 -smooth -saveimage png ds9_exercise10_ec5.png -quit
display < ds9_exercise10_ec5.png
Try using the vtpdetect tool.
vtpdetect acisf13858_broad_thresh.img \
exp=acisf13858_broad_thresh.expmap \
out=acisf13858_vtp.src \
clob+ mode=h
ds9 acisf13858_broad_thresh.img -block 2 -scale log -region acisf13858_vtp.src[src_region] \
-scale limits 0 20 -smooth -saveimage png ds9_exercise10_ec6.png -quit
display < ds9_exercise10_ec6.png
The detect tools are one way to create regions automatically. However, often users analyzing a single source in an observation will simply create their own region in ds9 and then use those regions with the CIAO tools.
ds9 acisf13858_broad_thresh.img -scale log -pan to 4096.5 4096.5 physical &
echo "physical;circle(4104.5,4115.5,10)" | xpaset ds9 region
xpaset -p ds9 saveimage png ds9_exercise11_a.png
xpaget ds9 regions -format ds9 -system wcs -skyformat sexagesimal > ds9_src.reg
xpaset -p ds9 regions delete all
display < ds9_exercise11_a.png
cat ds9_src.reg
echo "physical;annulus(4105.5,4113.5,45,125) # background" | xpaset ds9 region
xpaset -p ds9 saveimage png ds9_exercise11_b.png
xpaget ds9 regions -format ciao -system physical > ds9_bkg.reg
xpaset -p ds9 regions delete all
display < ds9_exercise11_b.png
cat ds9_bkg.reg
xpaset -p ds9 quit
Q: Record the source location used: see above
Q: Describe the differences between the CIAO format and the ds9 format regions:
ds9 format includes meta-data: color, linestyle, etc.
Compute source centroid. In this exercise students will use the dmstat tool to compute the source centroid in their region.
dmcopy "acisf13858_broad_thresh.img[sky=region(ds9_src.reg)]" dmcopy_e12_src.fits clob+
dmlist dmcopy_e12_src.fits blocks,cols
dmstat dmcopy_e12_src.fits cen+ sig- med+
dmcopy 13858/repro/acisf13858_repro_evt2.fits"[sky=region(ds9_src.reg)][bin sky=1]" \
dmcopy_e12_evtsrc.fits clob+
dmlist dmcopy_e12_evtsrc.fits blocks,cols
dmstat dmcopy_e12_evtsrc.fits cen+ sig- med+
Q: Is the filtered image the same size as the fluximage output image size? Why?
The CXCDM shrinks the image to the size of the region (by default)
Q: Record the centroid of fluximage output filtered image:
4104.4755245, 4115.8174825
Q: What are the good and null values that dmstat reports?
The number of pixels outside the circle
Q: Is the image created by filtering and binning the event file the same size as the other images? Why?
No. (20x20 vs 21x21). Filter image vs. Filter table then bin into an image
Q: Record the centroid of the event file filter and binned image:
4104.4519153, 4115.808491
Q: Is the centroid the same for the two images? Why? (Hint: what energies are being used?)
Opps, no energy filter on the event file -- but image was made with broad band (500:7000eV).
Obtaining the counts (or count rate) is the first step in computing the source flux.
In this exercise students will use the CIAO analysis menu, aka dax, to get the net counts in their regions (from Exercise 11).
$ASCDS_INSTALL/bin/ds9 acisf13858_broad_thresh.img -scale log -pan to 4096.5 4096.5 physical &
xpaset -p ds9 regions ds9_bkg.reg
xpaset -p ds9 regions select all
xpaset -p ds9 regions background
xpaset -p ds9 regions ds9_src.reg
xpaset -p ds9 saveimage png exercise_13_ds9a.png
display < exercise_13_ds9a.png
xpaset -p ds9 regions select all
xpaset -p ds9 analysis task "{Net Counts}"
import -window 'Net Counts' exercise_13_a.png
display < exercise_13_a.png
xpaset -p ds9 regions delete all
echo "physical; circle(4104.4434,4115.8478,17.145457)" | xpaset ds9 regions
echo "physical; annulus(4104.5,4115.5,26.305893,125) # background" | xpaset ds9 regions
xpaset -p ds9 saveimage png exercise_13_ds9b.png
display < exercise_13_ds9b.png
xpaset -p ds9 regions select all
xpaset -p ds9 analysis task "{Net Counts}"
import -window 'Net Counts' exercise_13_b.png
display < exercise_13_b.png
xpaset -p ds9 quit
Q: From step 7, record the following information from the Net Counts task:
Q: From step 8, describe how the net counts and net rates vary with changes to the source and background regions: net rate doesn't change much as bkg gets bigger/smaller (ie background is flat)
In this exercise students will use the srcflux
script to get various estimates of the flux of the source.
punlearn dmcoords
dmcoords 13858/repro/acisf13858_repro_evt2.fits op=sky x=4104.4755245 y=4115.8174825 celfmt=hms verb=0
pget dmcoords ra dec
punlearn srcflux
srcflux 13858/repro/acisf13858_repro_evt2.fits "09:14:49.088, +08:53:21.16" exercise14_step2 clob+ mode=h
punlearn srcflux
srcflux 13858/repro/acisf13858_repro_evt2.fits "09:14:49.088, +08:53:21.16" exercise14_step3 \
psfmethod=arfcorr \
clob+ mode=h verbose=0
cat exercise14_step3_summary.txt
punlearn srcflux
srcflux 13858/repro/acisf13858_repro_evt2.fits "09:14:49.088, +08:53:21.16" exercise14_step4 \
psfmethod=arfcorr \
model="xsbbody.black_body" paramvals="black_body.kT=1" \
clob+ mode=h verbose=0
cat exercise14_step4_summary.txt
punlearn srcflux
srcflux 13858/repro/acisf13858_repro_evt2.fits "09:14:49.088, +08:53:21.16" exercise14_step5 \
psfmethod=arfcorr \
model="xsbbody.black_body" paramvals="black_body.kT=1" \
band="hard" \
clob+ mode=h verbose=0
cat exercise14_step5_summary.txt
Q: Record the net count rate, model independent flux, model flux from steps 2 through 5
.
Estimate | Step 2 | Step 3 | Step 4 | Step 5 |
---|---|---|---|---|
Count Rate | 0.827 | 0.962 | 0.962 | 0.0185 |
Flux | 5.6E-13 | 7.3E-13 | 7.3E-13 | 3.12E-13 |
Model Flux | 5.8E-13 | 6.74E-13 | 1.0E-12 | 2.91E-13 |
.
Q: Discuss the differences in the estimated fluxes obtained in steps 2 through 5.
Try using psfmethod=quick and psfmethod=marx
punlearn srcflux
srcflux 13858/repro/acisf13858_repro_evt2.fits "09:14:49.088, +08:53:21.16" exercise14_step_ec1 \
psfmethod=quick \
clob+ mode=h verbose=0
cat exercise14_step_ec1_summary.txt
source $ASCDS_INSTALL/marx-5.4.0/setup_marx.sh
punlearn srcflux
srcflux 13858/repro/acisf13858_repro_evt2.fits "09:14:49.088, +08:53:21.16" exercise14_step_ec2 \
psfmethod=marx \
clob+ mode=h verbose=0
cat exercise14_step_ec2_summary.txt
Estimate | ideal | quick | arfcorr | marx |
---|---|---|---|---|
Count Rate | 0.827 | 0.962 | 0.955 | 0.0985 |
Flux | 5.6E-13 | 7.3E-13 | 6.5E-13 | 7.84E-13 |
Model Flux | 5.8E-13 | 6.74E-13 | 6.7E-12 | 6.90E-13 |
Try other spectral model and other model parameters.
Try other energy bands (2-10 keV is a common band in the literature).
punlearn srcflux
srcflux 13858/repro/acisf13858_repro_evt2.fits "09:14:49.088, +08:53:21.16" exercise14_step_ec4 \
psfmethod=quick band="2.0:10.0:3.0" \
clob+ mode=h verbose=0
cat exercise14_step_ec4_summary.txt
The Chandra Point Spread Function (PSF) varies spatially across the field of view as well with energy. There is no analytic model of the PSF. The only way to obtain an estimate of the Chandra PSF for detailed spatial analysis is by simulation.
The psfmethod used by srcflux are only appropriate when integrating over a region. The pixel-to-pixel variations used in those methods is insufficient for any type of analysis that requires detailed spatial information.
There are two different simulators users can use.
In this exercise Students will simulate the PSF using ChaRT and with MARX. The simulate_psf script is used to create PSF images that are appropriate for further data analysis.
source $ASCDS_INSTALL/marx-5.4.0/setup_marx.sh
dmkeypar exercise14_step2_broad.flux net_photflux_aper echo+
curl \
-F email=${USER}@head.cfa.harvard.edu \
-F coords=cel \
-F ra=09:14:49.088 \
-F dec=+08:53:21.16 \
-F asol=obi \
-F obsid=13858 \
-F obinum=0\
-F niter=10\
-F randseed=32767 \
-F energy=mono \
-F mono=2.3 \
-F flux=0.000221 \
http://saotrace.cfa.harvard.edu/cgi-bin/runwrapper
curl -o rays.tar.gz ftp://saotrace.cfa.harvard.edu/pub/ChaRT/HRMA_ra138.70453_dec8.88921_en2.3_flux0.000221_dithered.tar.gz
tar xvfz rays.tar.gz
simulate_psf 13858/repro/acisf13858_repro_evt2.fits \
chart_sim ra=138.70453 dec=8.88921 \
simulator=file rayfile="HRMA*rays.fits" \
minsize=128 mode=h
ds9 acisf13858_broad_thresh.img -scale log -pan to 4096.5 4096.5 physical \
-zoom to 4 \
chart_sim.psf -pan to 4096.5 4096.5 physical \
-view colorbar no \
-saveimage png ds9_exercise15_a.png -quit
display < ds9_exercise15_a.png
simulate_psf 13858/repro/acisf13858_repro_evt2.fits \
marx_sim ra=138.70453 dec=8.88921 \
simulator=marx \
minsize=128 mode=h \
mono=2.3 flux=0.000221 numiter=10
ds9 acisf13858_broad_thresh.img -scale log -pan to 4096.5 4096.5 physical \
-zoom to 4 \
marx_sim.psf -pan to 4096.5 4096.5 physical \
-view colorbar no \
-saveimage png ds9_exercise15_b.png -quit
display < ds9_exercise15_b.png
Q: Comment on a visual comparison of the ChaRT/SAOTrace PSF to the observation:
Q: Comment on a visual comparison of the MARX PSF to the observation:
Q: Describe any differences between the MARX and ChaRT PSFs:
Q: Is this a point source?
Try setting simulate_psf readout_streak=yes. Try pileup=yes. Try blur=0.20
simulate_psf 13858/repro/acisf13858_repro_evt2.fits \
marxA_sim ra=138.70453 dec=8.88921 \
simulator=marx \
minsize=128 mode=h \
mono=2.3 flux=0.000221 numiter=10 verb=0 readout+
simulate_psf 13858/repro/acisf13858_repro_evt2.fits \
marxB_sim ra=138.70453 dec=8.88921 \
simulator=marx \
minsize=128 mode=h \
mono=2.3 flux=0.000221 numiter=10 verb=0 pileup+ ext-
simulate_psf 13858/repro/acisf13858_repro_evt2.fits \
marxC_sim ra=138.70453 dec=8.88921 \
simulator=marx \
minsize=128 mode=h \
mono=2.3 flux=0.000221 numiter=10 verb=0 blur=0.2
ds9 marx_sim.psf -scale log -pan to 4105.5 4113.5 physical \
-zoom to 4 \
marxA_sim.psf -pan to 4105.5 4113.5 physical \
marxB_sim.psf -pan to 4105.5 4113.5 physical \
marxC_sim.psf -pan to 4105.5 4113.5 physical \
-view colorbar no \
-saveimage png ds9_exercise15_ec1.png -quit
display < ds9_exercise15_ec1.png
Run the Lucy-Richardson deconvolution tool arestore using the broad band flux image and the PSFs simulated here. Try with different numbers of iterations.
arestore "acisf13858_broad_thresh.img[sky=bounds(region(ds9_bkg.reg))]" \
chart_sim.psf arestore.out num=150 clob+
ds9 acisf13858_broad_thresh.img -scale log -pan to 4105.5 4113.5 physical \
-zoom to 4 \
chart_sim.psf -pan to 4105.5 4113.5 physical \
arestore.out -pan to 4105.5 4113.5 physical \
-view colorbar no \
-tile mode column \
-saveimage png ds9_exercise15_ec2.png -quit
display < ds9_exercise15_ec2.png
Three color (aka "true color" or "tri-color") images can be useful to help guide analysis by providing visual clues about changes in spectra.
ds9 -rgb -red root_soft_flux.img \
-green root_medium_flux.img \
-blue root_hard_flux.img
/bin/rm -rf 13736
download_chandra_obsid 13736 evt1,bpix,flt,mtl,msk,dtf,bias,pbk,asol,stat
download_obsid_caldb 13736 ./CALDB
chandra_repro 13736 out=
fluximage 13736 out=tricolor band=csc bin=8 clob+
ds9 -rgb -view colorbar no \
-red tricolor_soft_flux.img -scale log -smooth yes -smooth radius 1 \
-green tricolor_medium_flux.img -scale log -smooth yes -smooth radius 1 \
-blue tricolor_hard_flux.img -scale log -smooth yes -smooth radius 1 \
-saveimage png ds9_exercise_16.png -quit
display < ds9_exercise_16.png
dmimg2jpg \
infile=tricolor_soft_flux.img \
greenfile=tricolor_medium_flux.img \
bluefile=tricolor_hard_flux.img \
outfile=dmimg2jpg_exercise13.jpg \
clob+
display < dmimg2jpg_exercise13.jpg
Q: Discuss the differences in the output from ds9 and dmimg2jpg. ds9 interactive
Smooth the images with aconvolve before displaying
# Should convolve image and exposure map separately!
aconvolve tricolor_soft_flux.img tricolor_soft_flux_gaus.img "lib:gaus(2,5,1,1,1)" meth=slide clob+
aconvolve tricolor_medium_flux.img tricolor_medium_flux_gaus.img "lib:gaus(2,5,1,1,1)" meth=slide clob+
aconvolve tricolor_hard_flux.img tricolor_hard_flux_gaus.img "lib:gaus(2,5,1,1,1)" meth=slide clob+
dmimg2jpg \
infile=tricolor_soft_flux_gaus.img \
greenfile=tricolor_medium_flux_gaus.img \
bluefile=tricolor_hard_flux_gaus.img \
outfile=dmimg2jpg_exercise13_aconvolve.jpg \
clob+
display < dmimg2jpg_exercise13_aconvolve.jpg
Smooth the images with csmooth before displaying
# Csmooth really, really wants integer counts, so we smooth the counts image not the flux'ed image.
csmooth tricolor_soft_thresh.img none tricolor_soft_flux_csm.img sigmin=3 sclmax=20 mode=h clob+
csmooth tricolor_medium_thresh.img none tricolor_medium_flux_csm.img sigmin=3 sclmax=20 mode=h clob+
csmooth tricolor_hard_thresh.img none tricolor_hard_flux_csm.img sigmin=3 sclmax=20 mode=h clob+
dmimg2jpg \
infile=tricolor_soft_flux_csm.img \
greenfile=tricolor_medium_flux_csm.img \
bluefile=tricolor_hard_flux_csm.img \
outfile=dmimg2jpg_exercise13_csm.jpg \
clob+
display < dmimg2jpg_exercise13_csm.jpg
Smooth the images with dmimgadapt before displaying
dmimgadapt tricolor_soft_thresh.img tricolor_soft_flux_cone.img cone min=1 max=20 num=100 radscale=linear counts=25 clob+ mode=h
dmimgadapt tricolor_medium_thresh.img tricolor_medium_flux_cone.img min=1 max=20 num=100 radscale=linear counts=25 clob+ mode=h
dmimgadapt tricolor_hard_thresh.img tricolor_hard_flux_cone.img min=1 max=20 num=100 radscale=linear counts=25 clob+ mode=h
dmimg2jpg \
infile=tricolor_soft_flux_cone.img \
greenfile=tricolor_medium_flux_cone.img \
bluefile=tricolor_hard_flux_cone.img \
outfile=dmimg2jpg_exercise13_cone.jpg \
clob+
display < dmimg2jpg_exercise13_cone.jpg
Obtain overlapping multi-spectral images of this region from other archives. Use reproject_image to project the other datasets to the same tangent plane as the Chandra dataset. Create tri-color image using red for the lowest energy, and blue for the highest energy datasets.
Obtain the data for OBS_ID 635 and reprocess it with chandra_repro. Split the event file into 3 equal time intervals by filtering on the TIME column. Run fluximage on each time interval using the broad energy band. Use the 3 time-slices to create a 3 color image.
/bin/rm -rf 635
download_chandra_obsid 635 evt1,bpix,flt,mtl,msk,dtf,bias,pbk,asol,stat
download_obsid_caldb 635 ./CALDB
chandra_repro 635 out=
dmstat "635/repro/acisf00635_repro_evt2.fits[cols time]" cen- sig- med-
python -c 'a=72039166;b=72141142;dt3=(b-a)/3.0;print(a+0.0*dt3,a+1.0*dt3,a+2.0*dt3,a+3.0*dt3)'
dmcopy "635/repro/acisf00635_repro_evt2.fits[time=72039166.0:72073158.0]" 635/repro/begin_evt.fits clob+
dmcopy "635/repro/acisf00635_repro_evt2.fits[time=72073158.0:72107150.0]" 635/repro/middle_evt.fits clob+
dmcopy "635/repro/acisf00635_repro_evt2.fits[time=72107150.0:72141142.0]" 635/repro/end_evt.fits clob+
fluximage 635/repro/begin_evt.fits rhooph_begin bin=6 band=broad clob+ verb=0
fluximage 635/repro/middle_evt.fits rhooph_middle bin=6 band=broad clob+ verb=0
fluximage 635/repro/end_evt.fits rhooph_end bin=6 band=broad clob+ verb=0
dmimg2jpg \
infile=rhooph_begin_broad_thresh.img \
greenfile=rhooph_middle_broad_thresh.img \
bluefile=rhooph_end_broad_thresh.img \
outfile=rhooph_time_slice.jpg \
clob+
display < rhooph_time_slice.jpg
e=635/repro/acisf00635_repro_evt2.fits
f=635/repro/acisf00635_repro_fov1.fits
dmcopy "${e}[sky=region($f),energy=500:7000][bin sky=6,time=::#3]" 635_time_cube.fits clob+
ds9 -frame delete -scale log -rgbcube 635_time_cube.fits -saveimage ds9_ex16_timecube.png -quit
display < ds9_ex16_timecube.png
References:
In X-ray astronomy the term "light curve" is used to describe the flux of a source vs time.
chips
.chips> make_figure("out_bin100.lc[cols dt,count_rate]")
chips> print_window("out_bin100.png")
dmextract 13858/repro/acisf13858_repro_evt2.fits"[sky=circle(4104.5,4115.5,10),energy=500:7000][bin time=::100]" \
out=dme_100s.lc op=ltc1 \
bkg=13858/repro/acisf13858_repro_evt2.fits"[sky=region(ds9_bkg.reg),energy=500:7000]" \
mode=h clob+
cat << EOM > chips_ex17a.py
from pychips import *
from pychips.hlui import *
load_preferences()
make_figure("dme_100s.lc[cols dt,net_rate]")
print_window("out_bin100.png","export.clobber=True")
quit()
EOM
python chips_ex17a.py
display < out_bin100.png
for tt in 200 500 1000 1
do
dmextract 13858/repro/acisf13858_repro_evt2.fits"[sky=circle(4104.5,4115.5,10),energy=500:7000][bin time=::${tt}]" \
out=dme_${tt}s.lc op=ltc1 \
bkg=13858/repro/acisf13858_repro_evt2.fits"[sky=region(ds9_bkg.reg),energy=500:7000]" \
mode=h clob+
done
cat << EOM > chips_ex17b.py
from pychips import *
from pychips.hlui import *
load_preferences()
strip_chart(4)
set_current_plot("plot1")
add_curve("dme_200s.lc[cols dt,net_rate]")
set_current_plot("plot2")
add_curve("dme_500s.lc[cols dt,net_rate]")
set_current_plot("plot3")
add_curve("dme_1000s.lc[cols dt,net_rate]")
set_current_plot("plot4")
add_curve("dme_1s.lc[cols dt,net_rate]")
print_window("out_bin_strip.png","export.clobber=True")
quit()
EOM
python chips_ex17b.py
display < out_bin_strip.png
dmlist dme_100s.lc cols
dmlist 13858/repro/acisf13858_repro_evt2.fits blocks,subspace
Q: Why is there a gap at the beginning and at the end of the light curves with count_rate=0? outside of GTI but inside of TSTART-TSTOP
Q: Is the delta-time, dt, column actually a column in the dmextract output file? No, it's a coord on time column
Q: What can be learned from the light curve with 1 second time resolution? nothing
Q: What energy events were used to make these light curves? Does using an energy filter change things and if so how?
used energy=500:7000, if not then would have used all energies
Q: ACIS event files contain multiple good time intervals (one for each chip). Which GTI was use? The first one, which in this case is for CCD=7 which is correct
glvary 13858/repro/acisf13858_repro_evt2.fits"[sky=circle(4104.5,4115.5,10),energy=500:7000]" \
out=glvary_src.prob lc=glvary_src.lc clob+ mode=h
cat << EOM > chips_ex18a.py
from pychips import *
from pychips.hlui import *
load_preferences()
make_figure("glvary_src.lc[cols time,count_rate]")
print_window("glvary_src.png","export.clobber=True")
quit()
EOM
python chips_ex18a.py
display < glvary_src.png
glvary 13858/repro/acisf13858_repro_evt2.fits"[sky=region(ds9_bkg.reg),energy=500:7000]" \
out=glvary_bkg.prob lc=glvary_bkg.lc clob+ mode=h
cat << EOM > chips_ex18b.py
from pychips import *
from pychips.hlui import *
load_preferences()
make_figure("glvary_bkg.lc[cols time,count_rate]")
print_window("glvary_bkg.png","export.clobber=True")
quit()
EOM
python chips_ex18b.py
display < glvary_bkg.png
Q: Describe how the glvary lightcurves differ from the dmextract light curves.
it picked binning, no dt
column. No net-rate
Q: Could any variability in the source region be due to variability in the background? No variability, background is not variable either
Use the dither_region tool to compute the time resolved area fraction of the source and separately the background regions. Try using those as input to dmextract and glvary as exposure corrections. Compare to the results without those corrections.
punlearn ardlib
acis_set_ardlib 13858/repro/acisf13858_repro_bpix1.fits abs-
dither_region \
13858/repro/pcadf456520092N001_asol1.fits \
"circle(4104.5,4115.5,10)" \
dr_src.out \
wcs= 13858/repro/acisf13858_repro_evt2.fits \
mask= 13858/repro/acisf13858_000N001_msk1.fits \
mode=h clob+
dmlist dr_src.out cols
cat << EOM > chips_ex18_drout.py
from pychips import *
from pychips.hlui import *
load_preferences()
make_figure("dr_src.out[cols time,fracarea]")
print_window("dr_src.png","export.clobber=True")
quit()
EOM
python chips_ex18_drout.py
display < dr_src.png
dither_region \
13858/repro/pcadf456520092N001_asol1.fits \
"region(ds9_bkg.reg)" \
dr_bkg.out \
wcs= 13858/repro/acisf13858_repro_evt2.fits \
mask= 13858/repro/acisf13858_000N001_msk1.fits \
mode=h clob+ maxpix=10000
cat << EOM > chips_ex18_droutb.py
from pychips import *
from pychips.hlui import *
load_preferences()
make_figure("dr_bkg.out[cols time,fracarea]")
print_window("dr_bkg.png","export.clobber=True")
quit()
EOM
python chips_ex18_droutb.py
display < dr_bkg.png
The background is dither across bad columns/pixels, but only a tiny fraction of the area is lost.
Reference:
Background flares may influence results especially when working with large source regions such as supernovae remnants or clusters.
For an on-axis point source, users should carefully consider whether excluding time due to a background flare would be better than accepting the time in order to obtain higher signal to noise.
In this exercise, students will extract a light curve for a background, source-free region and use it to search for any evidence of a background flare.
the reprocessed Level 2 event file. 3. Use dmextract to create a light curve using the source free event file, use a CCD_ID filter to select events from chip you found in step 1. Use a bin-width of 259.28 seconds. 4. Use the deflare tool to search for background flares on the source chip using method=clean. 5. Use dmcopy to apply the deflare output good-time interval, GTI, file to the reprocessed level 2 event file.
punlearn dmcoords
dmcoords 13858/repro/acisf13858_repro_evt2.fits op=sky x=4104.5 y=4115.5 verb=0
pget dmcoords x y ra dec chip_id chipx chipy
python -c 'from region import *;wav=CXCRegion("acisf13858_wav.src");out=field()-wav;out.write("exclude.reg",fits=True, clobber=True)'
dmcopy 13858/repro/acisf13858_repro_evt2.fits"[sky=region(exclude.reg)]" 13858_evt.holes clob+
ds9 13858_evt.holes -scale log -bin factor 8 -saveimage png ds9_ex19.png -quit
display < ds9_ex19.png
dmextract 13858_evt.holes'[ccd_id=7][bin time=::259.28]' bkg_lc op=ltc1 clob+
cat << EOM > chips_ex19.py
from pychips import *
from pychips.hlui import *
load_preferences()
make_figure("bkg_lc[cols dt,count_rate]")
print_window("chips_ex19.png","export.clobber=True")
quit()
EOM
python chips_ex19.py
display < chips_ex19.png
deflare bkg_lc bkg_lc_flare method=clean save="bkg_lc_flare"
convert bkg_lc_flare.ps -trim bkg_lc_flare.png
display < bkg_lc_flare.png
dmcopy 13858/repro/acisf13858_repro_evt2.fits"[@bkg_lc_flare]" 13858_deflare.evt clob+
dmlist 13858/repro/acisf13858_repro_evt2.fits counts
dmlist 13858_deflare.evt counts
Q: What CCD_ID was found in step 1? Which tool(s) were used? chip 7,
dmcoords
Q: Was there any evidence of a background flare in this observation, on this chip? If so, describe the flare (when did it occur within the observation, duration, etc.): **Yes, a very short 1-time-bin duration fare in the 2nd half of the
observation **
Try different bin-widths when creating the light curve. Does it affect the results?
dmextract 13858_evt.holes'[ccd_id=7][bin time=::50]' bkg_lc_50s op=ltc1 clob+
deflare bkg_lc_50s bkg_lc_flare_50s method=clean save="bkg_lc_flare_50s" plot- verb=0
convert bkg_lc_flare_50s.ps -trim bkg_lc_flare50s.png
display < bkg_lc_flare50s.png
Try different deflare method options.
deflare bkg_lc_50s bkg_lc_sigmaclip_50s method=sigma save="bkg_lc_sigmaclip_50s" plot- verb=0 < /dev/null
convert bkg_lc_sigmaclip_50s.ps -trim bkg_lc_sigmaclip50s.png
display < bkg_lc_sigmaclip50s.png
Try looking for flares on the other CCD's used in this observation.
dmextract 13858_evt.holes'[ccd_id=6][bin time=::259]' bkg_6lc op=ltc1 clob+
deflare bkg_6lc bkg_6lc_flare method=clean save="bkg_6lc_flare" plot- verb=0
convert bkg_6lc_flare.ps -trim bkg_6lc_flare.png
display < bkg_6lc_flare.png
The energy resolution and sensitivity of the ACIS detector allows for spectral analysis roughly in the 0.3 to 7.0 keV range.
In this exercise Students will use the specextract tool to extract the spectrum, ARF, and RMF files for the source and background regions.
punlearn specextract
specextract 13858/repro/acisf13858_repro_evt2.fits"[sky=circle(4104.5,4115.5,10)]" \
out=specextract_01 \
bkgfile=13858/repro/acisf13858_repro_evt2.fits"[sky=region(ds9_bkg.reg)]" \
mode=h clob+
punlearn specextract
specextract 13858/repro/acisf13858_repro_evt2.fits"[sky=circle(4104.5,4115.5,10)]" \
out=specextract_02 \
bkgfile=13858/repro/acisf13858_repro_evt2.fits"[sky=region(ds9_bkg.reg)]" \
weight=no \
mode=h clob+
punlearn specextract
specextract 13858/repro/acisf13858_repro_evt2.fits"[sky=circle(4104.5,4115.5,10)]" \
out=specextract_03 \
bkgfile=13858/repro/acisf13858_repro_evt2.fits"[sky=region(ds9_bkg.reg)]" \
correctpsf=yes \
mode=h clob+
/bin/ls specextra*pi
dmdiff specextract_01.pi specextract_02.pi || echo
Few keyword diffs, data are the same
dmdiff specextract_01.pi specextract_03.pi || echo
Same, few keys, no data diffs.
dmdiff specextract_01.arf"[#row=1:10]" specextract_02.arf"[#row=1:10]" comment- || echo
dmstat specextract_01.arf"[cols specresp]" | egrep 'min|max'
dmstat specextract_02.arf"[cols specresp]" | egrep 'min|max'
dmstat specextract_03.arf"[cols specresp]" | egrep 'min|max'
dmlist specextract_01.pi,specextract_01_bkg.pi header,clean | egrep 'BACKSCAL|EXPOSURE'
Q: Record the BACKSCAL and EXPOSURE keywords. Why are these keywords important? Used to scale background to source
Q: Describe and discuss the differences, if any, from step 4. Smaller max by a little bit
Q: Describe and discuss the differences, if any, from step 5: None, warning 'corect=yes is not used when making weighted
Use the rmfimg tool to convert the RMF into an image and display in ds9.
rmfimg specextract_01.rmf specextract_01.rmf.img clob+
ds9 specextract_01.rmf.img -scale log -zoom to fit -view colorbar no -cmap heat \
-saveimage png ds9_exercise20.png -quit
display < ds9_exercise20.png
References
sherpa is the fitting and modeling package developed for CIAO. It allows users to obtain best fit model parameters based on a number of models, fitting methods, and statistics used for uncertainty.
The exercise shown here is an extremely simple sherpa use case. More advanced usage includes user supplied models, linked model parameters, simultaneously fitting multiple datasets, fitting images, etc.
In this exercise Students will begin to use sherpa to model and fit the spectrum of the source.
sherpa> load_data("source.pha")
sherpa> set_source(xswabs.abs1*xspowerlaw.p1)
Q: Which fit method works best?
Q: Which spectrum fits best?
Q: Which model/param is best?
Q: How does grouping affect the results?
Q: What are the advantages/disadvantages of one method over another?