Notebook

Run this in Python kernel¶

In [ ]:

%%html
<style>
body {
    font-family: "Arial", cursive, sans-serif; # font style of application
}
pre {
    font-family: "Courier New"; # font style of cell outputs
}
</style>

Run below in the backtesting_clojure kernel¶

In [1]:

; import libraries from kernel
(ns clojure-backtesting.demo
  (:require [clojure.test :refer :all]
            [clojure-backtesting.data :refer :all]
            [clojure-backtesting.data-management :refer :all]
            [clojure-backtesting.order :refer :all]
            [clojure-backtesting.evaluate :refer :all]
            [clojure-backtesting.plot :refer :all]
            [clojure-backtesting.counter :refer :all]
            [clojure-backtesting.portfolio :refer :all]
            [clojure-backtesting.parameters :refer :all]
            [clojure-backtesting.large-data :refer :all]
            [clojure.string :as str]
            [clojure.pprint :as pprint]
            [java-time :as t]
            [clojure.java.io :as io]
            [clojure.data.csv :as csv]
            [clojupyter.kernel.version :as ver]
            [clojupyter.misc.helper :as helper]
  ) ;; require all libriaries from core
  (:use clojure.pprint)
)

Out[1]:

nil

Import dataset¶

Initialise portfolio （Go back here everytime you want to restart.）¶

In [2]:

; path to dataset = "../resources/CRSP-extract.csv"
; change it to the relative to your own dataset
;
(load-large-dataset "../../data-sorted-cleaned/data-CRSP-sorted-cleaned.csv" "main" add-aprc-by-date)
;; load compustat
(load-large-dataset "../../data-sorted-cleaned/data-Compustat-sorted-cleaned.csv" "compustat")
;; (def y (swap! dataset-col assoc "compustat" (read-csv-lazy "../../data-sorted-cleaned/data-Compustat-sorted-cleaned.csv")))
(set-main "main")
(init-portfolio "1963-02-05" 10)

Out[2]:

"1963-02-05"

In [40]:

;; (next-date)
;; (keys (deref available-tics) )
;; (get-date)
(get-line)
;;  (first (get (deref dataset-col) "compustat"))

Out[40]:

{:ewretd "-0.002846", :HSICCD "3679.0", :CUM-RET 0.1454830196005322, :CFACSHR "1.05", :date "1963-02-11", :INIT-PRICE 5.875, :OPENPRC "", :SECSTAT "R", :SHROUT "0.468", :TICKER "ACC", :APRC 6.7950138076418956, :COMNAM "ALLIED CENTRAL INC", :PRIMEXCH "A", :TRDSTAT "A", :FACSHR "", :HEXCD "2", :RET "0.030769", :EXCHCD "2", :CFACPR "1.05", :DLRET "", :PRC "8.375", :vwretd "-0.004841", :FACPR "", :CUSIP "1910510", :NCUSIP "", :PERMCO "23583", :DIVAMT "", :PERMNO "28871", :SHRCD "10", :sprtrn "-0.006196", :VOL "500.0", :SICCD "3679"}

In [42]:

;; (next-date)
;; (keys (deref available-tics) )
;; (get-date)
;; (get-line)
;;  (first (get (deref dataset-col) "compustat"))
(get (get (deref available-tics) "IBM") :reference)

Out[42]:

{:ewretd "0.001045", :HSICCD "7379.0", :CUM-RET 0.11367103002080017, :CFACSHR "76.875", :date "1963-02-08", :INIT-PRICE 350.0, :OPENPRC "", :SECSTAT "R", :SHROUT "27.678", :TICKER "IBM", :APRC 392.13422210161104, :COMNAM "INTERNATIONAL BUSINESS MACHS COR", :PRIMEXCH "N", :TRDSTAT "A", :FACSHR "", :HEXCD "1", :RET "0.002387", :EXCHCD "1", :CFACPR "77.200386", :DLRET "", :PRC "420.0", :vwretd "0.000528", :FACPR "", :CUSIP "45920010", :NCUSIP "", :PERMCO "20990", :DIVAMT "", :PERMNO "12490", :SHRCD "11", :sprtrn "0.0", :VOL "8000.0", :SICCD "3573"}

In [41]:

;; (count (deref available-tics))
;; (end-order)
;; (next-date)
(get-compustat-line (get (get (deref available-tics) "IBM") :reference) "compustat")
;; (get-line)
;; (first(get (deref dataset-col) "compustat"))
;; (first (rest (get (deref dataset-col) "compustat")))
;; (compare "1963-01-26" (get-date))
;; (get-date)

1962-02-08
1962-11-08

Out[41]:

{:fincfq "", :dpcy "", :fic "USA", :itccy "", :cshoq "", :dltisq "", :wcapchq "", :niq "59.213", :dpcq "", :sstkq "", :cik "51143.0", :oancfq "", :loq "", :pstkrq "", :oibdpq "", :ipodate "", :datafqtr "1962.5", :prstkcq "", :ivncfy "", :pstkq "", :saleq "467.7", :ltq "", :lctq "", :txdbq "", :dlcq "", :ppentq "", :dpq "", :atq "", :dvpq "", :wcapchy "", :capxy "", :itccq "", :xintq "", :datacqtr "1962.5", :dltrq "", :ibq "59.213", :actq "", :mibtq "", :xrdq "", :sstky "", :invtq "", :dvy "", :oiadpq "", :mibq "", :icaptq "", :txditcq "", :prccq "353.1442", :ceqq "", :sppey "", :dvq "", :seqq "", :fincfy "", :capxq "", :revtq "467.7", :oancfy "", :ivncfq "", :cusip "459200101", :dltisy "", :gvkey "6066", :addzip "10504", :dlttq "", :prstkcy "", :rdq "", :sic "7370.0", :xsgaq "", :exchg "11.0", :dltry "", :rectq "", :sppeq "", :cogsq "", :tic "IBM", :cheq "", :datadate "1962-09-30", :conm "INTL BUSINESS MACHINES CORP"}

In [3]:

(order-lazy "PPL" 10 :print true)

Out[3]:

{"PPL" {:date "1963-02-05", :expiration 3, :quantity 10, :remaining false, :leverage true, :print true, :direct true}}

In [4]:

(time (while 
    (< (compare (get-date) "1965-02") 0)
          ;(order-lazy "PPL" 10 :direct false)
          (update-eval-report (get-date))
          (next-date)))

362.5
Order: 1963-02-06 | PPL | 10.000000.
"Elapsed time: 23677.080628 msecs"

Out[4]:

nil

In [11]:

(time (while 
    (< (compare (curr-date) "2016-02") 0)(next-day)))

"Elapsed time: 916551.674745 msecs"

Out[11]:

nil

In [16]:

;; (take 20 (deref portfolio-value)) 
;; (view-portfolio-record 20)
(get-date)

Out[16]:

"1965-02-01"

Write a strategy¶

The following code implements a trading strategy called Golden Rule:

MA 50 cross above the MA 200 (golden cross)

MA 200 cross below the MA 50 (death cross)

So in the codes, MA50 and MA200 are compared on a daily basis, if golden cross occurs, then you set a buy order; if death cross occurs, then you set a sell order first

In [ ]:

(time (do (def MA50-vec-aapl [])
          (def MA200-vec-aapl [])
          (def MA50-vec-f [])
          (def MA200-vec-f [])
          (while (not= (get-date) "2016-12-29")
            (do
    ;; write your trading strategy here
              (def tics (deref available-tics)) ;20 ms
              (def MA50-vec-aapl (get-prev-n-days :PRC 50 "AAPL" MA50-vec-aapl (get (get tics "AAPL"):reference)))
              (def MA200-vec-aapl (get-prev-n-days :PRC 200 "AAPL" MA200-vec-aapl (get (get tics "AAPL") :reference)))
              (def MA50-vec-f (get-prev-n-days :PRC 50 "F" MA50-vec-f (get (get tics "F"):reference)))
              (def MA200-vec-f (get-prev-n-days :PRC 200 "F" MA200-vec-f (get (get tics "F") :reference)))
              (let [[MA50 MA200] [(moving-average :PRC MA50-vec-aapl) (moving-average :PRC MA200-vec-aapl)]]
                (if (> MA50 MA200)
                  (order "AAPL" 1 :reference (get (get tics "AAPL") :reference) :print false) 
                  (order "AAPL" 0 :remaining true :reference (get (get tics "AAPL") :reference))))
              (let [[MA50 MA200] [(moving-average :PRC MA50-vec-f) (moving-average :PRC MA200-vec-f)]]
                (if (> MA50 MA200)
                  (order "F" 1 :reference (get (get tics "F") :reference) :print false) 
                  (order "F" 0 :remaining true :reference (get (get tics "F") :reference))))
              ;(update-eval-report (get-date))
              (next-date)))))
(.close wrtr)

In [ ]:

(count (deref order-record))

Check portfolio record¶

In [ ]:

;; view final portfolio
(view-portfolio)

In [6]:

;; view portfolio value and return
(view-portfolio-record 10)

|      :date | :tot-value | :daily-ret | :tot-ret |   :loan | :leverage |
|------------+------------+------------+----------+---------+-----------|
| 1963-02-05 |        $10 |      0.00% |    0.00% |   $0.00 |     0.00% |
| 1963-02-06 |        $11 |     -5.83% |   -5.83% | $362.50 |  -103.24% |
| 1963-02-07 |         $7 |      7.46% |   12.58% | $362.50 |  -102.12% |
| 1963-02-08 |         $6 |     -5.07% |   20.09% | $362.50 |  -101.78% |
| 1963-02-11 |         $8 |      3.38% |    5.69% | $362.50 |  -102.48% |
| 1963-02-12 |         $6 |     -0.00% |   20.40% | $362.50 |  -101.76% |
| 1963-02-13 |         $7 |     -8.78% |   11.62% | $362.50 |  -102.16% |
| 1963-02-14 |         $9 |      3.05% |    0.74% | $362.50 |  -102.77% |
| 1963-02-15 |        $12 |     -5.47% |   -8.79% | $362.50 |  -103.45% |
| 1963-02-18 |         $9 |     -0.00% |    1.49% | $362.50 |  -102.72% |

Out[6]:

nil

Generate evaluation report¶

In [5]:

;; (update-eval-report (get-date))
(eval-report 30)

|      :date | :tot-value |    :vol |   :sharpe | :pnl-pt |
|------------+------------+---------+-----------+---------|
| 1963-02-06 |        $11 | 4.1193% |  -1.4142% |      $1 |
| 1963-02-07 |         $7 | 6.6608% |   1.8884% |     $-2 |
| 1963-02-08 |         $6 | 6.1197% |   3.2828% |     $-3 |
| 1963-02-11 |         $8 | 5.6290% |   1.0103% |     $-1 |
| 1963-02-12 |         $6 | 5.0347% |   4.0522% |     $-3 |
| 1963-02-13 |         $7 | 5.6675% |   2.0504% |     $-2 |
| 1963-02-14 |         $9 | 5.4642% |   0.1348% |      $0 |
| 1963-02-15 |        $12 | 5.3506% |  -1.6425% |      $2 |
| 1963-02-18 |         $9 | 5.0600% |   0.2939% |      $0 |
| 1963-02-19 |         $9 | 4.8539% |   0.8618% |      $0 |
| 1963-02-20 |         $8 | 4.8346% |   1.5895% |     $-1 |
| 1963-02-21 |        $11 | 4.6563% |  -1.4574% |      $1 |
| 1963-02-25 |        $11 | 4.7606% |  -1.5677% |      $1 |
| 1963-02-26 |        $11 | 4.6208% |  -1.4181% |      $1 |
| 1963-02-27 |         $9 | 4.4684% |   0.3075% |      $0 |
| 1963-02-28 |         $8 | 4.3302% |   1.7501% |     $-1 |
| 1963-03-01 |         $6 | 4.3743% |   4.4066% |     $-3 |
| 1963-03-04 |         $8 | 4.3166% |   1.3664% |     $-1 |
| 1963-03-05 |         $8 | 4.2156% |   1.2531% |     $-1 |
| 1963-03-06 |         $7 | 4.2215% |   2.9620% |     $-2 |
| 1963-03-07 |        $12 | 4.4807% |  -1.9882% |      $2 |
| 1963-03-08 |        $13 | 4.3927% |  -3.0921% |      $3 |
| 1963-03-11 |        $11 | 4.5966% |  -1.8214% |      $1 |
| 1963-03-12 |        $13 | 4.7277% |  -2.8832% |      $3 |
| 1963-03-13 |        $15 | 4.6342% |  -4.2193% |      $5 |
| 1963-03-14 |        $13 | 4.6527% |  -2.8265% |      $3 |
| 1963-03-15 |        $14 | 4.5777% |  -3.8109% |      $4 |
| 1963-03-18 |        $13 | 4.5245% |  -3.3641% |      $3 |
| 1963-03-19 |        $16 | 4.4940% |  -5.1476% |      $6 |
| 1963-03-20 |        $27 | 4.4197% | -10.6472% |     $17 |

Out[5]:

nil

Plot variables¶

In [7]:

(def data (deref portfolio-value))

Out[7]:

#'clojure-backtesting.demo/data

In [8]:

; Add legend name to series
(def data-to-plot
 (map #(assoc % :plot "portfolio")
  data))

Out[8]:

#'clojure-backtesting.demo/data-to-plot

In [9]:

(first data-to-plot)

Out[9]:

{:date "1963-02-05", :tot-value 10, :daily-ret 0.0, :tot-ret 0.0, :loan 0.0, :leverage 0.0, :plot "portfolio"}

In [10]:

(plot data-to-plot :plot :date :total-ret false)

Error printing return value at clojure.data.json/write-double (json.clj:368).
JSON error: cannot write Double NaN

            json.clj:   368 clojure.data.json$write_double/invokeStatic
            json.clj:   365 clojure.data.json$write_double/invoke
            json.clj:   286 clojure.data.json$fn__2762$G__2757__2769/invoke
            json.clj:   335 clojure.data.json$write_object/invokeStatic
            json.clj:   319 clojure.data.json$write_object/invoke
            json.clj:   286 clojure.data.json$fn__2762$G__2757__2769/invoke
            json.clj:   348 clojure.data.json$write_array/invokeStatic
            json.clj:   342 clojure.data.json$write_array/invoke
            json.clj:   286 clojure.data.json$fn__2762$G__2757__2769/invoke
            json.clj:   335 clojure.data.json$write_object/invokeStatic
            json.clj:   319 clojure.data.json$write_object/invoke
            json.clj:   286 clojure.data.json$fn__2762$G__2757__2769/invoke
            json.clj:   335 clojure.data.json$write_object/invokeStatic
            json.clj:   319 clojure.data.json$write_object/invoke
            json.clj:   286 clojure.data.json$fn__2762$G__2757__2769/invoke
            json.clj:   475 clojure.data.json$write/invokeStatic
            json.clj:   424 clojure.data.json$write/doInvoke
         RestFn.java:   425 clojure.lang.RestFn/invoke
            AFn.java:   156 clojure.lang.AFn/applyToHelper
         RestFn.java:   132 clojure.lang.RestFn/applyTo
            core.clj:   669 clojure.core$apply/invokeStatic
            core.clj:   660 clojure.core$apply/invoke
            json.clj:   482 clojure.data.json$write_str/invokeStatic
            json.clj:   477 clojure.data.json$write_str/doInvoke
         RestFn.java:   410 clojure.lang.RestFn/invoke
      clojupyter.clj:    50 oz.notebook.clojupyter$fn__24566$live_embed__24569/invoke
      clojupyter.clj:    90 oz.notebook.clojupyter$fn__24566$embed__24579/invoke
      clojupyter.clj:   110 oz.notebook.clojupyter$fn__24566$view_BANG_$reify__24586/to_mime
nrepl_middleware.clj:    20 clojupyter.kernel.nrepl_middleware$mime_values$fn$reify__23032/send
            main.clj:   442 clojure.main$repl$read_eval_print__9086/invoke
            main.clj:   458 clojure.main$repl$fn__9095/invoke
            main.clj:   368 clojure.main$repl/doInvoke
         RestFn.java:  1523 clojure.lang.RestFn/invoke
            AFn.java:    22 clojure.lang.AFn/run
            AFn.java:    22 clojure.lang.AFn/run
         Thread.java:   832 java.lang.Thread/run

In [ ]: