It is important that you test your submission files before submitting them. All you have to do to make a submission is create or modify the Sumbission
class the file submission.py
in the starting_kit/example_submission/
directory, then run this test to make sure everything works fine. This is the actual program that will be run on the server to test your submission.
take note that on codalab, your local directory is program/
. Then if you want to load the file model.dump
run : open("program/model.dump")
even if the file is in at the root of your submission dir.
import sys
import os
import logging
import warnings
import json
from sys import path
import numpy as np
import pandas as pd
# Uncomment the next lines to auto-reload libraries (this causes some problem with pickles in Python 3)
%load_ext autoreload
%autoreload 2
%matplotlib inline
import matplotlib.pyplot as plt
import grid2op
warnings.simplefilter(action='ignore', category=FutureWarning)
Useful paths for the submission and for data
this_dir = os.getcwd()
above_dir, final_dir = os.path.split(this_dir)
above_above_dir, secondfinal_dir = os.path.split(above_dir)
# the first step tries to determine from where the jupyter notebook was started, and adapt the path accordingly
if final_dir == "starting_kit":
# path where "submission.py" should be
model_dir = 'agent_path'
# location of some utilities script to mimic codalab behaviour
utility_dir = "../l2rpn2019_utils"
utility_dir = os.path.abspath(utility_dir)
# training data set
# if you didn't we encourage you to download the training data
path_train_set = '../data/data_l2rpn_2019'
path_train_set = os.path.abspath(path_train_set)
# path where the final results and temporary data will be located
output_dir = 'output_notebook_2DevelopAndRunLocally'
tmp_outdir = os.path.join(output_dir, "tmp_results")
if not os.path.exists(output_dir):
print("Creating path \"{}\" where the output of this notebook will be stored".format(output_dir))
os.mkdir(output_dir)
if not os.path.exists(tmp_outdir):
print("Creating path \"{}\" where the temporary output will be stored".format(tmp_outdir))
os.mkdir(tmp_outdir)
else:
raise RuntimeError("For this notebook to work properly, you should not have modified the github clone "
"where you found the l2rpn_2019 utilities, starting kit and other materials")
The command bellow simulate the execution of your code on the training dataset
# codalab command line used on the competition server. So it is best to test it as it is,
# rather than directly importing the related python method here and execute it.
!$sys.executable $utility_dir/ingestion.py $path_train_set $tmp_outdir $utility_dir $model_dir
Done and data saved in : "/home/benjamin/Documents/grid2op_dev/l2rpn_2019/starting_kit/output_notebook_2DevelopAndRunLocally/tmp_results/saved_experiment"
!$sys.executable $utility_dir/evaluate.py $tmp_outdir $output_dir
# print("watch : http:/view/"+ scoring_output_dir +"/scores.html")
input_dir: output_notebook_2DevelopAndRunLocally/tmp_results output_dir: output_notebook_2DevelopAndRunLocally Score for scenario 0001: 1595.0512709067282 Score for scenario 0000: 1542.1342069852371 Your submission is valid, you may proceed with the next steps
Zip the contents of sample_code_submission/
(without the directory), or download the challenge public_data and run the command in the previous cell, after replacing sample_data by public_data.
Then zip the contents of sample_result_submission/
(without the directory).
Do NOT zip the data with your submissions.
import datetime
import zipfile
def zipdir(path, ziph):
# ziph is zipfile handle
for root, dirs, files in os.walk(path):
for file in files:
ziph.write(os.path.join(root, file))
the_date = datetime.datetime.now().strftime("%y%m%d_%H%M")
sample_code_submission = 'sample_code_submission_' + the_date + '.zip'
with zipfile.ZipFile(sample_code_submission, 'w', zipfile.ZIP_DEFLATED) as zipf:
zipdir(model_dir,zipf)
print("Submit this file:\n\t- {}\n\t- Located at{}".format(sample_code_submission, os.path.abspath(".")))