# Enter your name and student ID
name =
ID =
In this assignment, we will go through the Bayesian model design cycle:
In questions 1 and 2, you will build a simple model, fit it to data and evaluate its performance on future data. You will find that its performance is not great. In question 3, you will improve the model in multiple ways. Finally, in question 4, you will do model selection based on free energy.
The final questions will require knowledge from the last probabilistic programming session. But questions 1 and 2 can be done relatively early in the course.
using Pkg
Pkg.activate(".")
Pkg.instantiate();
using CSV
using DataFrames
using LinearAlgebra
using ProgressMeter
using RxInfer
using Plots
default(label="",
grid=false,
linewidth=3,
markersize=4,
guidefontsize=12,
margins=15Plots.pt)
Many Europeans suspect that the air quality in their city is declining. A recent study measured the air quality of a major city in North Italy using an electronic nose. The measurements were made in the middle of the city and reflect urban activity. We will inspect the specific chemical concentrations found and build a model to accurately predict CO for future time points.
Photograph taken by Claudio Furlan/LaPresse/Zuma Press/Rex/Shutterstock (link)
The data can be found here: https://archive.ics.uci.edu/ml/datasets/Air+Quality. I've done some pre-processing and selected the most important features. In this assignment we will infer parameters in a model of the data and predict air quality in the future. For that purpose, the data has been split into past and future.
# Load training data
past_data = DataFrame(CSV.File("data/airquality_past.csv"))
Let's visualize the carbon monoxide measurements over time.
scatter(past_data[:,1],
past_data[:,2],
size=(900,300),
color="black",
xlabel="time",
ylabel="CO (ppm)",
ylims=[400,2000])
We suspect that there is a temporal dependence in this dataset. In other words, the data changes relatively slowly over time and neighbouring data points end up being highly correlated. To exploit this correlation, we will build an auto-regressive model of the form:
$$ y_k = \theta y_{k-1} + \epsilon_k \, , $$where the noise $\epsilon_k$ is drawn from a zero-mean Gaussian with precision parameter $\tau$:
$$ \epsilon_k \sim \mathcal{N}(0, \tau^{-1}) \, .$$Tasks:
### YOUR CODE HERE
We want to evaluate the parameters inferred under the model. For now, we will do this by visually inspecting the 1-step ahead predictions on our data set. Later, we will use free energy as a metric.
The posterior predictive distribution for the next time step is:
$$ p(y_{t+1} \mid y_{t}, \mathcal{D}) = \int p(y_{t+1} \mid \theta, y_{t}) p(\theta \mid \mathcal{D}) \, \mathrm{d}\theta \, , $$where $\mathcal{D}$ refers to the data used to infer the posterior distribution. To make 1-step ahead predictions, you will have to loop over the data (i.e., for t in 1:T
), plug in the current data point and compute the parameters of the posterior predictive distribution for the next data point. You may start from $t=2$, using $y_1$ as initial "previous observation".
Tasks:
ribbon=
) along with $y_{2:11}$ (scatterplot).Note that if you failed to infer a posterior distribution in the previous question, you can still answer this question using a standard normal, $p(\theta) = \mathcal{N}(0,1)$.
### YOUR CODE HERE
From the results of the previous question, you may conclude that our initial model isn't great: it only considers extremely short-term changes, which are highly affected by noise.
The model can be improved in two ways:
where $M$ corresponds to model order.
Tasks:
# Number of iterations of variational inference
n_iters = 10;
# Model order
M = 5;
### YOUR CODE HERE
We now essentially have a different model for each value of $M$. Which is the best?
Tasks:
# Model order range
model_orders = [2,4,8,16,32];
### YOUR CODE HERE
# Load test data
future_data = DataFrame(CSV.File("data/airquality_future.csv"))
### YOUR CODE HERE
Before you submit the assignment, make sure your notebook runs correctly! You can do that by going to the Kernel
tab in the toolbar and pressing Restart & Run All
.
This is important! If your code doesn't run, we can't verify the correctness of your answer.
When you're ready, head on over to Canvas and upload your notebook.