from conx import Network, Layer, LSTMLayer, plot, frange
import math
Using Theano backend. conx, version 3.5.5
For this experiment, we will use the monthly money spent by international airline passengers on tickets between 1949 and 1961:
scaled_data = [round(math.sin(x),1) for x in frange(0, 2 * math.pi + .1, .1)]
[x for x in enumerate(scaled_data) if -0.1 < x[1] < 0.1]
[(0, 0.0), (31, 0.0), (63, 0.0)]
plot(["Scaled Data", scaled_data], default_symbol="o")
scaled_data[0], scaled_data[-1]
(0.0, 0.0)
For our dataset, we will contruct a history sequence. First, we need to put each scaled dollar amount into a list. This is the list of features. In our case, we just have the one feature:
sequence = [[datum] for datum in scaled_data]
We wish that the inputs -> targets are constructed as follows:
where Sn is a list of features in the sequence.
We need to inform the network of the shape of the sequence. We need the:
time_steps = 10 # history
batch_size = 1 # how many to load at once
features = 1 # features (length of input vector)
def create_dataset(sequence, time_steps):
dataset = []
for i in range(len(sequence)-time_steps-1):
dataset.append([sequence[i:(i+time_steps)],
sequence[i + time_steps]])
return dataset
dataset = create_dataset(sequence, time_steps)
print(dataset[0])
print(dataset[1])
[[[0.0], [0.1], [0.2], [0.3], [0.4], [0.5], [0.6], [0.6], [0.7], [0.8]], [0.8]] [[[0.1], [0.2], [0.3], [0.4], [0.5], [0.6], [0.6], [0.7], [0.8], [0.8]], [0.9]]
Now we construct the network giving the batch_shape in terms of (look_back, banks, width):
net = Network("LSTM - sin")
net.add(Layer("input", features, batch_shape=(batch_size, time_steps, features)))
net.add(LSTMLayer("lstm", 4))
net.add(Layer("output", 1))
net.connect()
net.compile(error="mse", optimizer="adam")
net.dataset.clear()
net.dataset.load(dataset)
net.dashboard()
Failed to display Jupyter Widget of type Dashboard
.
If you're reading this message in the Jupyter Notebook or JupyterLab Notebook, it may mean that the widgets JavaScript is still loading. If this message persists, it likely means that the widgets JavaScript library is either not installed or not enabled. See the Jupyter Widgets Documentation for setup instructions.
If you're reading this message in another frontend (for example, a static rendering on GitHub or NBViewer), it may mean that your frontend doesn't currently support widgets.
net["lstm"].get_output_shape()
(1, 4)
#net.dataset.split(.33)
net.propagate([[.02]] * time_steps)
[0.009405624121427536]
net.reset()
outputs = [net.propagate(i) for i in net.dataset.inputs]
plot([["Network", outputs], ["Training data", net.dataset.targets]])
#net.reset();
net.delete()
if net.saved():
net.load()
net.plot_results()
else:
net.train(500, batch_size=batch_size, accuracy=1.0, tolerance=0.1,
shuffle=False, plot=True, save=True)
======================================================================== | Training | Training Epochs | Error | Accuracy ------ | --------- | --------- # 91 | 0.00156 | 1.00000 Saving network... Saved!
outputs = [net.propagate(i) for i in net.dataset.inputs]
plot([["Network", outputs], ["Training data", net.dataset.targets]])
net.propagate(net.dataset.inputs[0]), net.dataset.targets[0]
([0.8412674069404602], [0.800000011920929])