after: https://www.tensorflow.org/versions/r0.12/get_started/index.html
Just to get a feeling for the basic code layout ...
import tensorflow as tf
import numpy as np
Create a dataset that is perfect for linear regression ;-)
# X has 20 rows and 2 columns
X_data = np.random.uniform(-10,10,(20,2))
X_data[:,1] = X_data[:,1] * 20 + 2
print(X_data.shape)
X_data
(20, 2)
array([[ -2.27427525e-01, -1.88098182e+02], [ 5.20758979e+00, 2.70275824e+00], [ -1.01162704e+00, -4.41401868e+00], [ -4.25334323e+00, -6.72107608e+01], [ -9.56918746e+00, 6.15759214e+01], [ -1.75206095e+00, -3.32513814e+01], [ -9.93876659e+00, -9.52794991e+00], [ 8.08328957e+00, 1.63652934e+02], [ 9.42527934e+00, -2.19506162e+01], [ 8.98195920e+00, 1.40256440e+02], [ -5.44693058e+00, 4.09508713e+01], [ 4.06423553e+00, -1.34027047e+02], [ -7.73125119e+00, -1.14416933e+02], [ 2.50624979e+00, -1.46120478e+02], [ -7.30325605e+00, -4.73212155e+01], [ -2.64987532e+00, 6.00638338e+01], [ -5.01904881e+00, 1.08949820e+02], [ -7.23765976e-02, 8.06592591e+01], [ -6.43722239e+00, -5.61475588e+01], [ 4.75940242e+00, -1.36225077e+02]])
# Our y happens to be perfectly correlated...
y_data = 30 + 3 * X_data[:,0] - X_data[:,1]
print(y_data.shape)
y_data
(20,)
array([ 217.41589985, 42.92001113, 31.37913754, 84.45073117, -60.28348374, 57.99519853, 9.71165013, -109.40306502, 80.22645418, -83.31056283, -27.29166308, 176.21975389, 121.22317906, 183.63922757, 55.41144738, -38.01345978, -94.00696616, -50.87638893, 66.83589158, 180.5032847 ])
Declare placeholders for the data:
X = tf.placeholder(tf.float64, [None, 2])
y = tf.placeholder(tf.float64, [None])
X, y
(<tf.Tensor 'Placeholder:0' shape=(?, 2) dtype=float64>, <tf.Tensor 'Placeholder_1:0' shape=(?,) dtype=float64>)
Declare a weight vector and the bias, a scalar:
# the weights will be randomly initialized to small values
W = tf.Variable(tf.truncated_normal([2,1], stddev=1,dtype='float64'))
b = tf.Variable(tf.zeros([1.0], dtype='float64'))
W.get_shape(), b.get_shape()
(TensorShape([Dimension(2), Dimension(1)]), TensorShape([Dimension(1)]))
If we now tried to evaluate W, we'd get an error, as we don't have created a Session yet:
W.eval()
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-6-c380b204c25c> in <module>() ----> 1 W.eval() /home/key/python/anaconda2/envs/tensorflow2/lib/python3.5/site-packages/tensorflow/python/ops/variables.py in eval(self, session) 443 A numpy `ndarray` with a copy of the value of this variable. 444 """ --> 445 return self._variable.eval(session=session) 446 447 def initialized_value(self): /home/key/python/anaconda2/envs/tensorflow2/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in eval(self, feed_dict, session) 557 558 """ --> 559 return _eval_using_default_session(self, feed_dict, self.graph, session) 560 561 /home/key/python/anaconda2/envs/tensorflow2/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in _eval_using_default_session(tensors, feed_dict, graph, session) 3745 session = get_default_session() 3746 if session is None: -> 3747 raise ValueError("Cannot evaluate tensor using `eval()`: No default " 3748 "session is registered. Use `with " 3749 "sess.as_default()` or pass an explicit session to " ValueError: Cannot evaluate tensor using `eval()`: No default session is registered. Use `with sess.as_default()` or pass an explicit session to `eval(session=sess)`
We need to define the equation for predicting y from X:
# Construct a linear model
y_hat = tf.add(tf.matmul(X, W), b)
y_hat
<tf.Tensor 'Add:0' shape=(?, 1) dtype=float64>
Now, we need to define our loss function, choosing MSE.
loss = tf.reduce_mean(tf.square(y - tf.squeeze(y_hat)))
loss
<tf.Tensor 'Mean:0' shape=<unknown> dtype=float64>
We need to choose an optimization method and a learning rate:
#optimizer = tf.train.GradientDescentOptimizer(1e-5)
optimizer = tf.train.AdamOptimizer(1e-2)
train = optimizer.minimize(loss)
Lastly, before executing the computation, define a graph node to initialize the variables:
init = tf.initialize_all_variables()
First, create a TensorFlow Session to execute the graph:
sess = tf.Session()
sess.run(init)
Let's have a look at the initialized weights vector and the bias:
W.eval(sess)
array([[ 0.85059504], [ 0.6539813 ]])
b.eval(sess)
array([ 0.])
Now, train the model:
for i in range(10000):
#for i in range(100000):
sess.run(train, feed_dict={X:X_data, y:y_data})
if i % 1000 == 0:
#if i % 1000 == 0:
print('Step {}: W = {}, b = {}, loss = {}'.format(i,
sess.run(W), sess.run(b), sess.run(loss, feed_dict={X:X_data, y:y_data})))
Step 0: W = [[ 0.84059504] [ 0.6439813 ]], b = [ 0.01], loss = 27866.63059475203 Step 1000: W = [[ 2.50593987] [-1.03109144]], b = [ 8.07585589], loss = 460.6994656903801 Step 2000: W = [[ 2.67625564] [-1.02035829]], b = [ 15.59202743], loss = 198.96517221566342 Step 3000: W = [[ 2.81728831] [-1.01148616]], b = [ 21.8586233], loss = 63.52821889884941 Step 4000: W = [[ 2.92038501] [-1.00500435]], b = [ 26.45060758], loss = 12.07477075081085 Step 5000: W = [[ 2.97831337] [-1.0013631 ]], b = [ 29.03301844], loss = 0.8962046788134147 Step 6000: W = [[ 2.99744354] [-1.00016068]], b = [ 29.88600818], loss = 0.012454266970805195 Step 7000: W = [[ 2.99992855] [-1.00000449]], b = [ 29.99681428], loss = 9.727156840375604e-06 Step 8000: W = [[ 2.99999983] [-1.00000001]], b = [ 29.99999251], loss = 5.379535438319684e-11 Step 9000: W = [[ 3.] [-1.]], b = [ 30.], loss = 4.4295179684599636e-20