Getting Started With TensorFlow: Writing Your First Program
Learn how to implement a very basic program in TensorFlow using Python by building and running a computational graph.
Join the DZone community and get the full member experience.
Join For FreeIn my previous article, we saw what TensorFlow is and learned some of its terminologies. In this article, we are going to go ahead and implement a very basic program in TensorFlow using Python to see it in action.
The computation in TensorFlow consists of two stages:
 Building the computational graph.
 Running the computational graph.
Computational graphs are nothing but the data flow graphs that I mentioned in my previous article. Each node of the data flow graph will represent an operation that will contribute toward evaluating the TensorFlow computation; hence, computational graph. In TensorFlow, each node takes zero or more tensors as inputs and produces a tensor as an output.
One type of node is constant, which takes no input, and outputs a value it stores internally. Let's see how to define a constant in TensorFlow.
constantValue1 = tf.constant(9.0, dtype=tf.float32)
constantValue2 = tf.constant(19.0)
print("constantValue1 = %s" % constantValue1)
print("constantValue2 = %s" % constantValue2)
The outputs of print statements will be:
constantValue1 = Tensor("Const:0", shape=(), dtype=float32)
constantValue2 = Tensor("Const_1:0", shape=(), dtype=float32)
Notice that the output wasn't 9.0 or 19.0 but Tensor objects. This is because we just built the computational graph but did not run it. Before running it, let's see what the above output means.
In the Tensor object, the first parameter is the name for that tensor. The Const
part of the name is assigned to it by the TensorFlow itself and is not explicitly given by the programmer. The name generated is then followed by a :
, which is followed by a number (0, in this case). This number is the index of that tensor that is being named.
What I mean by that is, a node can produce multiple outputs or multiple tensors as output. In that case, this number would be the index of each of the tensor in output. Here, though, there is only one output, so the tensor gets assigned 0. If there were one more output, that tensor would have been assigned 1.
The second parameter signifies the shape of that tensor. I have already talked about the shape of tensors in my previous blog. The third type is the data type of that tensor. You can either explicitly give it, as done for the first constant, or TensorFlow can also infer it, as done for the second constant.
If we want to see 9.0 and 19.0 as output, we will have to actually run the computational graph we just built. To do that, we will have to create a Session
object and invoke its run
method. We can do that as shown below:
sess = tf.Session()
print(sess.run(constantValue1))
print(sess.run(constantValue2))
The output of the above code will be 9.0
and 19.0
.
Now, let's add these two constants. Adding is an operation, and an operation is just another node in TensorFlow.
addConstants = constantValue1 + constantValue2
print("addConstants = ", addConstants)
sumOfConstants = sess.run(addConstants)
print("sum = ", sumOfConstants)
The output of the above code is:
addConstants = Tensor("add:0", shape=(), dtype=float32)
sum = 28.0
Here, +
is just a shorthand for tf.add()
.
Now, how do we supply our own values to TensorFlow? For these purposes, placeholder
comes into the picture. A placeholder
is a promise to provide a value later. Let's quickly create two placeholders and perform an operation on them to see them in action.
myValue1 = tf.placeholder(dtype=tf.float32)
myValue2 = tf.placeholder(dtype=tf.float32)
sumOfMyValuesNode = myValue1 + myValue2
sumOfMyValues = sess.run(sumOfMyValuesNode, {myValue1: 5.0, myValue2: 6.0})
print("Sum of myValues = ", sumOfMyValues)
Here, myValue1
and myValue2
both are placeholders whose value will be supplied later. Notice here that giving the data type is compulsory (dtype
). The values to the placeholder can be supplied when the run
method of the session object is invoked, as shown in the above example. The values are supplied in the feed_dict
argument of the run
method. So, the output of the above code is:
Sum of myValues = 11.0
But the whole point of machine learning is to make our data trainable so that we can train it, optimize it based on the training results, and achieve a model that can work almost perfectly on the real data.
So, how do we make our data trainable in TensorFlow? For this purpose, Variables
comes to our rescue. Variables
allow us to add trainable parameters to our program. Variables
can be defined as follows:
myVariable = tf.Variable(2.0, dtype=tf.float32)
Evey variable is initialized with some value (2.0, in this case) and giving a data type is optional. But the variable is only defined using the above way; it is not yet initialized. Variables are not initialized when you call tf.Variable
. To initialize all the variables in a TensorFlow program, you must explicitly call a special operation, as follows:
init = tf.global_variables_initializer()
sess.run(init)
It is important to realize that init
is a handle to the TensorFlow subgraph that initializes all the global variables. Until we call sess.run
, the variables are uninitialized.
print("myVariable = ", sess.run(myVariable))
This prints out myVariable = 2.0
. And if we want to change the value of our variable, we can use the assign
function, as shown below:
sess.run(tf.assign(myVariable, 10.0))
print(sess.run(myVariable))
(This prints 10.0
as output.)
OK, so now that we are clear with the basic terms of writing a TensorFlow program, we will take a very easy example and implement it. We will implement the following model:
y = W * x
We will provide our program with some training data, i.e. some values of x and desired values of y for that x, calculate the value of W on the basis of the training data, and then provide test data to see how accurate the results are on test data. Since we have taken a very simple model, our accuracy can easily reach 100%. But this almost never happens in real and more complex models. But for understanding purposes, this will do.
Since we will supply the values for x and y, we will declare them as placeholders. And since the value of W will have to be changed for every input, we will declare it as a variable with some initial value; let's say 1. Declarations will go something like this:
W = tf.Variable(1, dtype=tf.float32)
x = tf.placeholder(tf.float32)
y = tf.placeholder(tf.float32)
Now, we will define our simple model as below:
myModel = W * x
Now, to train the data and get closer to the real model, we will have to write a loss function and then minimize it. To keep things simple, we will take the sum of squared errors as the loss function. The error is nothing but the difference between what the result was using our model and what the desired value (y) was.
We will square those errors for each of the inputs and add them. Below is the implementation of the same:
delta = myModel  y
squaredDelta = tf.square(delta)
loss = tf.reduce_sum(squaredDelta)
To keep things simple, we will make our own little optimizer based on the concept of gradient descent optimizer (if you don't know about it, don't worry; just keep reading) to correct the value of W and then test it on some test data.
So, what we will be doing is calculating the loss of our model, manipulating the value of W to minimize the loss, checking if the loss has decreased, and manipulating the value of W further based on the result of the loss. The code I've written for this optimizer is shown below:
oldLoss = sys.float_info.max
adding = 0
subtracting = 0
def addOne():
sess.run(tf.assign(W, sess.run(W) + 1.0))
def subtractOne():
sess.run(tf.assign(W, sess.run(W)  1.0))
while oldLoss > 0:
currentLoss = sess.run(loss, {x: [1, 2, 3, 4], y: [10, 20, 30, 40]})
if currentLoss == 0:
break
elif adding == 0 and subtracting == 0:
addOne()
adding = 1
elif adding == 1 and currentLoss <= oldLoss:
addOne()
adding = 1
subtracting = 0
elif adding == 1 and currentLoss >= oldLoss:
subtractOne()
adding = 0
subtracting = 1
elif subtracting == 1 and currentLoss <= oldLoss:
subtractOne()
subtracting = 1
adding = 0
elif subtracting == 1 and currentLoss >= oldLoss:
addOne()
subtracting = 0
adding = 1
oldLoss = currentLoss
Please keep in mind that we are certain here that our loss can reach 0 because we have used a simple model. For more complex models, the conditions can be changed appropriately.
In the above code, adding
and subtracting
are flags that are used to remember what operation was performed last (addition or subtraction). currentLoss
is a variable that stores the value of the loss function at the starting of the loop and oldLoss
is a variable that stores the value of the loss function at the end of the loop. These two variables are compared between the loop to check how the operation (addition or subtraction) affected the loss value, i.e. decreased or increased it. On the basis of that, further operations are performed. We are either decreasing the value of W by 1 or increasing it by 1. This is just a sample optimizer. Good optimizers are much more complex and efficient and many are already implemented in TensorFlow, which we will talk about in future blogs. This is just a sample optimizer that may not work perfectly but it's good enough to give you an idea of how TensorFlow is working, which was my main objective here. The code written above is very simple to understand once you go through it, and everything used in the code has been discussed in this article.
For input, we are giving [1, 2, 3, 4]
for x and [10, 20, 30, 40]
for y (desired value). So, as we can see, the value of W should be 10.0, which we have currently initialized to 1.0. Our model should use the training data supplied to it and convert W from 1.0 to 10.0 and use this W on the test data.
To run our program, we have to initialize the global variables, make a session object, and invoke its run
method on the global variables handle, like below:
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
OK. We are done. To check the value of W, we will put a print
statement at the end:
print(sess.run(W))
This should print 10.0 as output when run. This means that the value of W has been changed from 1.0 to 10.0. If we supply some other data to our model to check the value of y, then we should always get 10 times of whatever value we supply. I put three print statements after the code to check the outputs:
print(sess.run(myModel, {x: 27.0}))
print(sess.run(myModel, {x: 10.0}))
print(sess.run(myModel, {x: 80.0}))
And the outputs I received were:
270.0
100.0
800.0
As expected.
I hope I was able to introduce the concepts to you in an easy and yet understandable way. This was a very simple example, I encourage you to go ahead and examine with the example, play around with it, look into optimizers(Gradient Descent Optimizer would be a great start) and try to implement them in TensorFlow. Many optimizers have been implemented in TensorFlow about which I'll be discussing in my future blogs. For my next blog, I'll be using MNIST dataset of handwritten digits and recognize them using TensorFlow.
References
 TensorFlow homepage
 Introduction to Gradient Descent Algorithm Along Its Variant
 GitHub repository for the implemented program
I hope this blog turned out to be helpful for you!
This article was originally published on the Knoldus blog.
If you enjoyed this article and want to learn more about TensorFlow, check out this collection of tutorials and articles on all things TensorFlow.
Published at DZone with permission of Akshansh Jain, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Trending

13 Impressive Ways To Improve the Developerâ€™s Experience by Using AI

Insider Threats and Software Development: What You Should Know

How To Design Reliable IIoT Architecture

Hyperion Essbase Technical Functionality
Comments