Generators and Yield in Python

generators and yield: these two keywords give us great power when we are supposed to produce data that is huge or infinite. The generators function provides us a way to declare a function that generates an iterator. The generators function doesn't have a return statement, but the yield statement does (more than one yield is possible). If the return statement gets placed inside the generator code, the execution will stop with an exception.

The beauty of generators is that all local variables and the execution start point are automatically saved between calls; this is because the generators function doesn't start execution at the beginning from the function for each call. Only the first call starts at the beginning of the function; the next call onwards, it resumes from the yield statement. In short, you can see generators as iterators where you can think of the next() method right there at yield.

Let's look at an example of an interactive shell:

def employee_generator():
    yield("Arun", "NewYork")
    yield("Tom", "Chicago")
    yield("San", "London")
    yield("Paul", "Delhi")

Output:

>>> empIterator=employee_generator()
>>> empIterator.next()
('Arun', 'NewYork')
>>> empIterator.next()
('Tom', 'Chicago')
>>> empIterator.next()
('San', 'London')
>>> empIterator.next()
('Paul', 'Delhi')
>>> empIterator.next()

Traceback (most recent call last):
  File "<pyshell#39>", line 1, in <module>
    empIterator.next()
StopIteration
>>> 

As you can see in the output, it's pretty much an iterator.

Now, let's look at the method level:

'''
Id:          "$Id$"
Description: This is to showcase the Generators and Yield behavior
Test:
'''
def listGenerator():
    '''
    Generator Function that yields data for a single value.
    '''
    print ("called 1st Line...")
    myList = list(range(11, 17))

    print ("called 2nd Line...")
    for val in myList:
        print ("called 3rd Line...")
        yield val
        print ("called 4th Line...")

    print ("called 5th Line...")


def main():
    listGen = listGenerator()
    for res in listGen:
        print (res)

Output:

### Run tests.test ###
called 1st Line...
called 2nd Line...
called 3rd Line...
11
called 4th Line...
called 3rd Line...
12
called 4th Line...
called 3rd Line...
13
called 4th Line...
called 3rd Line...
14
called 4th Line...
called 3rd Line...
15
called 4th Line...
called 3rd Line...
16
called 4th Line...
called 5th Line...
>>> 

As you can see in the output, from the second call onwards, it's directly calling yield. So if you have multiple tasks and you want to put them one by one in your program, generators and yield are there to help.

One good example is xrange, which acts as a generator.