Python Iterators

In this article, you will learn about Python Iterators and their implementation in Python program. You will also learn to build your own custom iterator.

Python Iterators: Introduction

Iterator simply is an object that can be iterated upon. It allows programmers to access or traverse through all the elements of the collection without any deeper understanding of its structure.

Python iterators implement iterator protocol which consists of two special methods __iter__() and __next__(). The __iter__() method returns an iterator object where as __next__() method returns the next element from the sequence.

We will discuss in detail about __iter__() and __next__() later in this article, but first, let’s discuss for loops and the underlying mechanism in for loop and how iterators are associated with it. This will help us gain a proper understanding of Python iterators.

How for loops actually work?

Let’s take a list and iterate through it.

x = ['Hey','there','Python','programmers']
for i in x:
   print(i)

Output

Hey
there
Python
programmers

That’s the simple program we already have learned to code.

But have you ever tried and dig deeper into the underlying mechanism behind such iteration.

So basically, the process of the for loop going through each element is called iteration and the object x through which the for loop is iterating is called iterable.

What actually is happening here?

Well, behind the scenes actually the loop is using a built-in function called __iter__() to go through all the elements one by one and the __next__() function is used for the next element in the collection.

Let’s use a Python built-in function dir() to find out all the associated attributes of the iterable x.

There is the __iter__() method working behind the scene for the iteration. But __iter__() alone cannot iterate through all items as we need to move to next item for iteration. So __next__() function is used for that.

So, here is how things actually work behind the iteration in for loop or any iterable in Python.

>>> obj = iter(x) #using iter function for x
>>> next(obj) #Iteration 1 using next function
'Hey'
>>> next(obj) #Iteration 2
'there'
>>> next(obj) #Iteration 3
'Python'
>>> next(obj) #Iteration 4
'programmers'
>>> next(obj) #Iteration 5
Traceback (most recent call last):
 ...
StopIteration

Note: obj is the iterator returned by the function __iter__().

So that’s the action behind the for loop in Python where special functions __iter__() and __next__() are internally called for iteration.

Notice in 5th iteration the __next__() function raises an exception called StopIteration because there is no any item left to iterate through. Hence the for loop ends there.

We can summarize above process in following points and picture.

iterable x has function __iter__() as we saw using dir()
__iter__() functions returns an iterator object called obj
using iterator obj and __next__() function we traverse through all the items in the list
once there are no items left to iterate through, the function __next__() raises an exception StopIteration and the iteration ends there.

Now that we have known about iterators in Python, let’s learn how to create our own Python iterator.

Creating our own Iterator in Python

Building our own Iterator is nothing different than what we explained above. We use the same __iter__() and __next__() functions.

But this time we will define these special functions inside a class as we need.

Example to create our own Python Iterator

Here is an example to build our own iterator to display odd number from 1 to the max number supplied as the argument.

class OddNum:
  """Class to implement iterator protocol"""

  def __init__(self, num = 0):
    self.num = num

  def __iter__(self):
    self.x = 1
    return self

  def __next__(self):
    if self.x <= self.num:
      odd_num = self.x
      self.x += 2
      return odd_num
    else:
      raise StopIteration

Now we can use directly use for loop or use __iter__() and __next__().

Using for loop

>>>for num in OddNum(10):
     print (num)

1
3
5
7
9

Using _iter__() and __next__()

>>> obj = OddNum(10)
>>> i = iter(obj)
>>> next(i)
1
>>> next(i)
3
>>> next(i)
5
>>> next(i)
7
>>> next(i)
9
>>> next(i)
Traceback (most recent call last):
 ...
StopIteration

Creating our own infinite iterator

An infinite iterator never stops itself because we don’t set limit in it and it does not raise an exception.

In infinite iterator, the user can impose a condition to stop the infinite iteration later in the program as per need using break statement and others.

Example of infinite iterator

Let’s again take the example we used above but this time we won’t set a max limit to display odd numbers. Instead, we will use a break condition to exit out of iteration in for loop.

class OddNum:
  """Class to implement iterator protocol"""

  def __init__(self, num = 0):
    self.num = num

  def __iter__(self):
    self.x = 1
    return self

  def __next__(self):
    odd_num = self.x
    self.x += 2
    return odd_num

for i in OddNum():
  if i < 16:
    print (i)
  else:
    break

As you can see in above program, we didn’t set a limit in __next__() function, instead, we used a condition in for loop later to prevent from infinite iterations and jump out of it.

So, the output is