In this article, you will learn about Python Iterators and their implementation in Python program. You will also learn to build your own custom iterator.
Python Iterators: Introduction
Iterator simply is an object that can be iterated upon. It allows programmers to access or traverse through all the elements of the collection without any deeper understanding of its structure.
Python iterators implement iterator protocol which consists of two special methods
__iter__() method returns an iterator object where as
__next__() method returns the next element from the sequence.
We will discuss in detail about
__next__() later in this article, but first, let’s discuss for loops and the underlying mechanism in for loop and how iterators are associated with it. This will help us gain a proper understanding of Python iterators.
How for loops actually work?
Let’s take a list and iterate through it.
x = ['Hey','there','Python','programmers'] for i in x: print(i)
Hey there Python programmers
That’s the simple program we already have learned to code.
But have you ever tried and dig deeper into the underlying mechanism behind such iteration.
So basically, the process of the for loop going through each element is called iteration and the object
x through which the for loop is iterating is called iterable.
What actually is happening here?
Well, behind the scenes actually the loop is using a built-in function called
__iter__() to go through all the elements one by one and the
__next__() function is used for the next element in the collection.
Let’s use a Python built-in function
dir() to find out all the associated attributes of the iterable x.
There is the
__iter__() method working behind the scene for the iteration. But
__iter__() alone cannot iterate through all items as we need to move to next item for iteration. So
__next__() function is used for that.
So, here is how things actually work behind the iteration in for loop or any iterable in Python.
>>> obj = iter(x) #using iter function for x >>> next(obj) #Iteration 1 using next function 'Hey' >>> next(obj) #Iteration 2 'there' >>> next(obj) #Iteration 3 'Python' >>> next(obj) #Iteration 4 'programmers' >>> next(obj) #Iteration 5 Traceback (most recent call last): ... StopIteration
obj is the iterator returned by the function
So that’s the action behind the for loop in Python where special functions
__next__() are internally called for iteration.
Notice in 5th iteration the
__next__() function raises an exception called StopIteration because there is no any item left to iterate through. Hence the for loop ends there.
We can summarize above process in following points and picture.
- iterable x has function
__iter__()as we saw using
__iter__()functions returns an iterator object called
- using iterator obj and
__next__()function we traverse through all the items in the list
- once there are no items left to iterate through, the function
__next__()raises an exception StopIteration and the iteration ends there.
Now that we have known about iterators in Python, let’s learn how to create our own Python iterator.
Creating our own Iterator in Python
Building our own Iterator is nothing different than what we explained above. We use the same
But this time we will define these special functions inside a class as we need.
Example to create our own Python Iterator
Here is an example to build our own iterator to display odd number from 1 to the max number supplied as the argument.
class OddNum: """Class to implement iterator protocol""" def __init__(self, num = 0): self.num = num def __iter__(self): self.x = 1 return self def __next__(self): if self.x <= self.num: odd_num = self.x self.x += 2 return odd_num else: raise StopIteration
Now we can use directly use for loop or use
Using for loop
>>>for num in OddNum(10): print (num) 1 3 5 7 9
Using _iter__() and __next__()
>>> obj = OddNum(10) >>> i = iter(obj) >>> next(i) 1 >>> next(i) 3 >>> next(i) 5 >>> next(i) 7 >>> next(i) 9 >>> next(i) Traceback (most recent call last): ... StopIteration
Creating our own infinite iterator
An infinite iterator never stops itself because we don’t set limit in it and it does not raise an exception.
In infinite iterator, the user can impose a condition to stop the infinite iteration later in the program as per need using break statement and others.
Example of infinite iterator
Let’s again take the example we used above but this time we won’t set a max limit to display odd numbers. Instead, we will use a break condition to exit out of iteration in for loop.
class OddNum: """Class to implement iterator protocol""" def __init__(self, num = 0): self.num = num def __iter__(self): self.x = 1 return self def __next__(self): odd_num = self.x self.x += 2 return odd_num for i in OddNum(): if i < 16: print (i) else: break
As you can see in above program, we didn’t set a limit in
__next__() function, instead, we used a condition in for loop later to prevent from infinite iterations and jump out of it.
So, the output is
1 3 5 7 9 11 13 15