If you’ve ever written any Python at all, the chances are you’ve used iterators without even realising it. Writing your own and using them in your programs can provide significant performance improvements, particularly when handling large datasets or running in an environment with limited resources. They can also make your code more elegant and give you “Pythonic” bragging rights.

Here we’ll walk through the details and show you how to roll your own, illustrating along the way just why they’re useful.

You’re probably familiar with looping over objects in Python using English-style syntax like this:

people = [['Sam', 19], ['Laura', 34], ['Jona', 23]]
for name, age in people:
    ...

info_file = open('info.txt')
for line in info_file:
    ...

hundred_squares = [x**2 for x in range(100)]

", ".join(["Punctuated", "by", "commas"])

These kind of statements are possible due to the magic of iterators. To explain the benefits of being able to write your own iterators, we first need to dive into some details and de-mystify what’s actually going on.

Iterators and Iterables

Iterators and iterables are two different concepts. The definitions seem finickity, but they’re well worth understanding as they will make everything else much easier, particularly when we get to the fun of generators. Stay with us!

Iterators

An iterator is an object which represents a stream of data. More precisely, an object that has a __next__ method. When you use a for-loop, list comprehension or anything else that iterates over an object, in the background the __next__ method is being called on an iterator.

Ok, so let’s make an example. All we have to do is create a class which implements __next__. Our iterator will just spit out multiples of a specified number.

class Multiple:
    def __init__(self, number):
        self.number = number
        self.counter = 0

    def __next__(self):
        self.counter += 1
        return self.number * self.counter

if __name__ == '__main__':
    m = Multiple(463)
    print(next(m))
    print(next(m))
    print(next(m))
    print(next(m))

When this code is run, it produces the following output:

$ python iterator_test.py
463
926
1389
1852

Let’s take a look at what’s going on. We made our own class and defined a __next__ method, which returns a new iteration every time it’s called. An iterator always has to keep a record of where it is in the sequence, which we do using self.counter. Instead of calling the object’s __next__ method, we called next on the object. This is the recommended way of doing things since it’s nicer to read as well as being more flexible.

Cool. But if we try to use this in a for-loop instead of calling next manually, we’ll discover something’s amiss.

if __name__ == '__main__':
    for number in Multiple(463):
        print(number)
$ python iterator_test.py
Traceback (most recent call last):
File "iterator_test.py", line 11, in <module>
for number in Multiple(463):
TypeError: 'Multiple' object ...read more

Source:: Hackaday