What are Python generators good for?

One of best features in Python (IMHO) is the “generator” thing.

Here’s a simple example of a generator in action:

https://gist.github.com/ruliana/8ebf09ce59d4a094caa9c0807469cb09

The function produces values and handles it to a “for loop” as a collection. The obvious use is to avoid creating whole arrays and fill up memory.

I like them for another reason; they allow me to remove nesting. Recently I created a simple CSV from a query in Neo4J. The code goes like this:

https://gist.github.com/ruliana/d23bd5f430eaca7ddbceb10b594bd986

Not pretty, the CSV (consumer) and Neo4J (producer) logics are interleaved. Generators to the rescue:

https://gist.github.com/ruliana/be90dc4c0386042b4b14665431c805de

I find this better from a reuse perspective. The logic from producer and consumer are separated now. The producer function can generate records for any use. Also, I can use the write_to_csv function to write any collection of hash-like things to csv.

However, it’s a bit longer than the previous version and a little harder to understand, as the reader needs to know about generators. So, the second version is not good as throwaway code.

There is an important caveat to notice, also, if you don’t go over all records in the “cypher” generator, none of the Neo4J stuff will be closed, and you’ll get a connection leak for free. Be careful.