Over a million developers have joined DZone.

Optimizing Complex Generator Expressions

DZone's Guide to

Optimizing Complex Generator Expressions

Need a quick rundown on optimizing complex generator expressions? Look no further!

· Big Data Zone ·
Free Resource

How to Simplify Apache Kafka. Get eBook.

Before diving in, check this out:   https://twitter.com/jakevdp/status/786920174595158018

The core expression is similar to this:

y = (f(x) for x in L if f(x) is not None)

There are a lot of variations on the filter. The point is that the function appears twice in the above expression.

We have a number of alternatives.
y = filter(None, f(x) for x in L) 

y = filter(None, map(f, L))

y = (x for x in map(f, L) if x)

y = (x for x in (f(y) for y in L) if x is not None)

y = (val for x in L for val in (f(x),) if val is not None)

My preference is two steps, even though I don't really have a good reason for this.
y1 = (f(x) for x in L)
y2 = (f for f in y1 if f)

The thread leads to this path:   https://twitter.com/TomAugspurger/status/786922167522828289  and the idea of "Let Bindings." We could extend the language slightly to bind a variable within the confines of the generator expression.

Like this:

y = (f(x) as val for x in L if val is not None)

The as clause binds the f(x) to val so that it can be used in the if clause.

Summary: Interesting.

12 Best Practices for Modern Data Ingestion. Download White Paper.

filter ,number ,expression ,function ,point ,big data

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}