Over a million developers have joined DZone.

Optimizing Complex Generator Expressions

DZone's Guide to

Optimizing Complex Generator Expressions

Need a quick rundown on optimizing complex generator expressions? Look no further!

· Big Data Zone
Free Resource

Need to build an application around your data? Learn more about dataflow programming for rapid development and greater creativity. 

Before diving in, check this out:   https://twitter.com/jakevdp/status/786920174595158018

The core expression is similar to this:

y = (f(x) for x in L if f(x) is not None)

There are a lot of variations on the filter. The point is that the function appears twice in the above expression.

We have a number of alternatives.
y = filter(None, f(x) for x in L) 

y = filter(None, map(f, L))

y = (x for x in map(f, L) if x)

y = (x for x in (f(y) for y in L) if x is not None)

y = (val for x in L for val in (f(x),) if val is not None)

My preference is two steps, even though I don't really have a good reason for this.
y1 = (f(x) for x in L)
y2 = (f for f in y1 if f)

The thread leads to this path:   https://twitter.com/TomAugspurger/status/786922167522828289  and the idea of "Let Bindings." We could extend the language slightly to bind a variable within the confines of the generator expression.

Like this:

y = (f(x) as val for x in L if val is not None)

The as clause binds the f(x) to val so that it can be used in the if clause.

Summary: Interesting.

Check out the Exaptive data application Studio. Technology agnostic. No glue code. Use what you know and rely on the community for what you don't. Try the community version.

filter ,number ,expression ,function ,point ,big data

Published at DZone with permission of Steven Lott, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.


Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.


{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}