Over a million developers have joined DZone.

Blaze: A Python Compiler for Big Data

DZone's Guide to

Blaze: A Python Compiler for Big Data

· Big Data Zone ·
Free Resource

The Architect’s Guide to Big Data Application Performance. Get the Guide.

Python developers working with NumPy or Big Data in general might be interested in Blaze, a Python library created by Continuum Analytics and referred to by Stephen Diehl as "the next generation of NumPy." Blaze expands on NumPy's array structures by utilizing a variety of table and array-like structures and supporting a number of new features. According to Diehl:

...Blaze is designed to handle out-of-core computations on large datasets that exceed the system memory capacity, as well as on distributed and streaming data. Blaze is able to operate on datasets transparently as if they behaved like in-memory NumPy arrays.

We aim to allow analysts and scientists to productively write robust and efficient code, without getting bogged down in the details of how to distribute computation, or worse, how to transport and convert data between databases, formats, proprietary data warehouses, and other silos.

Basically, it looks like NumPy, but a bit more flexible and efficient. If you're looking for something different in the world of Python and Big Data, check out the GitHub and the docs.

Learn how taking a DataOps approach will help you speed up processes and increase data quality by providing streamlined analytics pipelines via automation and testing. Learn More.


Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}