Over a million developers have joined DZone.
Platinum Partner

PyPy Opens Up a "Blackhole"

The Performance Zone is brought to you in partnership with New Relic. Quickly learn how to use Docker and containers in general to create packaged images for easy management, testing, and deployment of software.

Over the weekend, the PyPy blog announced the release of PyPy 1.3, which features a much faster JIT compiler and support for C-coded CPython extension modules.  The JIT compiler in PyPy has also been stabilized and many user issues and bugs have been addressed.  As PyPy slithers into mainstream attention, the main theme of this release is stability.

While the vast majority of Python developers currently use CPython, which is downloaded at python.org, another Python implementation written in Python - PyPy - has gained wider support from developers.  Some see PyPy as an even better platform for Python's growth than CPython.  One of the main reasons why PyPy is so well suited for growth is because changes can be made to the language without having to update the JIT.  The JIT backend is abstracted so that it is almost totally separate from the interpreter.  You can maintain the speed of the JIT while making updates to the language, therefore you don't need to be an assembly expert to write new bytecode.  

CPython modules
The CPython extension modules (written in C) are the only feature that breaks with the stability theme.  This support is considered alpha quality because it's unlikely that unmodified C extensions will work out of box (due to missing functions or refcounting details).  This support is disabled by default, so to turn it on you need to type:
import cpyext
Then you can try to import a .so file.  The libraries are source-compatible but not binary-compatible, so you'll need to recompile binaries using this, for example (details will vary depending on your build system):
pypy setup.py build
Be sure to include the above line at the beginning of setup.py or put it in your PYTHONSTARTUP.

The Blackhole Compiler
The secret behind the JIT compiler's speedup starts with the fact that its executable contains three interpreters.  They include a regular Python interpreter, for code that can't be JIT-compiled, a tracing interpreter, for JIT-compiling the most executed loops, and the "blackhole" interpreter.  The speed of compilation has increased substantially thanks to a redesigning of the blackhole interpreter as a time-critical component.  It is designed to finished the execution of the current Python opcode and it essentially works like the JIT compiler, but without the overhead of tracing (it doesn't need to box all values).  This post explains the blackhole compiler more thoroughly.  

PyPy has a number of benchmarks showing its speed compared to CPython.  PyPy is also able to automatically generate C code along with JVM and .NET versions of the interpreter (no need to write in Jython or IronPython).  See what PyPy projects are currently in progress during Google's Summer of Code.

The Performance Zone is brought to you in partnership with New Relic. Read more about providing a framework that gets you started on the right path to move your IT services to cloud computing, and give you an understanding as to why certain applications should not move to the cloud.


{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}