The Azul modifications in the OpenJDK and Linux kernel will allow Java runtimes to break through OS restrictions and take advantage of advanced chip architectures. "I’m excited about the Managed Runtime Initiative and the contribution Azul is making to the Open Source community,” said Gosling. “Managed runtimes have come a long way since the mid 90s. However, the rest of the systems stack has not evolved to meet the needs of these now pervasive application environments. This Initiative will serve to bring new functionality to the systems stack, freeing managed runtimes to continue their growth and evolution." With Azul's tech, you'll be able to add many more VMs on a single machine (hundreds more). Each VM will be able to draw around 300GB of memory with no specialized coding.
Why This Technology is SignificantAn increasing number of production servers today are running managed code, but the operating systems, hypervisors, and commodity hardware that run modern server loads are not optimally built for such jobs. Azul's Vega 3 compute appliance is optimally built for the Hot Spot JVM and running managed code. For a long time it has been one of the few technologies that end the pauses during garbage collection and the trade-offs between GC pauses and having more of a program stored in memory. Other techniques in enterprise Java have mitigated these problems as well, but they certainly aren't ideal compared to the Azul approach, which attacks the heart of the issue.
Today, Java shops can buy high-capacity servers for relatively low prices, but Azul Systems CEO Scott Sellers says that a given JVM instance can only tap into a tiny fraction of that memory. In an interview with DZone earlier this year, Sellers said, "The way that Java applications work today is a bit of a two-edged sword. When you get to the deployment side, this is when many of the problems associated with Java are seen," said Sellers. "For example, it's pretty much a given that when you're running on an off-the-shelf server, and you're trying to run a memory heap size that's beyond a couple of gigabytes, inevitably that application will get hammered by the internal garbage collection process of the Java runtime. Basically this means that at an architectural level, you have to architect your application to make sure that you never have any Java instances that are beyond 1-2 gigabytes in size."
There's a terrible disconnect between the capabilities of the hardware and what's possible given the limitations of the Java runtime. On general application servers, an application usually needs to grow its memory footprint. This might be because you need more state information to be held in the JRT for more users or you need to cache more information because, Sellers says, in today's three-tiered architectures you want to minimize accesses to the database. Sellers said that everyone today is faced with this limitation of a couple of gigabytes for each Java application instance. With Azul's appliance, Sellers says this isn't a problem. The garbage collection problem was mostly solved with direct support for read and write barriers built into the hardware, which allowed it to switch to a simpler GC algorithm that made concurrency and scalability easier. "We have customers that are running 300 to 400GB single-instance heaps, with worst case pause times measured in the tens-of-milliseconds," said Sellers.
Optimizations are made not just at the hardware level, but at the lower levels of Azul's JVM, Dr. Cliff, an expert in compiler technology who helped build Sun's Hot Spot JVM, has improved the Java server compiler's performance and instrumented code. Sellers explains that Azul's JVM has a tiered compilation system that balances the need for quick startup time and has excellent overall performance. Dr. Click has also done a lot of work that allows the JVM to scale much more effectively in terms of thread count.
"Today's systems typically are dealing with single digits to low double digit numbers of threads per JVM instance," said Sellers. "Our 14U [chassis] appliance has 864 individual processors in it," said Sellers. "So you can literally have a single JVM scale to 800-plus processors using 800-plus threads for a huge amount of horsepower within a single JVM instance." When you try to scale to that level on generic server hardware, then the classic limitations associated with multi-threading and serialization events around locks limit the scalability, as described by Amdahl's Law. Azul has developed a number of techniques and patents around speculating past locks and allowing parallel threads to continue to do work even in the presence of locks that would stop an off-the-shelf server.
Azul's MotivationAzul Systems is experiencing a period of exceptional growth, and to keep growing, they made an unprecedented decision to open source some of the integral patented technologies in their compute appliances. Their intention is to first of all, move vendors, partners, and ISVs toward more progressive technology for .NET, Windows, Ruby, Python, and other runtimes or platforms. Azul also wants vendors to support a commercial product that uses optimized Linux and OpenJDK.
For more information on the Managed Runtime Initiative, please visit www.managedruntime.org