JVM Advent Calendar: Project Loom
JVM Advent Calendar: Project Loom
Join the DZone community and get the full member experience.Join For Free
One of the drivers behind streams in Java 8 was concurrent programming.
You may also like: Project Loom: Java With a Stronger Fiber
In your stream pipeline, you specify what you want to have done, and your tasks are automatically distributed onto the available processors:
Parallel streams work great when the data structure is cheap to split into parts and the operations keep the processors busy. That's what it was designed for.
But this doesn't help you if your workload consists of tasks that mostly block. That's your typical web application, serving many requests, with each request spending much of its time waiting for the result of a REST service, a database query, and so on.
In 1998, it was amazing that the Sun Java Web Server (the precursor of Tomcat) ran each request in a separate thread, and not an OS process. It was able to serve thousands of concurrent requests this way! Nowadays, that's not so amazing. Each thread takes up a significant amount of memory, and you can't have millions of threads on a typical server.
That's why the modern mantra of server-side programming is: "Never block!" Instead, you specify what should happen once the data is available.
This asynchronous programming style is great for servers, allowing them to handily support millions of concurrent requests. It isn't so great for programmers.
Here is an asynchronous request with the
What we would normally achieve with statements is now encoded as method calls. If we loved this style of programming, we would not have statements in our programming language and merrily code in Lisp.
Project Loom takes its guidance from languages such as Erlang and Go, where blocking isn't a big deal. You run tasks in "fibers" or "lightweight threads" or "virtual threads". The name is up for discussion, but I prefer "fiber" since it nicely denotes the fact that multiple fibers execute in a carrier thread. Fibers are parked when a blocking operation occurs, such as waiting for a lock or for I/O. Parking is relatively cheap. A carrier thread can support a thousand fibers if each of them is parked much of the time.
Keep in mind that Project Loom does not solve all concurrency woes. It does nothing for you if you have computationally intensive tasks and want to keep all processor cores busy. It doesn't help you with user interfaces that use a single thread (for serializing access to data structures that aren't thread-safe). Keep using
Task for that usecase. Project Loom is useful when you have lots of tasks that spend much of their time blocking.
NB. If you have been around for a very long time, you may remember that early versions of Java had "green threads" that were mapped to OS threads. However, there is a crucial difference. When a green thread blocked, its carrier thread was also blocked, preventing all other green threads on the same carrier thread from making progress.
At this point, Project Loom is still very much exploratory. The API keeps changing, so be prepared to adapt to the latest API version when you try out the code after the holiday season.
You can download binaries of Project Loom at http://jdk.java.net/loom/, but they are updated infrequently. However, on a Linux machine or VM, it is easy to build the most current version yourself:
Depending on what you have already installed, you may have a couple of failures in
configure, but the messages tell you what packages you need to install so that you can proceed.
In the current version of the API, a fiber or, as it is called right now, virtual thread, is represented as an object of the
Thread class. Here are three ways of producing fibers. First, there is a new factory method that can construct OS threads or virtual threads:
However, manually creating threads has been considered poor practice for some time, so you probably shouldn't do either of these. Instead, use an executor with a thread factory:
Now, the familiar fixed thread pool will schedule virtual threads from the factory, in the same way as it has always done. Of course, there will also be OS-level carrier threads to run those virtual threads, but that's internal to the virtual thread implementation.
The fixed thread pool will limit the total number of concurrent virtual threads. By default, the mapping from virtual threads to carrier threads is done with a fork-join pool that uses as many cores as given by the system property
jdk.defaultScheduler.parallelism, or by default,
Runtime.getRuntime().availableProcessors(). You can supply your own scheduler in the thread factory:
I don't know if this is something that one would want to do. Why have more carrier threads than cores?
Back to our executor service. You execute tasks on virtual threads just like you used to execute tasks on OS-level threads:
As a simple test, we can just sleep in each task.
If you now set
1_000_000 and comment out the
.virtual() in the factory builder, the program will fail with an out of memory error. A million OS-level threads take a lot of memory. But with virtual threads, it works.
At least, it should work, and it did work for me with previous builds of Loom. Unfortunately, with the build I downloaded on December 5, I got a core dump. That has happened to me on and off as I experimented with Loom. Hopefully, it will be fixed by the time you try this.
Now you are ready to try something more complex. Heinz Kabutz recently presented a puzzler with a program that loaded thousands of Dilbert cartoon images. For each calendar day, there is a page such as https://dilbert.com/strip/2011-06-05. The program read those pages, located the URL of the cartoon image on each page, and loaded each image. It was a mess of completable futures, somewhat like:
With fibers, the code is much clearer:
Sure, each the call to
readPage blocks, but with fibers, we don't care.
Try this out with something you care about. Read a large number of web pages, process them, do more blocking reads, and enjoy the fact that blocking is cheap with fibers.
The initial motivation for Project Loom was to implement fibers, but earlier this year, the project embarked on an experimental API for structured concurrency. In this highly recommended article (from which the images below are taken), Nathaniel Smith proposes structured forms of concurrency. Here is his central argument. Launching a task in a new thread is really no better than programming with GOTO, i.e. harmful:
When multiple threads run without coordination, it's spaghetti code all over again. In the 1960s, structured programming replaced
goto with branches, loops, and functions:
Now the time has come for structured concurrency. When launching concurrent tasks, we should know, from reading the program text, when they have all finished.
That way, we can control the resources that the tasks use.
By summer 2019, Project Loom had an API to express structured concurrency. Unfortunately, that API is currently in tatters because of the more recent experiment in unifying the thread and fiber APIs, but you can try it with the prototype at http://jdk.java.net/loom/.
Here, we schedule a number of tasks:
scope.close() blocks until all fibers finish. Remember-blocking is not a problem with fibers. Once the scope is closed, you know for sure that the fibers have finished.
FiberScope is auto-closeable, so you can use a
But what if one of the tasks never finishes?
You can create a scope with a deadline (
Instant) or timeout (
All fibers that haven't finished by the deadline/timeout are canceled. How? Read on.
Cancellation has always been a pain in Java. By convention, you cancel a thread by interrupting it. If the thread is blocking, the blocking operation terminates with an
InterruptedException. Otherwise, the interrupted status flag is set. Getting the checks right is tedious. It is not helpful that the interrupted status can be reset, or that
InterruptedException is a checked exception.
Treatment of cancellation in
java.util.concurrent has been inconsistent. Consider
ExecutorService.invokeAny. If any task yields a result, the others are canceled. But
CompletableFuture.anyOf lets all tasks run to completion, even though their results will be ignored.
The Summer 2019 Project Loom API tackled cancellation. In that version, fibers have a
cancel operation, similar to
interrupt, but cancellation is irrevocable. The static
Fiber.cancelled method returns
true if the current fiber has been canceled.
When a scope times out, its fibers get canceled.
Cancelation can be controlled by the following options in the
CANCEL_AT_CLOSE: Closing scope cancels all scheduled fibers instead of blocking
PROPAGATE_CANCEL: If owning fiber is canceled, any newly scheduled fibers automatically canceled
IGNORE_CANCEL: Scheduled fibers can't be canceled
All these options are unset at the top level. The
IGNORE_CANCEL options are inherited from the parent scope.
As you can see, there was a fair amount of tweakability. We'll have to see what comes back when this issue is revisited. For structured concurrency, it must be automatic to cancel all fibers in the scope when the scope times out or is forcibly closed.
It came as a surprise to me that one of the pain points for the Project Loom implementors are
ThreadLocal variables, as well as more esoteric things-context class loaders,
AccessControlContext. I had no idea so much was riding along on threads.
If you have a data structure that isn't safe for concurrent access, you can sometimes use an instance per thread. The classic example is
SimpleDateFormat. Sure, you could keep constructing new formatter objects, but that's not efficient. So you want to share one. But a global
won't work. If two threads access it concurrently, the formatting can get mangled.
So, it makes sense to have one of them per thread:
To access an actual formatter, call:
The first time you call
get in a given thread, the lambda in the constructor is called. From then on, the get method returns the instance belonging to the current thread.
For threads, that is accepted practice. But do you really want to have a million instances when there are a million fibers?
This hasn't been an issue for me because it seems easier to use something threadsafe, like a
java.time formatter. But Project Loom has been pondering "scope local" objects-one those
FiberScope are reactivated.
Thread locals have also been used as an approximation for processor locality, in situations where there are about as many threads as processors. This could be supported with an API that actually models user intent.
State of the Project
Developers who want to use Project Loom are naturally preoccupied with the API which, as you have seen, is not settled. However, a lot of the implementation work is under the hood.
A crucial part is to enable parking of fibers when an operation blocks. This has been done for networking, so you can connect to web sites, databases and so on, within fibers. Parking when local file operations block is not currently supported.
In fact, reimplementations of these libraries are already in JDK 11, 12, and 13-a tribute to the utility of frequent releases.
Blocking on monitors (
synchronized blocks and methods) is not yet supported, but it needs to be eventually.
ReentrantLock is ok now.
If a fiber blocks in a native method, that will "pin" the thread, and none of its fibers will make progress. There is nothing that Project Loom can do about that.
Method.invoke needs more work to be supported.
Work on debugging and monitoring support is ongoing.
As already mentioned, stability is still an issue.
Most importantly, performance has a way to go. Parking and unparking fibers is not a free lunch. A section of the runtime stack needs to be replaced each time.
There has been a lot of progress in all these areas, so let's cycle back t what developers care about-the API. This is a really good time to look at Project Loom and think about how you want to use it.
Is it of value to you that the same class represents threads and fibers? Or would you prefer some of the baggage of
Thread to be chucked out? Do you buy into the promise of structured concurrency?
Take Project Loom out for a spin and see how it works with your applications and frameworks, and provide feedback for the intrepid development team!
Want to write for the Java Advent blog? We are looking for contributors to fill all 24 slots and would love to have your contribution! Contact the Java Advent Admin at firstname.lastname@example.org!
Published at DZone with permission of Cay Horstmann . See the original article here.
Opinions expressed by DZone contributors are their own.