Java

It looks like Java 21 is going to pose a strong challenge to Node JS! There are two massive performance enhancements in Java 21, and they address two of Java's often-criticized areas: Threads and blocking IO (somewhat fair criticism) and GC (relatively unfair criticism?) Major highlights of Java 21: Project Loom and virtual threads ZGC (upgraded) 1. Virtual Threads For the longest time, we have looked at non-blocking IO, async operations, and then Promises and Async/Await for orchestrating the async operations. So, we have had to deal with callbacks, and do things like Promises.all() or CompletableFuture.thenCompose() to join several async operations and process the results. More recently, Reactive frameworks have come into the picture to "compose" tasks as functional pipelines and then run them on thread pools or executors. The reactive functional programming is much better than "callback hell", and thus, we were forced to move to a functional programming model so that non-blocking/async can be done in an elegant way. Virtual Threads are bringing an end to callbacks and promises. Java team has been successful in providing an almost-drop-in-replacement for Threads with dirt-cheap Virtual Threads. So, even if you do the old Thread.sleep(5000) the virtual thread will detach instead of blocking. In terms of numbers, a regular laptop can do 2000 to 5000 threads whereas the same machine can do 1 Million + virtual threads. In fact, the official recommendation is to avoid the pooling of virtual threads. Every task is recommended to be run on a new virtual thread. Virtual threads support everything - sleep, wait, ThreadLocal, Locks, etc. Virtual Threads allow us to just write regular old iterative and "seemingly blocking" code, and let Java detach/attach real threads so that it becomes non-blocking and high-performance. However, we still need to wait for Library/Framework implementers like Apache Tomcat and Spring to move everything to Virtual Threads from native Threads. Once the frameworks complete the transition, all Java microservices/monoliths that use these upgraded frameworks will become non-blocking automatically. Take the example of some of the thread pools we encounter in our applications - Apache Tomcat NIO has 25 - 50 worker threads. Imagine NIO can have 50,000 virtual threads. Apache Camel listener usually has 10-20 threads. Imagine Camel can have 1000-2000 virtual threads. Of course, there are no more thread pools with virtual threads - so, they will just have unlimited 1000s of threads. This just about puts a full stop to "thread starvation" in Java. Just by upgrading to Frameworks / Libraries that fully take advantage of Java 21, all our Java microservices will become non-blocking simply with existing code. (Caveat: some operations like synchronized will block virtual threads also. However, if we replace them with virtual-threads-supported alternatives like Lock.lock(), then virtual threads will be able to detach and do other tasks till the lock is acquired. For this, a little bit of code change is needed from Library authors, and also in some cases in project code bases to get the benefits of virtual threads). 2. ZGC ZGC now supports Terabyte-size Java Heaps with permanent sub-millisecond pauses. No important caveats... it may use say 5-10% more memory or 5-10% slower allocation speed, but no more stop-the-world GC pauses and no more heap size limits. Together these two performance improvements are going to strengthen Java's position among programming languages. It may pause the rising dominance of Node JS and to some extent Reactive programming. Reactive/Functional programming may still be good for code-readability and for managing heavily event-driven applications, but we don't need reactive programming to do non-blocking IO in Java anymore.

When doing unit tests, you have probably found yourself in the situation of having to create objects over and over again. To do this, you must call the class constructor with the corresponding parameters. So far, nothing unusual, but most probably, there have been times when the values of some of these fields were irrelevant for testing or when you had to create nested "dummy" objects simply because they were mandatory in the constructor. All this has probably generated some frustration at some point and made you question whether you were doing it right or not; if that is really the way to do unit tests, then it would not be worth the effort. That is to say, typically, a test must have a clear objective. Therefore, it is expected that within the SUT (system under test) there are fields that really are the object of the test and, on the other hand, others are irrelevant. Let's take an example. Let's suppose that we have the class "Person" with the fields Name, Email, and Age. On the other hand, we want to do the unit tests of a service that, receiving a Person object, tells us if this one can travel for free by bus or not. We know that this calculation only depends on the age. Children under 14 years old travel for free. Therefore, in this case, the Name and Email fields are irrelevant. In this example, creating Person objects would not involve too much effort, but let's suppose that the fields of the Person class grow or nested objects start appearing: Address, Relatives (List of People), Phone List, etc. Now, there are several issues to consider: It is more laborious to create the objects. What happens when the constructor or the fields of the class change? When there are lists of objects, how many objects should I create? What values should I assign to the fields that do not influence the test? Is it good if the values are always the same, without any variability? Two well-known design patterns are usually used to solve this situation: Object Mother and Builder. In both cases, the idea is to have "helpers" that facilitate the creation of objects with the characteristics we need. Both approaches are widespread, are adequate, and favor the maintainability of the tests. However, they still do not resolve some issues: When changing the constructors, the code will stop compiling even if they are fields that do not affect the tests. When new fields appear, we must update the code that generates the objects for testing. Generating nested objects is still laborious. Mandatory and unused fields are hard coded and assigned by default, so the tests have no variability. One of the Java libraries that can solve these problems is "EasyRandom." Next, we will see details of its operation. What is EasyRandom? EasyRandom is a Java library that facilitates the generation of random data for unit and integration testing. The idea behind EasyRandom is to provide a simple way to create objects with random values that can be used in tests. Instead of manually defining values for each class attribute in each test, EasyRandom automates this process, automatically generating random data for each attribute. This library handles primitive data types, custom classes, collections, and other types of objects. It can also be configured to respect specific rules and data generation restrictions, making it quite flexible. Here is a basic example of how EasyRandom can be used to generate a random object: Java public class EasyRandomExample { public static void main(String[] args) { EasyRandom easyRandom = new EasyRandom(); Person randomPerson = easyRandom.nextObject(Person.class); System.out.println(randomPerson); } } In this example, Person is a dummy class, and easyRandom.nextObject(Person.class) generates an instance of Person with random values for its attributes. As can be seen, the generation of these objects does not depend on the class constructor, so the test code will continue to compile, even if there are changes in the SUT. This would solve one of the biggest problems in maintaining an automatic test suite. Why Is It Interesting? Using the EasyRandom library for testing your applications has several advantages: Simplified random data generation: It automates generating random data for your objects, saving you from writing repetitive code for each test. Facilitates unit and integration testing: By automatically generating test objects, you can focus on testing the code's behavior instead of worrying about manually creating test data. Data customization: Although it generates random data by default, EasyRandom also allows you to customize certain fields or attributes if necessary, allowing you to adjust the generation according to your needs. Reduced human error: Manual generation of test data can lead to errors, especially when dealing with many fields and combinations. EasyRandom helps minimize human errors by generating consistent random data. Simplified maintenance: If your class requirements change (new fields, types, etc.), you do not need to manually update your test data, as EasyRandom will generate them automatically. Improved readability: Using EasyRandom makes your tests cleaner and more readable since you do not need to define test values explicitly in each case. Faster test development: By reducing the time spent creating test objects, you can develop tests faster and more effectively. Ease of use: Adding this library to our Java projects is practically immediate, and it is extremely easy to use. Where Can You Apply It? This library will allow us to simplify the creation of objects for our unit tests, but it can also be of great help when we need to generate a set of test data. This can be achieved by using the DTOs of our application and generating random objects to later dump them into a database or file. Where it is not recommended: this library may not be worthwhile in projects where object generation is not complex or where we need precise control over all the fields of the objects involved in the test. How To Use EasyRandom Let's see EasyRandom in action with a real example, environment used, and prerequisites. Prerequisites Java 8+ Maven or Gradle Initial Setup Inside our project, we must add a new dependency. The pom.xml file would look like this: XML <dependency> <groupId>org.jeasy</groupId> <artifactId>easy-random-core</artifactId> <version>5.0.0</version> </dependency> Basic Use Case The most basic use case has already been seen before. In this example, values are assigned to the fields of the person class in a completely random way. Obviously, when testing, we will need to have control over some specific fields. Let's see this as an example. Recall that EasyRandom can also be used with primitive types. Therefore, our example could look like this. Java public class PersonServiceTest { private final EasyRandom easyRandom = new EasyRandom(); private final PersonService personService = new PersonService(); @Test public void testIsAdult() { Person adultPerson = easyRandom.nextObject(Person.class); adultPerson.setAge(18 + easyRandom.nextInt(80)); assertTrue(personService.isAdult(adultPerson)); } @Test public void testIsNotAdult() { Person minorPerson = easyRandom.nextObject(Person.class); minorPerson.setAge(easyRandom.nextInt(17)); assertFalse(personService.isAdult(minorPerson)); } } As we can see, this way of generating test objects protects us from changes in the "Person" class and allows us to focus only on the field we are interested in. We can also use this library to generate lists of random objects. Java @Test void generateObjectsList() { EasyRandom generator = new EasyRandom(); //Generamos una lista de 5 Personas List<Person> persons = generator.objects(Person.class, 5) .collect(Collectors.toList()); assertEquals(5, persons.size()); } This test, in itself, is not very useful. It is simply to demonstrate the ability to generate lists, which could be used to dump data into a database. Generation of Parameterized Data Let's see now how to use this library to have more precise control in generating the object itself. This can be done by parameterization. Set the value of a field. Let's imagine the case that for our tests, we want to keep certain values constant (an ID, a name, an address, etc.) To achieve this, we would have to configure the initialization of objects using "EasyRandomParameters" and locate the parameters by their name. Let's see how: Java EasyRandomParameters params = new EasyRandomParameters(); // Asignar un valor al campo por medio de una función lamba params.randomize(named("age"),()-> 5); EasyRandom easyRandom = new EasyRandom(params); // El objeto tendrá siempre una edad de 5 Person person = easyRandom.nextObject(Person.class); Of course, the same could be done with collections or complex objects. Let's suppose that our class Person, contains an Address class inside and that, in addition, we want to generate a list of two persons. Let's see a more complete example: Java EasyRandomParameters parameters = new EasyRandomParameters() .randomize(Address.class, () -> new Address("Random St.", "Random City")) EasyRandom easyRandom = new EasyRandom(parameters); return Arrays.asList( easyRandom.nextObject(Person.class), easyRandom.nextObject(Person.class) ); Suppose now that a person can have several addresses. This would mean the "Address" field will be a list inside the "Person" class. With this library, we can also make our collections have a variable size. This is something that we can also do using parameters. Java EasyRandomParameters parameters = new EasyRandomParameters() .randomize(Address.class, () -> new Address("Random St.", "Random City")) .collectionSizeRange(2, 10); EasyRandom easyRandom = new EasyRandom(parameters); // El objeto tendrá una lista de entre 2 y 10 direcciones Person person = easyRandom.nextObject(Person.class); Setting Pseudo-Random Fields As we have seen, setting values is quite simple and straightforward. But what if we want to control the randomness of the data? We want to generate random names of people, but still names and not just strings of unconnected characters. This same need is perhaps clearer when we are interested in having randomness in fields such as email, phone number, ID number, card number, city name, etc. For this purpose, it is useful to use other data generation libraries. One of the best-known is Faker. Combining both libraries, we could get a code like this: Java EasyRandomParameters params = new EasyRandomParameters(); //Generar número entre 0 y 17 params.randomize(named("age"), () -> Faker.instance().number().numberBetween(0, 17)); // Generar nombre "reales" aleatorios params.randomize(named("name"), () -> Faker.instance().name().fullName()); EasyRandom easyRandom = new EasyRandom(params); Person person = easyRandom.nextObject(Person.class); There are a multitude of parameters that allow us to control the generation of objects. Closing EasyRandom is a library that should be part of your backpack if you develop unit tests, as it helps maintain unit tests. In addition, and although it may seem strange, establishing some controlled randomness in tests may not be a bad thing. In a way, it is a way to generate new test cases automatically and will increase the probability of finding bugs in code.

By Francisco Moreno CORE

Backpressure is a critical concept in software development, particularly when working with data streams. It refers to the control mechanism that maintains the balance between data production and consumption rates. This article will explore the notion of backpressure, its importance, real-world examples, and how to implement it using Java code. Understanding Backpressure Backpressure is a technique employed in systems involving data streaming where the data production rate may surpass the consumption rate. This imbalance can lead to data loss or system crashes due to resource exhaustion. Backpressure allows the consumer to signal the producer when it's ready for more data, preventing the consumer from being overwhelmed. The Importance of Backpressure In systems without backpressure management, consumers may struggle to handle the influx of data, leading to slow processing, memory issues, and even crashes. By implementing backpressure, developers can ensure that their applications remain stable, responsive, and efficient under heavy loads. Real-World Examples Video Streaming Services Platforms like Netflix, YouTube, and Hulu utilize backpressure to deliver high-quality video content while ensuring the user's device and network can handle the incoming data stream. Adaptive Bitrate Streaming (ABS) dynamically adjusts the video stream quality based on the user's network conditions and device capabilities, mitigating potential issues due to overwhelming data. Traffic Management Backpressure is analogous to traffic management on a highway. If too many cars enter the highway at once, congestion occurs, leading to slower speeds and increased travel times. Traffic signals or ramp meters can be used to control the flow of vehicles onto the highway, reducing congestion and maintaining optimal speeds. Implementing Backpressure in Java Java provides a built-in mechanism for handling backpressure through the Flow API, introduced in Java 9. The Flow API supports the Reactive Streams specification, allowing developers to create systems that can handle backpressure effectively. Here's an example of a simple producer-consumer system using Java's Flow API: Java import java.util.concurrent.*; import java.util.concurrent.Flow.*; public class BackpressureExample { public static void main(String[] args) throws InterruptedException { // Create a custom publisher CustomPublisher<Integer> publisher = new CustomPublisher<>(); // Create a subscriber and register it with the publisher Subscriber<Integer> subscriber = new Subscriber<>() { private Subscription subscription; private ExecutorService executorService = Executors.newFixedThreadPool(4); @Override public void onSubscribe(Subscription subscription) { this.subscription = subscription; subscription.request(1); } @Override public void onNext(Integer item) { System.out.println("Received: " + item); executorService.submit(() -> { try { Thread.sleep(1000); // Simulate slow processing System.out.println("Processed: " + item); } catch (InterruptedException e) { e.printStackTrace(); } subscription.request(1); }); } @Override public void onError(Throwable throwable) { System.err.println("Error: " + throwable.getMessage()); executorService.shutdown(); } @Override public void onComplete() { System.out.println("Completed"); executorService.shutdown(); } }; publisher.subscribe(subscriber); // Publish items for (int i = 1; i <= 10; i++) { publisher.publish(i); } // Wait for subscriber to finish processing and close the publisher Thread.sleep(15000); publisher.close(); } } Java class CustomPublisher<T> implements Publisher<T> { private final SubmissionPublisher<T> submissionPublisher; public CustomPublisher() { this.submissionPublisher = new SubmissionPublisher<>(); } @Override public void subscribe(Subscriber<? super T> subscriber) { submissionPublisher.subscribe(subscriber); } public void publish(T item) { submissionPublisher.submit(item); } public void close() { submissionPublisher.close(); } } In this example, we create a CustomPublisher class that wraps the built-in SubmissionPublisher. The CustomPublisher can be further customized to generate data based on specific business logic or external sources. The Subscriber implementation has been modified to process the received items in parallel using an ExecutorService. This allows the subscriber to handle higher volumes of data more efficiently. Note that the onComplete() method now shuts down the executorService to ensure proper cleanup. Error handling is also improved in the onError() method. In this case, if an error occurs, the executorService is shut down to release resources. Conclusion Backpressure is a vital concept for managing data streaming systems, ensuring that consumers can handle incoming data without being overwhelmed. By understanding and implementing backpressure techniques, developers can create more stable, efficient, and reliable applications. Java's Flow API provides an excellent foundation for building backpressure-aware systems, allowing developers to harness the full potential of reactive programming.

JDK 21, released on September 19, 2023, marks a significant milestone in Java's evolution. It's a long-term support (LTS) release, ensuring stability and support from Oracle for at least eight years. In this release, Java developers are introduced to several new features, including virtual threads, record patterns, pattern matching for switch statements, the foreign function and memory API, and the ZGC garbage collector. The Significance of Virtual Threads Among these features, virtual threads stand out as a game-changer in the world of concurrent Java applications. Virtual threads have the potential to revolutionize the way developers write and manage concurrent code. They offer exceptional benefits, making them ideal for high-throughput concurrent applications. Advantages of Virtual Threads Virtual threads bring several advantages to the table, differentiating them from traditional platform threads: Efficient Memory Usage: Virtual threads consume significantly less memory compared to their traditional counterparts. The JVM intelligently manages memory for virtual threads, enabling reuse and efficient allocation for multiple virtual threads. Optimized CPU Usage: Virtual threads are more CPU-efficient. The JVM can schedule multiple virtual threads on the same underlying OS thread, eliminating the overhead of context switching. Ease of Use: Virtual threads are easier to work with and manage. The JVM takes care of their lifecycle, removing the need for manual creation and destruction. This simplifies concurrent Java application development and maintenance. Understanding Types of Threads To grasp the significance of virtual threads, let's briefly examine three types of threads: 1. OS Threads (Native Threads): These are the fundamental units of execution in modern operating systems. Each OS thread has its own stack, program counter, and register set, managed by the OS kernel. 2. Platform Threads (Traditional Java Threads): Platform threads are built on top of OS threads but managed by the Java Virtual Machine (JVM). They offer more efficient scheduling and resource utilization. A platform thread is an instance of java.lang.Thread implemented in the traditional way, as a thin wrapper around an OS thread. 3. Virtual Threads: Virtual threads are instances of java.lang.Thread is not bound to specific OS threads. Multiple virtual threads can be served by a single OS thread. Virtual threads are even lighter weight and more efficient than traditional threads, and they can be used to improve the performance, scalability, and reliability of concurrent Java applications. The Challenge of Traditional Threads Threads are the building blocks of concurrent server applications and have been integral to Java for nearly three decades. Unfortunately, traditional Java threads are expensive to create and maintain. The number of available threads is limited because the JDK implements threads as wrappers around operating system (OS) threads. OS threads are costly, so we cannot have too many of them, which makes the implementation ill-suited to the thread-per-request style limiting the number of concurrent requests that a Java server can handle and thus impacting the server application scalability. The Shift to Thread-Sharing Over a period of time, to be able to scale and to utilize hardware to its fullest, java developers transitioned from the thread-per-request style to a thread-sharing style. In thread-sharing, instead of handling a request on one thread from start to finish, the request-handling code returns its thread to a pool when it waits for another I/O operation to complete so that the thread can service other requests. This fine-grained sharing saves threads but requires asynchronous programming techniques. Asynchronous programming is a complex and error-prone way to improve scalability, and it is not compatible with the thread-per-request style that is common in Java server applications. Introducing Virtual Threads Virtual threads offer a compelling solution to preserve the thread-per-request style while enhancing concurrency. They are not tied to specific OS threads, allowing for greater concurrency. Virtual threads consume OS threads only during CPU-bound tasks. When they encounter blocking I/O operations, they are automatically suspended without monopolizing an OS thread. Creating and managing virtual threads is straightforward, eliminating the need for pooling. Virtual threads are short-lived, perform specific tasks, and are plentiful. Platform threads are heavyweight, long-lived, and may need pooling. Using virtual threads maintains the thread-per-request style while optimizing hardware utilization. The Future With JDK 21 It's essential to note that with JDK 21, the goal is not to replace traditional thread implementations but to provide an efficient alternative. Developers retain the flexibility to choose between virtual threads and platform threads based on their specific requirements. Here's a simple example that showcases the power of virtual threads: try (var executor = Executors.newVirtualThreadPerTaskExecutor()) { IntStream.range(0, 10_000).forEach(i -> { executor.submit(() -> { Thread.sleep(Duration.ofSeconds(1)); return i; }); }); // Wait for all tasks to complete. } // The executor is automatically closed, and it waits for task completion. In this example, modern hardware efficiently handles 10,000 virtual threads concurrently, executing straightforward code. Behind the scenes, the JDK efficiently manages these virtual threads, optimizing resource utilization. Conclusion Java developers have long relied on threads for concurrent programming. With the introduction of virtual threads in JDK 21, achieving high concurrency while preserving a familiar programming style becomes easier. Virtual threads are especially valuable for server applications that handle numerous user requests, offering efficiency and scalability in a changing landscape.

In October 2023, I visited Devoxx Belgium, and again, it was an awesome event! I learned a lot and received quite some information which I do not want to withhold from you. In this blog, you can find my takeaways from Devoxx Belgium 2023! Introduction Devoxx Belgium is the largest Java conference in Europe. This year, it was already the 20th edition. As always, Devoxx is being held in the fantastic theatres of Kinepolis Antwerp. Last year, there was a rush on the tickets. This was probably due to the fact that the previous two editions were canceled because of COVID-19. Not true. This year, the tickets were sold out even faster! The first batch of tickets was sold out in a few seconds. For the second batch, you could subscribe yourself, and tickets were assigned by means of a lottery system. The first two days of Devoxx are Deep Dive days, where you can enjoy more in-depth talks (about 2-3 hours) and hands-on workshops. Days three and five are the Conference Days, where talks are being held in a time frame of a maximum of 50 minutes. You receive a lot of information! Enough for the introduction. The next paragraphs contain my takeaways from Devoxx. This only scratches the surface of a topic, but it should be enough in order to make you curious to dive a bit deeper into the topic yourself. Do check out the Devoxx YouTube channel. All the sessions of the Deep Dive Days and the Conference Days are recorded and can be viewed here. If you intend to view them all, there are 210 of them… Java Ecosystem Development Java 21 With the release of Java 21 in September, quite some talks were related to the new features of Java 21. A good starting point is the keynote of Brian Goetz, which gives a good summary of what’s new in Java. From here on, you can choose the topics you want to know more about. Some of these topics include virtual threads, ZGC garbage collector (no longer worry about garbage collector pauses!), project Panama (interconnect with native code, but better than JNI), etc. Besides that, Oracle is working on a Visual Studio Code extension in order to make it easier to use VSCode in combination with Java. Also, the Java Playground is announced, where you can easily experiment with Java from within the browser. Brian also looked back at the release cadence of Java. The conclusion is that it worked better than they had hoped for. He also called the open-source maintainers to follow this same release cadence. This should make it easier for everyone to upgrade to a new Java version. jOOQ This deep dive by Simon Martinelli about jOOQ was illuminating (there is also a conference talk available). I do favor writing SQL instead of abstracting the database away by means of an ORM. jOOQ has a database-first approach. Besides that, it offers you typesafe SQL syntax, typesafe SQL constructors, and SQL dialect abstraction. jOOQ offers an open-source version that is free to use. You do need to use the latest stable version of the database you are using. If you need to use an older database version or some specific features, you need a paid license, but prices are not that expensive compared to what jOOQ offers you. jOOQ needs a running database in order to generate source code. This can now easily be done by means of this Maven plugin, which uses Testcontainers. Testing Microservices Victor Rentea is always a pleasure to watch: entertaining talks and a lot of information. Testing microservices is one of the talks he gave at Devoxx; do check out the others as well. You should start with API integration tests and complement them with unit tests. This way, you do not need to make extensive use of mocks. Integration tests are made easier with the help of Wiremock and Testcontainers. When you create tests, you need to ensure that the tests can be executed independently from each other. You can do so by annotating your tests with @transactional. Your database actions will be rolled back after the test has been executed. A warning was raised not to overuse @DirtiesContext. When you use this, Spring needs to be restarted for each test (takes approximately 15-20). In other words, the Spring Context cannot be reused between tests, and your tests just take longer. Performance and Scale This is an interesting talk by Donald Raab and Rustam Mehmandarov about how large data structures pose certain restrictions on performance and scalability. They show you how you can use the Java Object Layout (JOL) tool to analyze the layout of your objects. They show you what you can do in order to minimize the memory footprint of your data structures. Artificial Intelligence Introduction to Building AI Applications This talk by Greg Kamradt introduces you to the world of LangChain. LangChain allows you to create a context-aware reasoning application. The tutorials can be found here. What was interesting to me was the use case where you can add your own data sources and your own content to the Large Language Model (LLM) and ask questions about this content. This is actually how chatbots work. You ask the LLM a question, and it will respond with a human-readable message. LangChain is a Python library, but if you want to integrate this into your Java application, you can use LangChain4j. Do watch this talk by Lize Raes. It is amazing what can be done with just a few lines of code. I hardly took any notes because I was too much paying attention. AI Coding Assistants Something that will impact our lives as a developer is AI coding assistants. Bouke Nijhuis gave a clear overview and demonstration of some coding assistants. He compared Tabnine (which already exists since 2013!), GitHub Copilot, and ChatGPT. ChatGPT does not have an IDE plugin, so it is a bit less user-friendly. Some features like code generation, test generation, explaining code, finding bugs, and refactoring code are demonstrated. Privacy is also covered in this talk. Your code is sent over the wire to the coding assistant. Only Tabnine offers a self-hosting option. So, why should you use a coding assistant? You pair the program with the coding assistant, and you can write code faster. Besides these three coding assistants, JetBrains also announced their AI assistant during the keynote. It is still in preview, but the demonstration was very promising. MLOps If you want to learn more about MLOps, watch this talk by Bert Gossey. MLOps is a model development lifecycle, just like we have the software development lifecycle. It consists of the following steps: Data gathering; Model training; Model evaluation: here, you will verify by means of a subset of the training data whether the model is okay; Model deployment; Monitoring in order to detect data model drift. The steps are executed by specialists: data engineers, data scientists, software engineers, and operations. This means that you will also have handovers. And that is where MLOps comes to the rescue. MLOps actually extends DevOps with Machine Learning (data and model). Tools that support MLOps workflow are KubeFlow and MLFlow. Other Software Architecture as Code Simon Brown, the creator of the C4 model, introduced Structurizr. A tool that allows you to create your C4 diagrams as code. It will also ensure consistency between your diagrams, and it will allow you to create diffs. Structurizr is a complete tool and is quite interesting. I will definitely take a closer look at it. Authorization There were quite some talks about authorization, mainly about OAuth2, OpenID Connect, and WebAuthn. WebAuthn can be used as a 2FA using a Passkey (i.e., passwordless login). You can try it at webauthn.io. It is important to know how these authorization methods work. Spring Authorization Server is a framework that allows you to build something like Keycloak. It is not meant to become a competitor to Keycloak. Use cases are when you need advanced customization, when you need a lightweight authorization server, or for development purposes. Keynotes Do check out the keynotes. The opening and closing keynotes are worth watching. You learn something, and you laugh quite a bit. Embracing Imposter Syndrome by Dom Hodgson Introducing Flow: the worst software development approach in history by Sander Hoogendoorn and Kim van Wilgen Conclusion Devoxx 2023 was great, and I am glad I was able to attend the event. As you can read in this blog, I learned a lot, and I need to take a closer look at many topics. At least I do not need to search for inspiration for future blogs!

By Gunter Rotsaert CORE

In this Java 21 tutorial, we dive into virtual threads, a game-changing feature for developers. Virtual threads are a lightweight and efficient alternative to traditional platform threads, designed to simplify concurrent programming and enhance the performance of Java applications. In this article, we’ll explore the ins and outs of virtual threads, their benefits, compatibility, and the migration path to help you leverage this powerful Java 21 feature. Introducing Virtual Threads Virtual threads represent a significant evolution in the Java platform’s threading model. They are designed to address the challenges of writing, maintaining, and optimizing high-throughput concurrent applications. It’s essential to differentiate virtual threads from traditional platform threads to understand them. In traditional Java, every instance of java.lang.Thread is a platform thread. A platform thread runs Java code on an underlying OS thread and occupies that OS thread for the duration of its execution. It means that the number of platform threads is limited to the number of available OS threads, leading to potential resource constraints and suboptimal performance in highly concurrent applications. On the other hand, a virtual thread is also an instance of java.lang.Thread, but it operates differently. Virtual threads run Java code on an underlying OS thread without capturing the OS thread for its entire lifecycle. This crucial difference means multiple virtual threads can share the same OS thread, offering a highly efficient way to utilize system resources. Unlike platform threads, virtual threads do not monopolize precious OS threads, which can lead to a significantly higher number of virtual threads than the number of available OS threads. The Roots of Virtual Threads Virtual threads draw inspiration from user-mode threads, successfully employed in other multithreaded languages such as Go (with goroutines) and Erlang (with processes). In the early days of Java, user-mode threads were implemented as “green threads” due to the immaturity and limited support for OS threads. These green threads were eventually replaced by platform threads, essentially wrappers for OS threads, operating under a 1:1 scheduling model. Virtual threads take a more sophisticated approach, using an M:N scheduling model. In this model, many virtual threads (M) are scheduled to run on fewer OS threads (N). This M:N scheduling approach allows Java applications to achieve a high concurrency level without the resource constraints typically associated with platform threads. Leveraging Virtual Threads In Java 21, developers can easily harness the power of virtual threads. A new thread builder is introduced to create virtual and platform threads, providing flexibility and control over the threading model. To create a virtual thread, you can use the following code snippet: Java Thread.Builder builder = Thread.ofVirtual().name("Virtual Thread"); Runnable task = () -> System.out.println("Hello World"); Thread thread = builder.start(task); System.out.println(thread.getName()); thread.join(); It’s important to note that virtual threads are significantly cheaper in terms of resource usage when compared to platform threads. You can create multiple virtual threads, allowing you to exploit the advantages of this new threading model fully: Java Thread.Builder builder = Thread.ofVirtual().name("Virtual Thread", 0); Runnable task = () -> System.println("Hello World: " + Thread.currentThread().threadId()); Thread thread1 = builder.start(task); Thread thread2 = builder.start(task); thread1.join(); thread2.join(); Virtual threads can also be effectively utilized with the ExecutorService, as demonstrated in the code below: Java try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { Future<String> future = executor.submit(() -> "Hello World"); System.out.println(future.get()); System.println("The end!"); } The Virtual vs. Platform Thread Trade-Off It’s crucial to understand that platform threads are not deprecated in Java 21, and virtual threads are not a one-size-fits-all solution. Each type of thread has its own set of trade-offs, and the choice between them should be made based on your application’s specific requirements. Virtual threads: Virtual threads are excellent for high-throughput concurrent tasks, especially when managing many lightweight threads without OS thread limitations. They are well-suited for I/O-bound operations, event-driven tasks, and workloads with many short-lived threads. Platform threads: Platform threads are still valuable for applications where fine-grained control over thread interactions is essential. They are ideal for CPU-bound operations, real-time applications, and scenarios that require precise thread management. In conclusion, Java 21’s virtual threads are a groundbreaking addition to the Java platform, offering developers a more efficient and scalable way to handle concurrency. By understanding the differences and trade-offs between virtual and platform threads, you can make informed decisions on when and how to leverage these powerful features to unlock the full potential of your Java applications. Video References Source JPE

By Otavio Santana CORE

Have you ever wondered what happens when the Java Virtual Machine (JVM) encounters a critical error and crashes unexpectedly? Well, that’s when the JVM generates a mysterious file called ‘hs_err_pid.’ In this post, we’re about to unravel the enigma of this file. We’ll delve into its purpose, learn how to decipher its contents and explore the vital information it provides when your Java application goes awry. So, if you’ve ever been perplexed by the ‘hs_err_pid‘ file, or if you’re simply curious about how to make sense of it, read on to discover the key insights you need. What Is the hs_err_pid File? When the Java Virtual Machine (JVM) encounters a severe error and crashes, it leaves behind a trail of breadcrumbs in the form of an ‘hs_err_pid‘ file. This file is a goldmine of information containing details at various levels, such as thread, library, application, resource, environment, and system. It serves as a comprehensive report of the JVM’s state at the moment of the crash. These details can be invaluable for diagnosing the cause of the crash. Where Is the ‘hs_err_pid‘ File Created? When the JVM crashes, the ‘hs_err_pid‘ file’s location is determined as follows: -XX:ErrorFile: If the JVM argument ‘-XX:ErrorFile‘ is specified, the ‘hs_err_pid‘ file will be created in the path specified by this argument. Working Directory: In cases where the ‘-XX:ErrorFile‘ argument is not used, the JVM generates the ‘hs_err_pid‘ file in the working directory of the Java application. Temporary Directory: If, for any reason, the file cannot be created in the working directory (e.g., due to insufficient space, permission issues, or other constraints), the JVM resorts to creating the ‘hs_err_pid‘ file in the temporary directory designated by the operating system. How To Read the ‘hs_err_pid‘ File? The ‘hs_err_pid‘ file is a plain text document, and while it is possible to access and inspect its contents by opening it with a standard text editor, interpreting the raw data within the file can be a challenging task due to its technical nature. In many cases, deciphering the file in its raw format can be complex and time-consuming. To simplify the process and make the information more accessible, many developers opt to use specialized tools like fastThread. This tool is designed to parse ‘hs_err_pid‘ files and presents the data in a more readable and organized format, complete with graphs and metrics. How to analyze the ‘hs_err_pid‘ file? Fig: fastThread tool to analyze hs_err_pid file You can analyze the ‘hs_err_pid’ file using the fastThread tool. Sign In to fastThread Upload the hs_err_pid file Click on the Analyze button Upon completing these steps, fastThread will instantly generate a comprehensive report. This report is designed to provide you with a wealth of information. It includes multiple sections that will help you gain a deep understanding of the JVM issue. Continue reading this post to learn about these sections. JVM Information Fig: JVM version information The first section of the report is dedicated to providing key details about the Java Virtual Machine (JVM). It encompasses: JRE Version: This section reveals the Java Runtime Environment (JRE) version in use at the time of the JVM crash. Crash Time: You’ll find the precise date and time when the JVM encountered a critical error and crashed. Elapsed Time: This valuable metric indicates how long the JVM was operational before the crash occurred. Reason To Crash Fig: High-Level Reason for JVM to crash In this section, you’ll find a high-level reason that led to the JVM crash. Common crash reasons include: SIGSEGV SIGBUS EXCEPTION_ACCESS_VIOLATION EXCEPTION_STACK_OVERFLOW Out of Memory Error For explanations of these reasons, you can refer to this Oracle documentation. Heap Size Fig: JVM Memory regions utilization This section provides a breakdown of the allocated and consumed memory of the JVM’s internal memory regions: Young Gen Old Gen MetaSpace Understanding the allocated and used sizes of these memory regions is essential for diagnosing issues related to memory consumption. In the case of memory leaks, you may observe the used size approaching its maximum capacity. Executed Code/Library In this section, you will find the exact line of code or library that your JVM was executing when the crash occurred. The following are a few examples: Line of Code # J 11538 C2 com.buggyapp.StoryContentPushProcessor.scribeUpsert(Lcom/espn/cricket/data/domain/StoryType;Ljava/nio/file/Path;)V (224 bytes) @ 0x0000000002629d49 [0x0000000002626ce0+0x3069] # J 17883 c2 java.util.concurrent.ConcurrentSkipListMap.doPut(Ljava/lang/Object;Ljava/lang/Object;Z)Ljava/lang/Object; java.base@10.0.2 (548 bytes) @ 0x00007fe0bd97e957 [0x00007fe0bd97b740+0x0000000000003217] Libraries [jvm.dll+0x374bae] [libCSTBk5.so+0x43949] Active Thread Fig: Active Thread’s stack trace This section is probably the most important section in the report as it reveals the thread that was actively executing at the precise moment of the JVM crash, along with its accompanying stack trace. In many instances, the thread actively executing at the time of the crash is a key focal point for identifying the root cause. In the above example, you can notice that the thread is working on the ‘com.sap.conn.rfc.driver‘ package. This package is present in a SAP driver library. Apparently, this application was running on an old version of SAP driver, known to have bugs. Due to this, this application crashed. Once the SAP drivers were upgraded, the JVM crashes within the application ceased. Core Dump Location In the event of a JVM crash, core dumps may be generated. This section informs you about the specific file path where these core dumps are written. All Threads Fig: All Threads running in JVM at the time of crash This section offers insights into the threads within the JVM at the time of the crash. It includes details on the number of threads, their names, states, and types. The number of threads in the JVM can be a critical factor to consider, and in the provided example, there were 1464 threads, which is notably high for the application in question. The names of the threads often provide valuable clues regarding their origin or association with specific thread pools. For instance, in this example, you can observe that there were over a thousand threads originating from the ‘I/O dispatcher’ thread pool. Understanding the thread landscape can be helpful in diagnosing performance and concurrency issues within your Java application. JVM Arguments Fig: JVM arguments with the application were launched This section reveals the System properties (i.e., ‘D’) and JVM arguments (-i.e. -‘X’ and ‘-XX:’) with which your application was launched. Environment Variables Fig: Environment Variables of the device This section contains a comprehensive list of the environment variables that were in effect when the JVM was launched. These variables can include critical elements like ‘PATH,’ ‘SHELL, ‘‘JAVA_HOME, ‘‘CLASSPATH, ‘and more. Understanding the environment variables is essential for assessing the context in which your Java application operates. Dynamic Libraries This section presents a complete list of all the libraries and dependencies, including application libraries, 3rd party libraries, frameworks, and native libraries, with which your Java application was launched. The inventory includes a diverse range of components, such as dynamic libraries (.dll), shared objects (.so), and Java archives (*.jar). System Fig: System-level details This section provides system-level details pertaining to your application: Operating System: It provides detailed information about the operating system on which your application was running. Memory: This section covers the memory configuration of the device where your application was executed. It also reports memory utilization at the time of the crash, providing insights into resource consumption. CPU: You’ll find information about the CPU configuration of the device where your application was running, which can be instrumental in assessing performance and compatibility. JVM Version: This part of the report discloses the JVM version in use, which is critical for compatibility and debugging. Events Info The events Info section of the report contains the following subsections: Internal exceptions: Most recent exceptions that are thrown in JVM are reported here. DeOptimization events: Sometimes, JVM converts compiled (or more optimized) stack frames into interpreted (or less optimized) stack frames. Example: A compiler initially assumes a reference value is never null and tests for it using a trapping memory access. Later on, the application uses null values, and the method is deoptimized and recompiled to use an explicit test-and-branch idiom to detect such nulls. Most recent such deoptimized events are reported in this section. Class redefined: Most recent classes that are redefined are reported in this section. Sometimes, classes get redefined by the APM agents and by other agent technologies. Compilation events: Shows which methods have been recently compiled from Java bytecode to native code. Conclusion In this exploration of the ‘hs_err_pid‘ file in Java, we’ve delved into a valuable source of information that can be the key to diagnosing JVM crashes. By decoding the ‘hs_err_pid‘ file, you can transform error messages into actionable insights, making troubleshooting a smoother journey than it needs to be.

By Ram Lakshmanan CORE

Among Java developers, a prevailing assumption is that the number of native threads that can be created within the Java Virtual Machine (JVM) is linked to the stack size. To scrutinize this widespread notion, an experiment was conducted. The results revealed that stack size plays a less significant role in native thread creation than previously thought. The Experiment The experiment utilized the following Java program, which continuously creates threads and counts them using an AtomicInteger. Java import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.locks.LockSupport; public class ThreadCounter { public static void main(String[] args) { AtomicInteger counter = new AtomicInteger(); while (true) { Thread thread = new Thread(() -> { counter.incrementAndGet(); if (counter.get() % 100 == 0) { System.out.printf("Number of threads created so far: %d%n", counter.get()); } LockSupport.park(); }); thread.start(); } } } Test Environment The test ran on a machine with the following configuration: Processor: Apple M1 Max Memory: 64 GB Operating System: macOS Vancura Java Version: OpenJDK 21 Experimental Results Default Stack Size The initial test was conducted using the default stack size of 2048 KB. Approximately 16,300 threads were created before an OutOfMemoryError was triggered. Number of threads created so far: 16300. Plain Text Number of threads created so far: 16300 [3.566s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 2048k, guardsize: 16k, detached. [3.566s][warning][os,thread] Failed to start the native thread for java.lang.Thread "Thread-16354" Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached Calculations indicate that the memory required for these 16,300 threads was around 31.875 GB, well below the total memory of 64 GB. 10 MB Stack Size To change the stack size to 10 MB, the following JVM option was used: -XX:ThreadStackSize=10240 The program yielded a similar result- 16,300 threads- before the OutOfMemoryError occurred again. Number of threads created so far: 16300. Plain Text Number of threads created so far: 16300 [3.995s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 10240k, guardsize: 16k, detached. [3.995s][warning][os,thread] Failed to start the native thread for java.lang.Thread "Thread-16354" Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached at java.base/java.lang.Thread.start0(Native Method) In case you want to use IntelliJ IDEA, you can configure the stack size, and check the following image: 1 GB Stack Size Finally, for the most extreme case, the stack size was set to 1 GB using: -XX:ThreadStackSize=1048576 Remarkably, the program again maxed out at approximately 16,300 threads, reinforcing the pattern observed in the previous tests. Number of threads created so far: 16300. Plain Text Number of threads created so far: 16200 Number of threads created so far: 16300 [3.497s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1048576k, guardsize: 16k, detached. [3.497s][warning][os,thread] Failed to start the native thread for java.lang.Thread "Thread-16354" Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached Insights From the Data The consistency across all three tests underscores that stack size does not affect the number of native threads that can be created. Rather, the limitation appears to be imposed by the operating system. It's worth noting that the OS was capable of creating more threads than the available physical memory, thanks to the concept of virtual memory. Virtual memory leverages disk space to extend RAM, allowing applications to allocate more memory than is physically present, albeit at a lower access speed. Conclusion Contrary to the commonly held belief, this experiment proves that stack size does not have an impact on the number of native threads that can be created in a JVM environment. The constraint is primarily set by the operating system. This investigation effectively dispels the myth that stack size is the determining factor in native thread limitations. Thank you for reading this. If you have a different perspective on this topic or if your experiments yield different results, I would be very interested to hear about it. Please feel free to share your findings with me. Don't forget to share this post!

By A N M Bazlur Rahman CORE

Apache JMeter is an open-source, Java-based tool used for load and performance testing of various services, particularly web applications. It supports multiple protocols like HTTP, HTTPS, FTP, and more. JMeter can simulate heavy loads on a server to analyze performance under different conditions. It offers both GUI and non-GUI modes for test configuration and can display test results in various formats. JMeter also supports distributed testing, enabling it to handle multiple test threads simultaneously. Its functionality can be extended through plugins, making it a versatile and widely used tool in performance testing. JMeter stands out from other testing tools due to its exceptional concurrency model, which governs how it executes requests in parallel. The concurrency model of JMeter relies on Thread Pools, widely recognized as the standard method for parallel processing in Java and several other programming languages. However, as with any advantage, there comes a significant trade-off: the resource-intensive nature of JMeter’s concurrency model. In JMeter, each thread corresponds to a Java thread, further utilizing an Operating System (OS) thread for its execution. OS threads, although effective in accomplishing concurrent tasks, carry a certain level of weightiness, manifested in terms of memory consumption and CPU usage during context switching. This attribute poses a noteworthy challenge to JMeter’s performance. Moreover, certain operating systems enforce strict limitations on the total number of threads that can be generated, imposing implicit restrictions on JMeter’s capabilities. Unleashing the True Power, Java 21 to the Rescue Project Loom, which has gained significant attention within the Java community over the past several years, has finally been incorporated into Java 21 after several early preview releases with JEP-444. Java’s virtual threads, also known as lightweight or user-mode threads, are introduced as an experimental feature under Project Loom, which is now officially included in Java 21. While the details of this feature are interesting, they’re not the main focus of our discussion today, so that we won’t delve deeper into them at this moment. JMeter The code review reveals a straightforward process for creating a new thread group by coying ThreadGroup class. In this instance, we have simply duplicated the logic from the ThreadGroup JMeter class that we wish to modify. A key method to note is startNewThread, which is responsible for creating the threads. We have altered one line in this method: The original line of code: Thread newThread = new Thread(jmThread, jmThread.getThreadName()); Has been replaced with the following: Thread newThread = Thread.ofVirtual() .name(jmThread.getThreadName()) .unstarted(jmThread); In this modification, instead of creating a traditional thread, we’re creating a virtual thread, as introduced in Java’s Project Loom. This change allows for more lightweight, efficient thread handling. Also, other modifications, such as removing the synchronized block from addNewThreadmethod and updating similar thread creation logic at a few other places. Setup I have quickly set up nginx, which always returns a 200 ok response: # nginx.conf location /test { return 200 'OK'; } 2. Add Virtual Thread Group element: 3. Configure Threads: you will see the title Virtual Thread Properties header for the right thread group. 4. Get set, go....!! and final result: My primary focus was not on the server’s responsiveness or its ability to scale up to 50k users (which, with some tuning, could be easily achieved). Instead, I was more interested in observing how JMeter generates and handles the load, irrespective of whether the server responses were successful or failed. Summary JMeter has traditionally been resource-intensive, primarily due to its I/O-bound nature involving network requests. However, with the introduction of virtual threads, it has significantly improved in performance. The utilization of virtual threads has enabled JMeter to operate smoothly and efficiently without any glitches, even when handling heavy loads. Source Code Anyone interested in trying on their own, see the following GitHub Project for more details.

Many libraries for AI app development are primarily written in Python or JavaScript. The good news is that several of these libraries have Java APIs as well. In this tutorial, I'll show you how to build a ChatGPT clone using Spring Boot, LangChain, and Hilla. The tutorial will cover simple synchronous chat completions and a more advanced streaming completion for a better user experience. Completed Source Code You can find the source code for the example in my GitHub repository. Requirements Java 17+ Node 18+ An OpenAI API key in an OPENAI_API_KEY environment variable Create a Spring Boot and React project, Add LangChain First, create a new Hilla project using the Hilla CLI. This will create a Spring Boot project with a React frontend. Shell npx @hilla/cli init ai-assistant Open the generated project in your IDE. Then, add the LangChain4j dependency to the pom.xml file: XML <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j</artifactId> <version>0.22.0</version> <!-- TODO: use latest version --> </dependency> Simple OpenAI Chat Completions With Memory Using LangChain We'll begin exploring LangChain4j with a simple synchronous chat completion. In this case, we want to call the OpenAI chat completion API and get a single response. We also want to keep track of up to 1,000 tokens of the chat history. In the com.example.application.service package, create a ChatService.java class with the following content: Java @BrowserCallable @AnonymousAllowed public class ChatService { @Value("${openai.api.key}") private String OPENAI_API_KEY; private Assistant assistant; interface Assistant { String chat(String message); } @PostConstruct public void init() { var memory = TokenWindowChatMemory.withMaxTokens(1000, new OpenAiTokenizer("gpt-3.5-turbo")); assistant = AiServices.builder(Assistant.class) .chatLanguageModel(OpenAiChatModel.withApiKey(OPENAI_API_KEY)) .chatMemory(memory) .build(); } public String chat(String message) { return assistant.chat(message); } } @BrowserCallable makes the class available to the front end. @AnonymousAllowed allows anonymous users to call the methods. @Value injects the OpenAI API key from the OPENAI_API_KEY environment variable. Assistant is the interface that we will use to call the chat API. init() initializes the assistant with a 1,000-token memory and the gpt-3.5-turbo model. chat() is the method that we will call from the front end. Start the application by running Application.java in your IDE, or with the default Maven goal: Shell mvn This will generate TypeScript types and service methods for the front end. Next, open App.tsx in the frontend folder and update it with the following content: TypeScript-JSX export default function App() { const [messages, setMessages] = useState<MessageListItem[]>([]); async function sendMessage(message: string) { setMessages((messages) => [ ...messages, { text: message, userName: "You", }, ]); const response = await ChatService.chat(message); setMessages((messages) => [ ...messages, { text: response, userName: "Assistant", }, ]); } return ( <div className="p-m flex flex-col h-full box-border"> <MessageList items={messages} className="flex-grow" /> <MessageInput onSubmit={(e) => sendMessage(e.detail.value)} /> </div> ); } We use the MessageList and MessageInput components from the Hilla UI component library. sendMessage() adds the message to the list of messages, and calls the chat() method on the ChatService class. When the response is received, it is added to the list of messages. You now have a working chat application that uses the OpenAI chat API and keeps track of the chat history. It works great for short messages, but it is slow for long answers. To improve the user experience, we can use a streaming completion instead, displaying the response as it is received. Streaming OpenAI Chat Completions With Memory Using LangChain Let's update the ChatService class to use a streaming completion instead: Java @BrowserCallable @AnonymousAllowed public class ChatService { @Value("${openai.api.key}") private String OPENAI_API_KEY; private Assistant assistant; interface Assistant { TokenStream chat(String message); } @PostConstruct public void init() { var memory = TokenWindowChatMemory.withMaxTokens(1000, new OpenAiTokenizer("gpt-3.5-turbo")); assistant = AiServices.builder(Assistant.class) .streamingChatLanguageModel(OpenAiStreamingChatModel.withApiKey(OPENAI_API_KEY)) .chatMemory(memory) .build(); } public Flux<String> chatStream(String message) { Sinks.Many<String> sink = Sinks.many().unicast().onBackpressureBuffer(); assistant.chat(message) .onNext(sink::tryEmitNext) .onComplete(sink::tryEmitComplete) .onError(sink::tryEmitError) .start(); return sink.asFlux(); } } The code is mostly the same as before, with some important differences: Assistant now returns a TokenStream instead of a String. init() uses streamingChatLanguageModel() instead of chatLanguageModel(). chatStream() returns a Flux<String> instead of a String. Update App.tsx with the following content: TypeScript-JSX export default function App() { const [messages, setMessages] = useState<MessageListItem[]>([]); function addMessage(message: MessageListItem) { setMessages((messages) => [...messages, message]); } function appendToLastMessage(chunk: string) { setMessages((messages) => { const lastMessage = messages[messages.length - 1]; lastMessage.text += chunk; return [...messages.slice(0, -1), lastMessage]; }); } async function sendMessage(message: string) { addMessage({ text: message, userName: "You", }); let first = true; ChatService.chatStream(message).onNext((chunk) => { if (first && chunk) { addMessage({ text: chunk, userName: "Assistant", }); first = false; } else { appendToLastMessage(chunk); } }); } return ( <div className="p-m flex flex-col h-full box-border"> <MessageList items={messages} className="flex-grow" /> <MessageInput onSubmit={(e) => sendMessage(e.detail.value)} /> </div> ); } The template is the same as before, but the way we handle the response is different. Instead of waiting for the response to be received, we start listening for chunks of the response. When the first chunk is received, we add it as a new message. When subsequent chunks are received, we append them to the last message. Re-run the application, and you should see that the response is displayed as it is received. Conclusion As you can see, LangChain makes it easy to build LLM-powered AI applications in Java and Spring Boot. With the basic setup in place, you can extend the functionality by chaining operations, adding external tools, and more following the examples on the LangChain4j GitHub page, linked earlier in this article. Learn more about Hilla in the Hilla documentation.