Designing AI Multi-Agent Systems in Java

We will design an AI Multi Agent, using Fibry. Being backed by Actors, it allows us to take control the parallelism in great detail.

Luca Venturi

Feb. 10, 25 · Tutorial

Likes (1)

Comment

Save

6.8K Views

The year 2025 is the year of AI agents. For the purposes of this article, an AI agent is a system that can leverage AI to achieve a goal by following a series of steps, possibly reasoning on its results and making corrections. In practice, the steps that an agent follows can constitute a graph.

We will build a reactive agent (meaning that it reacts to a stimulus, in our case, the input from a user) to help people find their perfect vacation. Our agent will find the best city in the specified country, considering the food, sea, and activity specified by the user.

The agent will look like this:

In the first phase, it will collect information in parallel, ranking the cities by a single characteristic. The last step will use this information to choose the best city.

You could use a search engine to collect information, but we will use ChatGPT for all the steps, though we will use different models.

You could write all the code by hand or use some library to help you simplify the code a bit. Today, we will use a new feature that I added to Fibry, my Actor System, to implement the graph and control the parallelism in great detail.

Fibry is a simple and small Actor System that provides an easy way to leverage actors to simplify multi-threading code and does not have any dependency. Fibry also implements a Finite State Machine, so I decided to extend it to make it easier to write agents in Java. My inspiration has been LangGraph.

As Fibry is about multi-threading, the new features allow plenty of flexibility in deciding the level of parallelism while keeping everything as simple as possible.

You should use Fibry 3.0.2, for example:

    Plain Text
   
   compile group: 'eu.lucaventuri', name: 'fibry', version: '3.0.2'

Defining the Prompts

The first step is defining the prompts that we need for the LLM:

    Java
   
 

   public static class AiAgentVacations {
  private static final String promptFood = "You are a foodie from {country}. Please tell me the top 10 cities for food in {country}.";
  private static final String promptActivity = "You are from {country}, and know it inside out. Please tell me the top 10 cities in {country} where I can {goal}";
  private static final String promptSea = "You are an expert traveler, and you {country} inside out. Please tell me the top 10 cities for sea vacations in {country}.";
  private static final String promptChoice = """
    You enjoy traveling, eating good food and staying at the sea, but you also want to {activity}. Please analyze the following suggestions from your friends for a vacation in {country} and choose the best city to visit, offering the best mix of food and sea and where you can {activity}.
    Food suggestions: {food}.
    Activity suggestions: {activity}.
    Sea suggestions: {sea}.
    """;

}
  

Defining the States

Normally, you would define four states, one for each step. However, since branching out and back is quite common, I added a feature to handle this with only a single state. As a result, we need only two states: CITIES, where we collect information, and CHOICE, where we choose the city.

    Plain Text
   
   enum VacationStates { CITIES, CHOICE }

Defining the Context

The different steps of the agent will collect information that needs to be stored somewhere; let’s call it context. Ideally, you would want every step to be independent and know as little as possible of the other, but achieving this in a simple way, with a low amount of code while keeping as much type safety as possible and maintaining thread safety, is not exactly straightforward.

As a result, I choose to force the context to be a record, providing some functionality to update the values of the record (using reflection underneath) while we wait for JEP 468 (Derived Record Creation) to be implemented.

    Java
   
 

   public record VacationContext(String country, String goal, String food, String activity, String sea, String proposal) {
  public static VacationContext from(String country, String goal) {
    return new VacationContext(country, goal, null, null, null, null);
  }
}
  

Defining the Nodes

Now, we can define the logic of the agent. We will allow the user to use two different LLM models, for example, a “normal” LLM for the search and a “reasoning” one for the choice step.

This is where things become a bit trickier, as it is quite dense:

    Java
   
 

   AgentNode<VacationStates, VacationContext> nodeFood = state -> state.setAttribute("food", modelSearch.call("user", replaceField(promptFood, state.data(), "country")));
AgentNode<VacationStates, VacationContext> nodeActivity = state -> state.setAttribute("activity", modelSearch.call("user", replaceField(promptActivity, state.data(), "country")));
AgentNode<VacationStates, VacationContext> nodeSea = state -> state.setAttribute("sea", modelSearch.call("user", replaceField(promptSea, state.data(), "country")));
AgentNode<VacationStates, VacationContext> nodeChoice = state -> {
  var prompt = replaceAllFields(promptChoice, state.data());
  System.out.println("***** CHOICE PROMPT: " + prompt);
  return state.setAttribute("proposal", modelThink.call("user", prompt));
};
  

As you might have guessed, modelSearch is the model used for search (e.g., ChatGPT 4o), and modelThink could be a “reasoning model” (e.g., ChatGPT o1). Fibry provides a simple LLM interface and a simple implementation for ChatGPT, exposed by the class ChatGpt.

Please note that calling ChatPGT API requires an API key that you need to define using the “-DOPENAI_API_KEY=xxxx” JVM parameter.

Different and more advanced use cases will require custom implementations or the usage of a library.

There is also a small issue related to the philosophy of Fibry, as Fibry is meant not to have any dependencies, and this gets tricky with JSON. As a result, now Fibry can operate in two ways:

If Jackson is detected, Fibry will use it with reflection to parse JSON.
If Jackson is not detected, a very simple custom parser (that seems to work with ChatGPT output) is used. This is recommended only for quick tests, not for production.
Alternatively, you can provide your own JSON processor implementation and call JsonUtils.setProcessor(), possibly checking JacksonProcessor for inspiration.
The replaceField() and replaceAllFields() methods are defined by RecordUtils and are just convenience methods to replace text in the prompt, so that we can provide our data to the LLM.The setAttribute() function is used to set the value of an attribute in the state without you having to manually recreate the record or define a list of “withers” methods. There are other methods that you might use, like mergeAttribute(), addToList(), addToSet(), and addToMap().

Building the Agent

Now that we have the logic, we need to describe the graph of dependencies between states and specify the parallelism we want to achieve. If you imagine a big multi-agent system in production, being able to express the parallelism required to maximize performance without exhausting resources, hitting rate limiters, or exceeding the parallelism allowed by external systems is a critical feature. This is where Fibry can help, making everything explicit but relatively easy to set up.

Let’s start creating the agent builder:

    Plain Text
   
   var builder = AiAgent.<VacationStates, VacationContext>builder(true);

The parameter autoGuards is used to put automatic guards on the states, which means that they are executed with an AND logic, and a state is executed only after all the incoming states have been processed.

If the parameter is false, the state is called once for each incoming state.

In the previous example, if the intention is to execute D once after A and once after C, then autoGuards should be false, while if you want it to be called only once after both have been executed, then autoGuards should be true.

But let’s continue with the vacation agent.

    Plain Text
   
   builder.addState(VacationStates.CHOICE, null, 1, nodeChoice, null);

Let’s start with the method addState(). It is used to specify that a certain state should be followed by another state and execute a certain logic. In addition, you can specify the parallelism (more on that soon) and the guards.

In this case:

The state is CHOICE
There is no default following state (e.g., this is a final state)
The parallelism is 1
There is no guard

The next state is just a default because the node has the possibility to overwrite the next state, which means that the graph can dynamically change at runtime, and in particular, it can perform cycles, for example, if some steps need to be repeated to collect more or better information. This is an advanced use case.

An unexpected concept might be the parallelism. This has no consequences in a single run of the agent, but it is meaningful in production at scale.

In Fibry, every node is backed by an actor, which, from a practical point of view, is a thread with a list of messages to process. Every message is an execution step. So parallelism is the number of messages that can be executed at a single time. In practice:

parallelism == 1 means there is only one thread managing the step, so only one execution at a time.
parallelism > 1 means that there is a thread pool backing the actor, with the number of threads specified by the user. By default, it uses virtual threads.
parallelism == 0 means that every message creates a new actor backed by a virtual thread, so the parallelism can be as high as necessary.

Every step can be configured configured independently, which should allow you to configure performance and resource usage quite well. Please consider that if parallelism != 1, you might have multi-threading, as the thread confinement typically associated with actors is lost.

This was a lot to digest. If it is clear, you can check state compression.

State Compression

As said earlier, it is quite common to have a few states that are related to each other, they need to be performed in parallel and join before moving to a common state.

In this case, you do not need to define multiple states, but you can use only one:

    Plain Text
   
   builder.addStateParallel(VacationStates.CITIES, VacationStates.CHOICE, 1, List.of(nodeFood, nodeActivity, nodeSea), null);

In this case, we see that the CITIES state is defined by three nodes, and addStateParallel() takes care of executing them in parallel and waits for the execution of all of them to be finished. In this case, the parallelism is applied to each node, so in this case, you will get three single-thread actors.

Please note that if you do not use autoGuards, this basically allows you to mix OR and AND logic.

In case you want to merge some nodes in the same state, but they need to be executed serially (e.g., because they need information generated by the previous node), the addStateSerial() method is also available.

AIAgent creation is simple, but there are a few parameters to specify:

The initial state
The final state (which can be null)
A flag to execute states in parallel when possible

    Plain Text
   
   var vacationAgent = builder.build(VacationStates.CITIES, null, true);

Now we have an agent, and we can use it, calling process:

    Plain Text
   
   vacationsAgent.process(AiAgentVacations.VacationContext.from("Italy", "Dance Salsa and Bachata"), (state, info) -> System.out.println(state + ": " + info));

This version of process() takes two parameters:

The initial state, which contains the information required by the agent to perform its actions
An optional listener, for example, if you want to print the output of each step

If you need to start the action and check its return value, later, you can use processAsync().

If you are interested in learning more about the parallelism options, I recommend you check the unit test TestAIAgent. It simulates an agent with nodes that sleep for a while and can help you see the impact of each choice:

But I promised you a multi-agent, didn’t I?

Extending to Multi-Agents

The AIAgent that you just created is an actor, so it runs on its own thread (plus all the threads used by the nodes), and it also implements the Function interface, in case you need it.

There is actually nothing special about a multi-agent; just one or more nodes of an agent ask another agent to perform an action. However, you can build a library of agents and combine them in the best way while simplifying the whole system.

Let’s imagine that we want to leverage the output of our previous agent and use it to calculate how much that vacation would cost so the user can decide if it is affordable enough. Like a real Travel Agent!

This is what we want to build:

First, we need prompts to extract the destination and compute the cost.

    Java
   
   private static final String promptDestination = "Read the following text describing a destination for a vacation and extract the destination as a simple city and country, no preamble. Just the city and the country. {proposal}";
private static final String promptCost = "You are an expert travel agent. A customer asked you to estimate the cost of travelling from {startCity}, {startCountry} to {destination}, for {adults} adults and {kids} kids}";

We just need two states, one to research the cities, which is done by the previous agent, and one to calculate the cost.

    Plain Text
   
   enum TravelStates { SEARCH, CALCULATE }

We also need a context, that should also hold the proposal from the previous agent.

    Plain Text
   
   public record TravelContext(String startCity, String startCountry, int adults, int kids, String destination, String cost, String proposal) { }

Then we can define the agent logic, which requires as a parameter another agent.

The first node calls the previous agent to get the proposal.

    Java
   
 

   var builder = AiAgent.<TravelStates, TravelContext>builder(false);
AgentNode<TravelStates, TravelContext> nodeSearch = state -> {
  var vacationProposal = vacationsAgent.process(AiAgentVacations.VacationContext.from(country, goal), 1, TimeUnit.MINUTES, (st, info) -> System.out.print(debugSubAgentStates ? st + ": " + info : ""));
  return state.setAttribute("proposal", vacationProposal.proposal())
  .setAttribute("destination", model.call(promptDestination.replaceAll("\\{proposal\\}", vacationProposal.proposal())));
};
  

The second node computes the cost:

    Plain Text
   
   AgentNode<TravelStates, TravelContext> nodeCalculateCost = state -> state.setAttribute("cost", model.call(replaceAllFields(promptCost, state.data())));

Then, we can define the graph and build the agent

    Java
   
   builder.addState(TravelStates.SEARCH, TravelStates.CALCULATE, 1, nodeSearch, null);
builder.addState(TravelStates.CALCULATE, null, 1, nodeCalculateCost, null);

var agent = builder.build(TravelStates.SEARCH, null, false);

Now we can instantiate the two agents (I chose to use ChatGPT 4o and ChatGPT 01-mini) and use them:

    Java
   
 

   try (var vacationsAgent = AiAgentVacations.buildAgent(ChatGPT.GPT_MODEL_4O, ChatGPT.GPT_MODEL_O1_MINI)) {
  try (var travelAgent = AiAgentTravelAgency.buildAgent(ChatGPT.GPT_MODEL_4O, vacationsAgent, "Italy", "Dance Salsa and Bachata", true)) {
    var result = travelAgent.process(new AiAgentTravelAgency.TravelContext("Oslo", "Norway", 2, 2, null, null, null), (state, info) -> System.out.println(state + ": " + info));

    System.out.println("*** Proposal: " + result.proposal());
    System.out.println("\n\n\n*** Destination: " + result.destination());
    System.out.println("\n\n\n*** Cost: " + result.cost());
  }
}
  

Final Outputs

If you wonder what the result is, here is the long output that you can get when stating that what you want to do is to dance Salsa and Bachata:

Destination

    Plain Text
   
   Naples, Italy

Proposal

    Plain Text
   
 

   Based on the comprehensive analysis of your friends' suggestions, **Naples** emerges as the ideal city for your vacation in Italy. Here's why Naples stands out as the best choice, offering an exceptional mix of excellent food, beautiful seaside experiences, and a vibrant salsa and bachata dance scene:

### **1. Vibrant Dance Scene**
- **Dance Venues:** Naples boasts numerous venues and events dedicated to salsa and bachata, ensuring that you can immerse yourself in lively dance nights regularly.
- **Passionate Culture:** The city's passionate and energetic atmosphere enhances the overall dance experience, making it a hotspot for Latin dance enthusiasts.

### **2. Culinary Excellence**
- **Authentic Neapolitan Pizza:** As the birthplace of pizza, Naples offers some of the best and most authentic pizzerias in the world.
- **Fresh Seafood:** Being a coastal city, Naples provides access to a wide variety of fresh seafood dishes, enhancing your culinary adventures.
- **Delicious Pastries:** Don't miss out on local specialties like **sfogliatella**, a renowned Neapolitan pastry that is a must-try for any foodie.

### **3. Stunning Seaside Location**
- **Bay of Naples:** Enjoy breathtaking views and activities along the Bay of Naples, including boat tours and picturesque sunsets.
- **Proximity to Amalfi Coast:** Naples serves as a gateway to the famous Amalfi Coast, allowing you to explore stunning coastal towns like Amalfi, Positano, and Sorrento with ease.
- **Beautiful Beaches:** Relax on the city's beautiful beaches or take short trips to nearby seaside destinations for a perfect blend of relaxation and exploration.

### **4. Cultural Richness**
- **Historical Sites:** Explore Naples' rich history through its numerous museums, historic sites, and UNESCO World Heritage landmarks such as the Historic Centre of Naples.
- **Vibrant Nightlife:** Beyond dancing, Naples offers a lively nightlife scene with a variety of bars, clubs, and entertainment options to suit all tastes.

### **5. Accessibility and Convenience**
- **Transportation Hub:** Naples is well-connected by air, rail, and road, making it easy to travel to other parts of Italy and beyond.
- **Accommodation Options:** From luxury hotels to charming boutique accommodations, Naples offers a wide range of lodging options to fit your preferences and budget.

### **Conclusion**
Naples perfectly balances a thriving dance scene, exceptional culinary offerings, and beautiful seaside attractions. Its unique blend of culture, history, and vibrant nightlife makes it the best city in Italy to fulfill your desires for travel, good food, and lively dance experiences. Whether you're dancing the night away, savoring authentic pizza by the sea, or exploring nearby coastal gems, Naples promises an unforgettable vacation.

### **Additional Recommendations**
- **Day Trips:** Consider visiting nearby attractions such as Pompeii, the Isle of Capri, and the stunning Amalfi Coast to enrich your travel experience.
- **Local Experiences:** Engage with locals in dance classes or attend festivals to dive deeper into Naples' vibrant cultural scene.

Enjoy your trip to Italy, and may Naples provide you with the perfect blend of everything you're looking for!
  

Cost

    Plain Text
   
 

   To estimate the cost of traveling from Oslo, Norway, to Naples, Italy, for two adults and two kids, we need to consider several key components of the trip: flights, accommodations, local transportation, food, and activities. Here's a breakdown of potential costs:

1. **Flights**:
   - Round-trip flights from Oslo to Naples typically range from $100 to $300 per person, depending on the time of booking, the season, and the airline. Budget airlines might offer lower prices, while full-service carriers could be on the higher end.
   - For a family of four, the cost could range from $400 to $1,200.

2. **Accommodations**:
   - Hotels in Naples can vary significantly. Expect to pay approximately $70 to $150 per night for a mid-range hotel room that accommodates a family. Vacation rentals might offer more flexibility and potentially lower costs.
   - For a typical 5-night stay, this would range from $350 to $750.

3. **Local Transportation**:
   - Public transportation in Naples (buses, metro, trams) is affordable, and daily tickets cost around $4 per person.
   - Assume about $50 to $100 for the family's local transport for the entire trip, depending on usage.

4. **Food**:
   - Dining costs are highly variable. A budget for meals might be around $10-$20 per person per meal at casual restaurants, while dining at mid-range restaurants could cost $20-$40 per person.
   - A family of four could expect to spend around $50 to $100 per day, reaching a total of $250 to $500 for five days.

5. **Activities**:
   - Entry fees for attractions can vary. Some museums and archaeological sites charge around $10 to $20 per adult, with discounts for children.
   - Budget around $100 to $200 for family activities and entrance fees.

6. **Miscellaneous**:
   - Always allow a little extra for souvenirs, snacks, and unexpected expenses. A typical buffer might be $100 to $200.

**Estimated Total Cost**:
- **Low-end estimate**: $1,250
- **High-end estimate**: $2,950

These are general estimates and actual costs can vary based on when you travel, how far in advance you book, and your personal preferences for accommodation and activities. For the most accurate assessment, consider reaching out to airlines for current flight prices, hotels for room rates, and looking into specific attractions you wish to visit.
  

That was a lot, and this is only the output of the two “reasoning” models!

But the result is quite interesting. Naples is on my bucket list, and I am curious to see if the agent is correct!

Let’s also check the intermediate results to see how it reached this conclusion, which seems reasonable to me.

Intermediate Outputs

If you are curious, there are intermediate results.

Food

    Plain Text
   
 

   As a foodie exploring Italy, you're in for a treat, as the country boasts a rich culinary heritage with regional specialties. Here's a list of the top 10 cities in Italy renowned for their food:
**Bologna** - Often referred to as the gastronomic heart of Italy, Bologna is famous for its rich Bolognese sauce, tasty mortadella, and fresh tagliatelle.
**Naples** - The birthplace of pizza, Naples offers authentic Neapolitan pizza, as well as delicious seafood and pastries like sfogliatella.
**Florence** - Known for its Florentine steak, ribollita (a hearty bread and vegetable soup), and delicious wines from the surrounding Tuscany region.
**Rome** - Enjoy classic Roman dishes such as carbonara, cacio e pepe, and Roman-style artichokes in the bustling capital city.
**Milan** - A city that blends tradition and innovation, Milan offers risotto alla milanese, ossobuco, and an array of high-end dining experiences.
**Turin** - Known for its chocolate and coffee culture, as well as traditional dishes like bagna cauda and agnolotti.
**Palermo** - Sample the vibrant street food scene with arancini, panelle, and sfincione, as well as fresh local seafood in this Sicilian capital.
**Venice** - Famous for its seafood risotto, sarde in saor (sweet and sour sardines), and cicchetti (Venetian tapas) to enjoy with a glass of prosecco.
**Parma** - Home to the famous Parmigiano-Reggiano cheese and prosciutto di Parma, it’s a haven for lovers of cured meats and cheeses.
**Genoa** - Known for its pesto Genovese, focaccia, and variety of fresh seafood dishes, Genoa offers a unique taste of Ligurian cuisine.

Each of these cities offers a distinct culinary experience influenced by local traditions and ingredients, making them must-visit destinations for any food enthusiast exploring Italy.
  

Sea

    Plain Text
   
 

   Italy is renowned for its stunning coastline and beautiful seaside cities. Here are ten top cities and regions perfect for a sea vacation:

**Amalfi** - Nestled in the famous Amalfi Coast, this city is known for its dramatic cliffs, azure waters, and charming coastal villages.
**Positano** - Also on the Amalfi Coast, Positano is famous for its colorful buildings, steep streets, and picturesque pebble beachfronts.
**Sorrento** - Offering incredible views of the Bay of Naples, Sorrento serves as a gateway to the Amalfi Coast and provides a relaxing seaside atmosphere.
**Capri** - The island of Capri is known for its rugged landscape, upscale hotels, and the famous Blue Grotto, a spectacular sea cave.
**Portofino** - This quaint fishing village on the Italian Riviera is known for its picturesque harbor, pastel-colored houses, and luxurious coastal surroundings.
**Cinque Terre** - Comprising five stunning villages along the Ligurian coast, Cinque Terre is a UNESCO World Heritage site known for its dramatic seaside and hiking trails.
**Taormina** - Situated on a hill on the east coast of Sicily, Taormina offers sweeping views of the Ionian Sea and beautiful beaches like Isola Bella.
**Rimini** - Located on the Adriatic coast, Rimini is known for its long sandy beaches and vibrant nightlife, making it a favorite for beach-goers and party enthusiasts.
**Alghero** - A city on the northwest coast of Sardinia, Alghero is famous for its medieval architecture, stunning beaches, and Catalan culture.
**Lerici** - Near the Ligurian Sea, Lerici is part of the stunning Gulf of Poets and is known for its beautiful bay, historic castle, and crystal-clear waters.

Each of these destinations offers a unique blend of beautiful beaches, cultural sites, and local cuisine, making Italy a fantastic choice for a sea vacation.
  

Activity

    Plain Text
   
 

   Italy has a vibrant dance scene with many cities offering great opportunities to enjoy salsa and bachata. Here are ten cities where you can indulge in these lively dance styles:

**Rome** - The capital city has a bustling dance scene with numerous salsa clubs and events happening regularly.
**Milan** - Known for its nightlife, Milan offers various dance clubs and events catering to salsa and bachata enthusiasts.
**Florence** - A cultural hub, Florence has several dance studios and clubs where you can enjoy Latin dances.
**Naples** - Known for its passionate culture, Naples offers several venues and events for salsa and bachata lovers.
**Turin** - This northern city has a growing salsa community with events and social dances.
**Bologna** - Known for its lively student population, Bologna has a number of dance clubs and events for salsa and bachata.
**Venice** - While famous for its romantic canals, Venice also hosts various dance events throughout the year.
**Palermo** - In Sicily, Palermo has a vibrant Latin dance scene reflecting the island's festive culture.
**Verona** - Known for its romantic setting, Verona has several dance studios and clubs for salsa and bachata.
**Bari** - This coastal city in the south offers dance festivals and clubs perfect for salsa and bachata enthusiasts.

These cities offer a mix of cultural experiences and lively dance floors, ensuring you can enjoy salsa and bachata across Italy.
  

Interestingly enough, Naples does not top any of the lists, though the first four cities in the sea list are all close to Naples.

Licensing Details

Before closing the article, just two words on the license of Fibry. Fibry is no longer distributed as a pure MIT license. The main difference now is that if you want to build a system to generate code at scale for third parties (like a software engineer agent), you need a commercial license. Also, it is forbidden to include it in any datasets to train systems to generate code (e.g., ChatGPT should not be trained on the source code of Fibry). Anything else, you are good to go.

I can provide commercial support and develop features on demand.

Conclusion

I hope you had fun and could get an idea of how to use Fibry to write AI agents.

If you think that a multi-agent system needs to be distributed and run on multiple nodes, Fibry has got you covered! While we’ll save the details for another article, it’s worth noting that setting up Fibry actors in a distributed system is straightforward, and your agents are already actors: when you call process() or processAsync(), a message is sent to the underlying actor.

In Fibry, sending and receiving messages over the network is abstracted away, so you don’t even need to modify your agent logic to enable distribution. This makes Fibry uniquely simple for scaling across nodes without rewriting core logic.

Happy coding!

AI Java (programming language) systems

Published at DZone with permission of Luca Venturi. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

Trending