Building a ChatBot in Neo4j (Part Three)

You may also like: Building a ChatBot in Neoj4

In part one, we learned to listen to our users, in part two we began learning how to talk back. Before we go any further into the stored procedure, how about we build a little front end to show off the work we've done so far on this proof of concept? That will also make things easier to test out and let us get into the mindset of the user.

There are a ton of options here, lots of folks like Spring and Spring Boot. Others are more hipsters and into Micronaut. I am even more of a hipster and prefer to use Jooby, but it doesn't matter. We'll be using Cypher, the Neo4j Drivers and the Stored Procedures we build along the way so technically you can do this in just about any language.

I'm not a designer, so I searched around online for a simple chat platform theme and found " Swipe: The Simplest Chat Platform" for $19 bucks. That will save me a ton of time messing with CSS which I consider the dark magic of web development.

Jooby projects start pretty simple, follow along with the documentation and you will end up with something like this:

import io.jooby.Jooby;

public class App extends Jooby {

  {
    get("/", ctx -> "Welcome to Jooby!");
  }

  public static void main(String[] args) {
    runApp(args, App::new);
  }
}

We don't want to welcome people to Jooby, we want to let them try out our chatbot application so let's change that text to instead ask the user to sign in or register instead:

Before we get much further, we need to be able to talk to Neo4j. We need to tell it that a user is trying to register for that we will add the Neo4j Java driver to the pom.xml file and build a little Jooby extension. The Neo4jExtension will connect to Neo4j using a configured URI, username and password, then register the driver on to the application. It looks like this:

public class Neo4jExtension implements Extension {

    @Override
    public void install(@Nonnull Jooby application) throws Exception {
        Environment env = application.getEnvironment();
        Config conf = env.getConfig();

        Driver driver = GraphDatabase.driver(conf.getString("neo4j.uri"),
                AuthTokens.basic(conf.getString("neo4j.username"), 
                                 conf.getString("neo4j.password")));

        ServiceRegistry registry = application.getServices();
        registry.put(Driver.class, driver);
        application.onStop(driver);
    }

}

Now we need to write the post register endpoint. Our form is passing in an id, a password and a phone number for our user. We will encrypt the password, require the Neo4j Driver and use it to send a Cypher Query to Neo4j to create the user.

        post("/register", ctx -> {
            Formdata form = ctx.form();
            String id = form.get("id").toOptional().orElse("");
            String password = form.get("password").toOptional().orElse("");
            String phone = form.get("phone").toOptional().orElse("");
            password = BCrypt.hashpw(password, BCrypt.gensalt());
            Driver driver = require(Driver.class);
            Map<String, Object> user = CypherQueries.CreateMember(driver, id, phone, password);

Some people like having Cypher queries all over their codebase. I kinda like having them mostly huddled together. So we'll create a CypherQueries interface where will stick them for now. We'll need a few helper methods as well. One to create a session from our driver and execute the query given a set of parameters, and return an iterator of maps to make things easier to work with:

    static Iterator<Map<String, Object>> query(Driver driver, String query, 
                                               Map<String, Object> params) {
        try (Session session = driver.session()) {
            List<Map<String, Object>> list = session.run(query, params)
                    .list( r -> r.asMap(CypherQueries::convert));
            return list.iterator();
        }
    }

The convert method to bring back everything back in a way that is easily convertible to a map:

    static Object convert(Value value) {
        switch (value.type().name()) {
            case "PATH":
                return value.asList(CypherQueries::convert);
            case "NODE":
            case "RELATIONSHIP":
                return value.asMap();
        }
        return value.asObject();
    }

With that plumbing out of the way, we can get down to what's needed. A cipher query to create the account and member:

String createMember = "CREATE (a:Account { id: $id, password: $password })
                              -[:HAS_MEMBER]->(member:Member { phone: $phone }) 
RETURN a.id AS id, member.phone AS phone";

...and a method to tie things together.

    static Map<String, Object> CreateMember(Driver driver, String id, 
                                            String phone, String password) {
        Map<String, Object> response = Iterators.singleOrNull(query(driver, createMember,
                new HashMap<String, Object>() {{
                    put("id", id);
                    put("phone", phone);
                    put("password", password); }}
            ));

If you take a close look at that picture, you'll see that the member node has a whole bunch of properties we didn't ask for. Like Name, Location, Gender, etc. What gives? Well, I don't want to waste the user's time asking them things I can figure out on my own. So I wrote another extension to call the FullContact API with the email and phone number used in the registration to enrich that members' information. One of the nice things about working with Neo4j is that its schema optional. So whatever properties I can get from FullContact I can add to the member via this cipher query:

String enrichUser = "MATCH (a:Account)-[:HAS_MEMBER]->(member) 
                     WHERE a.id = $id AND member.phone = $phone 
                     SET member += $properties 
                     RETURN member";

But there are some caveats to that. First, Neo4j doesn't allow null values, so we have to get rid of those. Second, Neo4j doesn't allow nested properties and both the "details" and "dataAddOns" come in as JSON blobs, so they have to go.

properties.values().removeIf(Objects::isNull);
properties.remove("details");
properties.remove("dataAddOns");

Third, the FullContact API has a rate limiter (especially for the free plan) so instead of doing the request at registration, it is sent to a queue that runs a background job once a second so we don't exceed the limit.

enrichmentJob.queue.add(new HashMap<String, Object>() {{
    put("email", id);
    put("phone", phone);
}});

Now we need to be able to use the stored procedure to chat with our user. We can call it just like any Cypher query from the driver, passing in the id and phone of the member as well as the text they sent us:

    String chat = "CALL com.maxdemarzi.chat($id, $phone, $text)";

    static List<Map<String, Object>> Chat(Driver driver, String id, String phone, String text) {
        return Iterators.asList(
            query(driver, chat, new HashMap<String, Object>() {{
                put("id", id);
                put("phone", phone);
                put("text", text);
            }})
        );
    }

We'll add an endpoint to accept their chat post request to our Jooby application that uses the method above and returns the response to our app.

        post("/chat", ctx -> {
            String id = ctx.session().get("id").value();
            String phone = ctx.session().get("phone").value();
            Formdata form = ctx.form();
            String chatText = form.get("chatText").toOptional().orElse("");
            Driver driver = require(Driver.class);
            List<Map<String, Object>> response = CypherQueries.Chat(driver, id, phone, chatText);
            return response;
        });

We will also wire it all together with some old school Javascript because that's all I remember how to do and now for the fruits of our labor. When I type "hello" into the chatbox and press enter, the stored procedure is called and it attaches a new Message to our Member node, figures out what to reply, adds that to our graph and returns the result.

Then we can visualize our reply in the chat window:

...and there we have it. I took some effort to get this far. In the next part, we'll go beyond saying hello and get to some real functionality. The source code, as always, is on Github.

Further Reading

Building a Chatbot in Neo4j (Part Two)

How to Develop a Chatbot From Scratch

Information and Document Chatbot