Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Building a Twitter Clone With Neo4j: Part V

DZone's Guide to

Building a Twitter Clone With Neo4j: Part V

The next step in building a Twitter clone with Neo4j is to build the back-end data service of a Twitter clone with Neo4j using Extensions to the existing Neo4j REST API.

· Database Zone
Free Resource

What if you could learn how to use MongoDB directly from the experts, on your schedule, for free? We've put together the ultimate guide for learning MongoDBSign up and you'll receive instructions for how to get started!

In part four, we continued cloning Twitter by adding hashtag and mentions functionality. Then, we went beyond that by adding the ability to edit a post. So, we have a social network where people can follow each other and post stuff. Today, we’re adding the ability to say a user likes a post, reposts a post, and, the most important query of all, is finally able to see our feed or timeline.

To create a like, we find the user liking and the author of the post, then we use the getPost method we created in the last blog post to find the post we want to like. Before we can add a LIKES relationship, we have to first check if one already exists. That’s what he userLikesPost method does below. I’m not going to show you that since we see the same code in removeLike a little bit later. From there, it is pretty straight forward. We create the LIKES relationship, add timestamps, and return the result.

@POST
@Path("/{username2}/{time}")
public Response createLike(@PathParam("username") final String username,
                           @PathParam("username2") final String username2,
                           @PathParam("time") final Long time,
                           @Context GraphDatabaseService db) throws IOException {
    Map<String, Object> results;
 
    try (Transaction tx = db.beginTx()) {
        Node user = Users.findUser(username, db);
        Node user2 = Users.findUser(username2, db);
        Node post = getPost(user2, time);
 
        if (userLikesPost(user, post)) {
            throw LikeExceptions.alreadyLikesPost;
        }
 
        Relationship like = user.createRelationshipTo(post, RelationshipTypes.LIKES);
        LocalDateTime dateTime = LocalDateTime.now(utc);
        like.setProperty(TIME, dateTime.toEpochSecond(ZoneOffset.UTC));
        results = post.getAllProperties();
        results.put(USERNAME, user2.getProperty(USERNAME));
        results.put(NAME, user2.getProperty(NAME));
        results.put(LIKES, post.getDegree(RelationshipTypes.LIKES));
        results.put(REPOSTS, post.getDegree(Direction.INCOMING)
                - 1 // for the Posted Relationship Type
                - post.getDegree(RelationshipTypes.LIKES)
                - post.getDegree(RelationshipTypes.REPLIED_TO));
        results.put(LIKED, true);
        results.put(REPOSTED, userRepostedPost(user, post));
        tx.success();
    }
    return Response.ok().entity(objectMapper.writeValueAsString(results)).build();
}

Now, our users are armed with the LIKES functionality. On Twitter, most posts don’t get any likes; on Facebook, however, it seems most posts have some likes. The alternative way to model likes is to skip the relationship altogether and use a set of bitmaps — one bitmap per user where we store the post node IDs they liked, and one bitmap per post where we store the user node IDs of the users who liked the post. We have seen an example of half of this before on a previous post on creating one-way relationships. If we used a bitmap, we’d lose the order of the likes. So, another alternative is to use a fast integer array compression library, which lets us keep the order, but not the LIKES times. We could use the Post times as a proxy. For now, we will continue with proper relationships.

We could also build the functionality to unlike a post. The missing “thumbs down” button in Facebook and “broken heart” in Twitter. It may be a useful feature when building a bulletin board system, but we’ll skip it for now. Instead, we need to add the ability to remove a like when somebody accidentally clicked on a like or later decided they didn’t actually like what the author had to say or what they linked to. We start off the same was as creating a like, but then must find the relationship between the user and the post. We are going to be smart about this by looking at all the likes of the user or all the likes of the post based on which one has the least amount of relationships and try to find the other node. Once we do, we can delete that relationship and we are good to go. The userLikesPost method mentioned earlier just returns true or false instead of deleting the relationship.

@DELETE
@Path("/{username2}/{time}")
public Response removeLike(@PathParam("username") final String username,
                           @PathParam("username2") final String username2,
                           @PathParam("time") final Long time,
                           @Context GraphDatabaseService db) throws IOException {
    boolean liked = false;
    try (Transaction tx = db.beginTx()) {
        Node user = Users.findUser(username, db);
        Node user2 = Users.findUser(username2, db);
        Node post = getPost(user2, time);
 
        if (user.getDegree(RelationshipTypes.LIKES, Direction.OUTGOING)
                < post.getDegree(RelationshipTypes.LIKES, Direction.INCOMING) ) {
            for (Relationship r1 : user.getRelationships(Direction.OUTGOING, RelationshipTypes.LIKES)) {
                if (r1.getEndNode().equals(post)) {
                    r1.delete();
                    liked = true;
                    break;
                }
            }
        } else {
            for (Relationship r1 : post.getRelationships(Direction.INCOMING, RelationshipTypes.LIKES)) {
                if (r1.getStartNode().equals(user)) {
                    r1.delete();
                    liked = true;
                    break;
                }
            }
        }
        tx.success();
    }
 
    if(!liked) {
        throw LikeExceptions.notLikingPost;
    }
 
    return Response.noContent().build();
}

We are almost there with our LIKES functionality. We also need to get all the likes of our user. This is going to flow just like our getFollowers method seen in part 3. From our user, we will traverse out the LIKES relationships. The “limit” and “since” will trim and order our output. I realized when I wrote this that I would need to account for seeing the LIKES of a user when logged on, and when browsing anonymously. To accomplish this we will add a “username2” parameter and with it find out if our browsing user has also liked or reposted the posts on our list.

@Path("/users/{username}/likes")
public class Likes {
 
    private static final ObjectMapper objectMapper = new ObjectMapper();
 
    @GET
    public Response getLikes(@PathParam("username") final String username,
                             @QueryParam("limit") @DefaultValue("25") final Integer limit,
                             @QueryParam("since") final Long since,
                             @QueryParam("username2") final String username2,
                             @Context GraphDatabaseService db) throws IOException {
        ArrayList<Map<String, Object>> results = new ArrayList<>();
        LocalDateTime dateTime;
        if (since == null) {
            dateTime = LocalDateTime.now(utc);
        } else {
            dateTime = LocalDateTime.ofEpochSecond(since, 0, ZoneOffset.UTC);
        }
        Long latest = dateTime.toEpochSecond(ZoneOffset.UTC);
 
        try (Transaction tx = db.beginTx()) {
            Node user = Users.findUser(username, db);
            Node user2 = null;
            if (username2 != null) {
                user2 = Users.findUser(username2, db);
            }
            for (Relationship r1 : user.getRelationships(Direction.OUTGOING, RelationshipTypes.LIKES)) {
                Node post = r1.getEndNode();
                Map<String, Object> properties = post.getAllProperties();
                Long time = (Long)r1.getProperty("time");
                if(time < latest) {
                    Node author = getAuthor(post, (Long)properties.get(TIME));
                    properties.put(LIKED_TIME, time);
                    properties.put(USERNAME, author.getProperty(USERNAME));
                    properties.put(NAME, author.getProperty(NAME));
                    properties.put(HASH, author.getProperty(HASH));
                    properties.put(LIKES, post.getDegree(RelationshipTypes.LIKES));
                    properties.put(REPOSTS, post.getDegree() - 1 - post.getDegree(RelationshipTypes.LIKES));
                    if (user2 != null) {
                        properties.put(LIKED, userLikesPost(user2, post));
                        properties.put(REPOSTED, userRepostedPost(user2, post));
                    }
                    results.add(properties);
                }
            }
            tx.success();
        }
 
        results.sort(Comparator.comparing(m -> (Long) m.get(LIKED_TIME), reverseOrder()));
 
        return Response.ok().entity(objectMapper.writeValueAsString(
                results.subList(0, Math.min(results.size(), limit))))
                .build();
    }

We’re going to have to add this LIKED and REPOSTED functionality to our getPosts  and getMentions methods, as well — really wherever we return a Post.

Speaking of posts, let's add the createRepost method. It’s pretty similar to our createLikes method, but instead of a straight forward LIKES relationships, we are going to be adding a dated REPOSTED_ON relationship.

@POST
@Path("/{username2}/{time}")
public Response createRepost(@PathParam("username") final String username,
                           @PathParam("username2") final String username2,
                           @PathParam("time") final Long time,
                           @Context GraphDatabaseService db) throws IOException {
    Map<String, Object> results;
 
    try (Transaction tx = db.beginTx()) {
        Node user = Users.findUser(username, db);
        Node user2 = Users.findUser(username2, db);
        Node post = getPost(user2, time);
 
        LocalDateTime dateTime = LocalDateTime.now(utc);
        if (userRepostedPost(user, post)) {
            throw PostExceptions.postAlreadyReposted;
        } else {
            Relationship r1 = user.createRelationshipTo(post, RelationshipType.withName("REPOSTED_ON_" +
                    dateTime.format(dateFormatter)));
            r1.setProperty(TIME, dateTime.toEpochSecond(ZoneOffset.UTC));
            results = post.getAllProperties();
            results.put(REPOSTED_TIME, dateTime.toEpochSecond(ZoneOffset.UTC));
            results.put(TIME, time);
            results.put(USERNAME, user2.getProperty(USERNAME));
            results.put(NAME, user2.getProperty(NAME));
            results.put(LIKES, post.getDegree(RelationshipTypes.LIKES));
            results.put(REPOSTS, post.getDegree(Direction.INCOMING)
                    - 1 // for the Posted Relationship Type
                    - post.getDegree(RelationshipTypes.LIKES)
                    - post.getDegree(RelationshipTypes.REPLIED_TO));
            results.put(LIKED, userLikesPost(user, post));
            results.put(REPOSTED, true);
 
        }
        tx.success();
    }
    return Response.ok().entity(objectMapper.writeValueAsString(results)).build();
}

Since we are creating a repost, we know the REPOSTED value is going to be true. However, I’ve been calling userRepostedPost to get REPOSTED without showing you what that looks like. It’s a bit ugly, I admit. Most posts will have few if any reposts, so if this is the case, we can traverse all its REPOSTED_ON_{date} relationships looking for the user. If, however, this is a very popular post, we will follow a different strategy. We will find the date of the post and check if the user reposted it on the same day it was posted. If not we will check the REPOSTED_ON_{date} relationship types from creation day forward until today and check the user or the post for a connection. We could optimize away this method by caching the results of the REPOSTED_ON_{date} relationships for either the user or the post. If the user only has a few reposts, we can even cache them all. Caching is an optimization strategy we can employ if needed, but don’t forget Neo4j can traverse millions of relationships per second, so it may not be necessary at all.

public static boolean userRepostedPost(Node user, Node post) {
    boolean alreadyReposted = false;
    LocalDateTime now = LocalDateTime.now(utc);
    LocalDateTime dateTime = LocalDateTime.ofEpochSecond((Long)post.getProperty(TIME), 0, ZoneOffset.UTC).truncatedTo(ChronoUnit.DAYS);
 
    if (post.getDegree(Direction.INCOMING) < 1000) {
        for (Relationship r1 : post.getRelationships(Direction.INCOMING)) {
            if (r1.getStartNode().equals(user) && r1.getType().name().startsWith("REPOSTED_ON_")) {
                alreadyReposted = true;
                break;
            }
        }
    }
 
    while (dateTime.isBefore(now) && !alreadyReposted) {
        RelationshipType repostedOn = RelationshipType.withName("REPOSTED_ON_" +
                dateTime.format(dateFormatter));
 
        if (user.getDegree(repostedOn, Direction.OUTGOING)
                < post.getDegree(repostedOn, Direction.INCOMING)) {
            for (Relationship r1 : user.getRelationships(Direction.OUTGOING, repostedOn)) {
                if (r1.getEndNode().equals(post)) {
                    alreadyReposted = true;
                    break;
                }
            }
        } else {
            for (Relationship r1 : post.getRelationships(Direction.INCOMING, repostedOn)) {
                if (r1.getStartNode().equals(user)) {
                    alreadyReposted = true;
                    break;
                }
            }
        }
        dateTime = dateTime.plusDays(1);
    }
    return alreadyReposted;
}

…and after all that, I think we are ready for the timeline query.

About 100 lines of Java goodness taking us from the user to all of the people the user follows, to all of their posts and reposts on a set of dates — ordered by date, and limited to a specified value for pagination. Here it is in all its splendor:

@GET
public Response getTimeline(@PathParam("username") final String username,
                         @QueryParam("limit") @DefaultValue("100") final Integer limit,
                         @QueryParam("since") final Long since,
                         @Context GraphDatabaseService db) throws IOException {
    ArrayList<Map<String, Object>> results = new ArrayList<>();
    LocalDateTime dateTime;
    if (since == null) {
        dateTime = LocalDateTime.now(utc);
    } else {
        dateTime = LocalDateTime.ofEpochSecond(since, 0, ZoneOffset.UTC);
    }
    Long latest = dateTime.toEpochSecond(ZoneOffset.UTC);
 
    try (Transaction tx = db.beginTx()) {
        Node user = Users.findUser(username, db);
        HashSet<Long> seen = new HashSet<>();
        ArrayList<Node> follows = new ArrayList<>();
        follows.add(user); // Adding user to see their posts on timeline as well
        for (Relationship r : user.getRelationships(Direction.OUTGOING, RelationshipTypes.FOLLOWS)) {
            follows.add(r.getEndNode());
        }
 
        LocalDateTime earliest = LocalDateTime.ofEpochSecond((Long)user.getProperty(TIME), 0, ZoneOffset.UTC);
 
        while (seen.size() < limit && (dateTime.isAfter(earliest))) {
            RelationshipType posted = RelationshipType.withName("POSTED_ON_" +
                    dateTime.format(dateFormatter));
            RelationshipType reposted = RelationshipType.withName("REPOSTED_ON_" +
                    dateTime.format(dateFormatter));
 
            for (Node follow : follows) {
                Map followProperties = follow.getAllProperties();
 
                for (Relationship r1 : follow.getRelationships(Direction.OUTGOING, posted)) {
                    Node post = r1.getEndNode();
                    if(seen.add(post.getId())) {
                        Long time = (Long)r1.getProperty("time");
                        Map<String, Object> properties = r1.getEndNode().getAllProperties();
                        if (time < latest) {
                            properties.put(TIME, time);
                            properties.put(USERNAME, followProperties.get(USERNAME));
                            properties.put(NAME, followProperties.get(NAME));
                            properties.put(HASH, followProperties.get(HASH));
                            properties.put(LIKES, post.getDegree(RelationshipTypes.LIKES));
                            properties.put(REPOSTS, post.getDegree(Direction.INCOMING)
                                    - 1 // for the Posted Relationship Type
                                    - post.getDegree(RelationshipTypes.LIKES)
                                    - post.getDegree(RelationshipTypes.REPLIED_TO));
                            properties.put(LIKED, userLikesPost(user, post));
                            properties.put(REPOSTED, userRepostedPost(user, post));
                            results.add(properties);
                        }
                    }
                }
 
                for (Relationship r1 : follow.getRelationships(Direction.OUTGOING, reposted)) {
                    Node post = r1.getEndNode();
                    if(seen.add(post.getId())) {
                        Map<String, Object> properties = r1.getEndNode().getAllProperties();
                        Long reposted_time = (Long)r1.getProperty(TIME);
                        if (reposted_time < latest) {
                            properties.put(REPOSTED_TIME, reposted_time);
                            properties.put(REPOSTER_USERNAME, followProperties.get(USERNAME));
                            properties.put(REPOSTER_NAME, followProperties.get(NAME));
                            properties.put(HASH, followProperties.get(HASH));
                            properties.put(LIKES, post.getDegree(RelationshipTypes.LIKES));
                            properties.put(REPOSTS, post.getDegree(Direction.INCOMING)
                                    - 1 // for the Posted Relationship Type
                                    - post.getDegree(RelationshipTypes.LIKES)
                                    - post.getDegree(RelationshipTypes.REPLIED_TO));
                            properties.put(LIKED, userLikesPost(user, post));
                            properties.put(REPOSTED, userRepostedPost(user, post));
 
                            Node author = getAuthor(post, (Long)properties.get(TIME));
                            properties.put(USERNAME, author.getProperty(USERNAME));
                            properties.put(NAME, author.getProperty(NAME));
                            results.add(properties);
                        }
                    }
                }
            }
            dateTime = dateTime.minusDays(1);
        }
        tx.success();
    }
 
    results.sort(Comparator.comparing(m -> (Long) m.get("time"), reverseOrder()));
 
    return Response.ok().entity(objectMapper.writeValueAsString(
            results.subList(0, Math.min(results.size(), limit))))
            .build();

…and that is how you build the back-end data service of a Twitter clone with Neo4j using Extensions to the existing Neo4j REST API. We’ll look at building our front end in an upcoming post.

What if you could learn how to use MongoDB directly from the experts, on your schedule, for free? We've put together the ultimate guide for learning MongoDBSign up and you'll receive instructions for how to get started!

Topics:
twitter ,neo4j ,tutorial ,database

Published at DZone with permission of Max De Marzi, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}