Over a million developers have joined DZone.

JAX RS: Streaming a Response using StreamingOutput

· Java Zone

Microservices! They are everywhere, or at least, the term is. When should you use a microservice architecture? What factors should be considered when making that decision? Do the benefits outweigh the costs? Why is everyone so excited about them, anyway?  Brought to you in partnership with IBM.

A couple of weeks ago Jim and I were building out a neo4j unmanaged extension from which we wanted to return the results of a traversal which had a lot of paths.

Our code initially looked a bit like this:

package com.markandjim

@Path("/subgraph")
public class ExtractSubGraphResource {
    private final GraphDatabaseService database;

    public ExtractSubGraphResource(@Context GraphDatabaseService database) {
        this.database = database;
    }

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    @Path("/{nodeId}/{depth}")
    public Response hello(@PathParam("nodeId") long nodeId, @PathParam("depth") int depth) {
        Node node = database.getNodeById(nodeId);

        final Traverser paths =  Traversal.description()
                .depthFirst()
                .relationships(DynamicRelationshipType.withName("whatever"))
                .evaluator( Evaluators.toDepth(depth) )
                .traverse(node);

        StringBuilder allThePaths = new StringBuilder();

        for (org.neo4j.graphdb.Path path : paths) {
            allThePaths.append(path.toString() + "\n");
        }

        return Response.ok(allThePaths.toString()).build();
    }
}

We then compiled that into a JAR, placed it in ‘plugins’ and added the following line to ‘conf/neo4j-server.properties’:

org.neo4j.server.thirdparty_jaxrs_classes=com.markandjim=/unmanaged

After we’d restarted the neo4j server we were able to call this end point using cURL like so:

$ curl -v  http://localhost:7474/unmanaged/subgraph/1000/10

This approach works quite well but Jim pointed out that it was quite inefficient to load all those paths up into memory so we thought it would be quite cool if we could stream it as we got to each path. Traverser wraps an iterator so we are lazily evaluating the result set in any case.

After a bit of searching we came StreamingOutput which is exactly what we need. We adapted our code to use that instead:

package com.markandjim

@Path("/subgraph")
public class ExtractSubGraphResource {
    private final GraphDatabaseService database;

    public ExtractSubGraphResource(@Context GraphDatabaseService database) {
        this.database = database;
    }

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    @Path("/{nodeId}/{depth}")
    public Response hello(@PathParam("nodeId") long nodeId, @PathParam("depth") int depth) {
        Node node = database.getNodeById(nodeId);

        final Traverser paths =  Traversal.description()
                .depthFirst()
                .relationships(DynamicRelationshipType.withName("whatever"))
                .evaluator( Evaluators.toDepth(depth) )
                .traverse(node);

        StreamingOutput stream = new StreamingOutput() {
            @Override
            public void write(OutputStream os) throws IOException, WebApplicationException {
                Writer writer = new BufferedWriter(new OutputStreamWriter(os));

                for (org.neo4j.graphdb.Path path : paths) {
                    writer.write(path.toString() + "\n");
                }
                writer.flush();
            }
        };

        return Response.ok(stream).build();
    }

As far as I can tell the only discernible difference between the two approaches is that you get an almost immediate response from the streamed approached whereas the first approach has to put everything in the StringBuilder first.

Both approaches make use of chunked transfer encoding which according to tcpdump seems to have a maximum packet size of 16332 bytes:

00:10:27.361521 IP localhost.7474 > localhost.55473: Flags [.], seq 6098196:6114528, ack 179, win 9175, options [nop,nop,TS val 784819663 ecr 784819662], length 16332

00:10:27.362278 IP localhost.7474 > localhost.55473: Flags [.], seq 6147374:6163706, ack 179, win 9175, options [nop,nop,TS val 784819663 ecr 784819663], length 16332

Discover how the Watson team is further developing SDKs in Java, Node.js, Python, iOS, and Android to access these services and make programming easy. Brought to you in partnership with IBM.

Topics:
java ,jax rs ,neo4j

Published at DZone with permission of Mark Needham, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}