REST Pagination in Spring

This is the seventh of a series of articles about setting up a secure RESTful Web Service using Spring 3.1 and Spring Security 3.1 with Java based configuration. This article will focus on the implementation of pagination in a RESTful web service.

The REST with Spring series:

• Part 1 – Bootstrapping a web application with Spring 3.1 and Java based Configuration
• Part 2 – Building a RESTful Web Service with Spring 3.1 and Java based Configuration
• Part 3 – Securing a RESTful Web Service with Spring Security 3.1
• Part 4 – RESTful Web Service Discoverability
• Part 5 – REST Service Discoverability with Spring
• Part 6 – Basic and Digest authentication for a RESTful Service with Spring Security 3.1

You can check out the entire REST with Spring Series here.

Page as resource vs Page as representation

The first question when designing pagination in the context of a RESTful architecture is whether to consider the page an actual resource or just a representation of resources. Treating the page itself as a resource introduces a host of problems such as no longer being able to uniquely identify resources between calls. This, coupled with the fact that outside the RESTful context, the page cannot be considered a proper entity, but a holder that is constructed when needed makes the choice straightforward: the page is part of the representation.

The next question in the pagination design in the context of REST is where to include the paging information:

• in the URI path: /foo/page/1
• the URI query: /foo?page=1

Keeping in mind that a page is not a resource, encoding the page information in the URI is no longer an option.

Page information in the URI query

Encoding paging information in the URI query is the standard way to solve this issue in a RESTful service. This approach does however have one downside – it cuts into the query space for actual queries:

/foo?page=1&size=10

The Controller

Now, for the  implementation – the Spring MVC Controller for pagination is straightforward:

@RequestMapping( value = "admin/foo",params = { "page", "size" },method = GET )
@ResponseBody
public List< Foo > findPaginated(
 @RequestParam( "page" ) int page, @RequestParam( "size" ) int size,
 UriComponentsBuilder uriBuilder, HttpServletResponse response ){

   Page< Foo > resultPage = service.findPaginated( page, size );
   if( page > resultPage.getTotalPages() ){
      throw new ResourceNotFoundException();
   }
   eventPublisher.publishEvent( new PaginatedResultsRetrievedEvent< Foo >
    ( Foo.class, uriBuilder, response, page, resultPage.getTotalPages(), size ) );

   return resultPage.getContent();
}

The two query parameters are defined in the request mapping and injected into the controller method via @RequestParam; the HTTP response and the Spring UriComponentsBuilder are injected in the Controller method to be included in the event, as both will be needed to implement discoverability.

Discoverability for REST pagination

Withing the scope of pagination, satisfying the HATEOAS constraint of REST means enabling the client of the API to discover the next and previous pages based on the current page in the navigation. For this purpose, the Link HTTP header will be used, coupled with the official “next“, “prev“, “first” and “last” link relation types.

In REST, Discoverability is a cross cutting concern, applicable not only to specific operations but to types of operations. For example, each time a Resource is created, the URI of that resource should be discoverable by the client. Since this requirement is relevant for the creation of ANY Resource, it should be dealt with separately and decoupled from the main Controller flow.

With Spring, this decoupling is achieved with events, as was thoroughly discussed in the previous article focusing on Discoverability of a RESTful service. In the case of pagination, the event – PaginatedResultsRetrievedEvent – was fired in the Controller, and discoverability is achieved in a listener for this event:

void addLinkHeaderOnPagedResourceRetrieval(
 UriComponentsBuilder uriBuilder, HttpServletResponse response,
 Class clazz, int page, int totalPages, int size ){

   String resourceName = clazz.getSimpleName().toString().toLowerCase();
   uriBuilder.path( "/admin/" + resourceName );

   StringBuilder linkHeader = new StringBuilder();
   if( hasNextPage( page, totalPages ) ){
      String uriNextPage = constructNextPageUri( uriBuilder, page, size );
      linkHeader.append( createLinkHeader( uriForNextPage, REL_NEXT ) );
   }
   if( hasPreviousPage( page ) ){
      String uriPrevPage = constructPrevPageUri( uriBuilder, page, size );
      appendCommaIfNecessary( linkHeader );
      linkHeader.append( createLinkHeader( uriForPrevPage, REL_PREV ) );
   }
   if( hasFirstPage( page ) ){
      String uriFirstPage = constructFirstPageUri( uriBuilder, size );
      appendCommaIfNecessary( linkHeader );
      linkHeader.append( createLinkHeader( uriForFirstPage, REL_FIRST ) );
   }
   if( hasLastPage( page, totalPages ) ){
      String uriLastPage = constructLastPageUri( uriBuilder, totalPages, size );
      appendCommaIfNecessary( linkHeader );
      linkHeader.append( createLinkHeader( uriForLastPage, REL_LAST ) );
   }
   response.addHeader( HttpConstants.LINK_HEADER, linkHeader.toString() );
}

In short, the listener logic checks if the navigation allows for a next, previous, first and last pages and, if it does, adds the relevant URIs to the Link HTTP Header. It also makes sure that the link relation type is the correct one – “next”, “prev”, “first” and “last”. This is the single responsibility of the listener (the full code here).

Test Driving Pagination

Both the main logic of pagination and discoverability should be extensively covered by small, focused integration tests; as in the previous article, the rest-assured library is used to consume the REST service and to verify the results.

These are a few example of pagination integration tests; for a full test suite, check out the github project (link at the end of the article):

@Test
public void whenResourcesAreRetrievedPaged_then200IsReceived(){
   Response response = givenAuth().get( paths.getFooURL() + "?page=1&size=10" );

   assertThat( response.getStatusCode(), is( 200 ) );
}
@Test
public void
 whenPageOfResourcesAreRetrievedOutOfBounds_then404IsReceived(){
   Response response = givenAuth().get(
    paths.getFooURL() + "?page=" + randomNumeric( 5 ) + "&size=10" );

   assertThat( response.getStatusCode(), is( 404 ) );
}
@Test
public void
 givenResourcesExist_whenFirstPageIsRetrieved_thenPageContainsResources(){
   restTemplate.createResource();

   Response response = givenAuth().get( paths.getFooURL() + "?page=1&size=10" );

   assertFalse( response.body().as( List.class ).isEmpty() );
}

Test Driving Pagination Discoverability

Testing Discoverability of Pagination is relatively straightforward, although there is a lot of ground to cover. The tests are focused on the position of the current page in navigation and the different URIs that should be discoverable from each position:

@Test
public void whenFirstPageOfResourcesAreRetrieved_thenSecondPageIsNext(){
   Response response = givenAuth().get( paths.getFooURL()+"?page=0&size=10" );

   String uriToNextPage = extractURIByRel( response.getHeader( LINK ), REL_NEXT );
   assertEquals( paths.getFooURL()+"?page=1&size=10", uriToNextPage );
}
@Test
public void whenFirstPageOfResourcesAreRetrieved_thenNoPreviousPage(){
   Response response = givenAuth().get( paths.getFooURL()+"?page=0&size=10" );

   String uriToPrevPage = extractURIByRel( response.getHeader( LINK ), REL_PREV );
   assertNull( uriToPrevPage );
}
@Test
public void whenSecondPageOfResourcesAreRetrieved_thenFirstPageIsPrevious(){
   Response response = givenAuth().get( paths.getFooURL()+"?page=1&size=10" );

   String uriToPrevPage = extractURIByRel( response.getHeader( LINK ), REL_PREV );
   assertEquals( paths.getFooURL()+"?page=0&size=10", uriToPrevPage );
}
@Test
public void whenLastPageOfResourcesIsRetrieved_thenNoNextPageIsDiscoverable(){
   Response first = givenAuth().get( paths.getFooURL()+"?page=0&size=10" );
   String uriToLastPage = extractURIByRel( first.getHeader( LINK ), REL_LAST );

   Response response = givenAuth().get( uriToLastPage );

   String uriToNextPage = extractURIByRel( response.getHeader( LINK ), REL_NEXT );
   assertNull( uriToNextPage );
}

These are just a few examples of integration tests consuming the RESTful service.

Getting All Resources

On the same topic of pagination and discoverability, the choice must be made if a client is allowed to retrieve all the Resources in the system at once, or if the client MUST ask for them paginated.

If the choice is made that the client cannot retrieve all Resources with a single request, and pagination is not optional but required, then several options are available for the response to a get all request.

One option is to return a 404 (Not Found) and use the Link header to make the first page discoverable:

Link=<http://localhost:8080/rest/api/admin/foo?page=0&size=10>; rel=”first“, <http://localhost:8080/rest/api/admin/foo?page=103&size=10>; rel=”last“

Another option is to return redirect – 303 (See Other) – to the first page of the pagination.

A third option is to return a 405 (Method Not Allowed) for the GET request.

REST Paginag with Range HTTP headers

A relatively different way of doing pagination is to work with the HTTP Range headers – Range, Content-Range, If-Range, Accept-Ranges – and HTTP status codes – 206 (Partial Content), 413 (Request Entity Too Large), 416 (Requested Range Not Satisfiable). One view on this approach is that the HTTP Range extensions were not intended for pagination, and that they should be managed by the Server, not by the Application.

Implementing pagination based on the HTTP Range header extensions is nevertheless technically possible, although not nearly as common as the implementation discussed in this article.

Conclusion

This article covered the implementation of Pagination in a RESTful service with Spring, discussing how to implement and test Discoverability. For a full implementation of pagination, check out the github project.

From the original REST Pagination in Spring of the REST with Spring series