In the past, we've taught you how to easily render a data set in a Kendo UI Grid within any Progress Rollbase page. We also showed you how to leverage the smart automatic responsive system. With this blog, we look at how we can use more advanced features to render highly performant grids when you have to work with a large dataset. 

If you are not familiar with Rollbase, don’t be mistaken that you have to code these grids for all your objects. Rollbase provides out-of-the-box (with no coding), highly scalable Grid rendering. That is, as soon as you create an object definition with some fields, you get an auto-generated page with a grid (we call it the list view), and in the page designer, you get a list view widget that you can drag in any section. For a better introduction to Rollbase, our low-code platform, check out this older blog.

Grids With Large Data Sets

Before we get started, please make sure you understand the techniques covered in previous entries. In this blog, we'll simply highlight the specifics to working with large data set.

The key to dealing with large data sets is not transmitting the entire data set to the client. To this end, we want to leverage server-side paging. If you have never worked with server-side paging, this is simply a technique to deliver a subset of the data (a page) by specifying where to begin and how many records to get in one chunk. Subsequently, you can request the next page or any arbitrary page.

You turn server side paging by setting the property serverPaging: true in the data source.

Next, we need to specify a transport read function. The function will be called with one object containing the characteristics of the page you are requesting. For example, the initial request will say start at 0 and get me 20 records. The next request will say start at 21 and get me 20 records.

Now for the big question: how do we use these parameters to efficiently request the data from the Rollbase server? The answer is simple: use the client-side function rbf_selectQuery2.

This function is similar to rbf_selectQuery but it lets you specify the record number at which to start and the number of records to retrieve on the server side. Thus, we can use the parameters from the transport read function directly in the rbf_selectQuery2 function, as outlined in the code sample below.

transport: {
     read: function(options) {
         var
 

records=[];
         …
         sortedQuery += getSQLSortCommand ( options.data );
         rbf_selectQuery2(sortedQuery, options.data.skip, options.data.pageSize,                function(returnData){
                 createAllRecordsObjects( returnData, records );
                 options.success( {total: totalCountRead, data: records} );
             });
         }
     }
}

More details on this function can be found online in our documentation.

Sorting

You cannot sort your data set on the client side since you don’t have all the data. That means you have to do it on the server side. 

Part one of this is achieved by simply setting the following property in the data source: serverSorting: true.

Part two consists in creating the proper SQL query based on how the user is interacting with the grid.

Here again, Kendo helps us by providing access to the current sort states. That is, which columns the user has clicked on to sort (we are getting an array of states as users may sort multiple columns). Here is a key code extract showing how to access this data:         

var sortOptions = options.data;

var sortCommand = " ORDER BY ";
var dir = sortOptions.sort[0].dir;
if ( dir === undefined || dir === null )
   return "";
else {
    var col = sortOptions.sort[0].field;
    sortCommand += col;
    sortCommand += " ";
    sortCommand += dir;

    return sortCommand;

}

Line 4, we get the sort direction. The string we get is directly usable in a SQL query.

And Line 8, we get the column name by which we need to sort.

Also, note in Line 5 and 6 how we detect if the user has clicked a third time on the column, where we need to reset to no sorting.

We return a string that we can use in the rbf_selectQuery2.

In conclusion, we learned how the client-side API rbf_selectQuery2 provides a powerful tool to do server-side paging and sorting. When combined with Kendo Grid, for example, we can implement custom grids that scale to a very large data set.