Client side paging of Server Paged OData Entity Sets using dataJS

Standard

I am back with one more article today. Had a little halt to my writings as I moved to my new house and was busy setting up things including my internet connection and all. Today I will be talking about an interesting concept in OData world. So sit back, read and enjoy.

 

Objective/Goal:

Odata, if you are not familiar is an open data exchange protocol from Microsoft released under Open Specification Promise. Uses the HTTP as backbone with all its VERBS for handling CRUD functionality, adheres to ATOM/ATOMPUB formats for read and write and adds Queryability feature to the consumers. More on OData here.

Since OData provide the complete control over the data to the consumers, it may often happen that the client may not restrict the data read in terms of number of records to fetch. For e.g. if you have a products catalog with lets say 10K products and you have this set exposed as a OData feed, if a client access this set without any record limit, the service is going to return all 10K product rows. As you can see there two things we need to tackle here:

- How do we avoid the situation of a client requesting without a page size – as a Producer.

- How do we know what is the next page for the entity set, if it has a page size set at server – as a Consumer.

Lets try to answer the above questions one by one

Background:

For more information on the concept of Page Size limit, do have a read of the following blog post from WCF Data Services Team:

http://blogs.msdn.com/b/astoriateam/archive/2010/02/02/server-paging-in-data-services.aspx

The article talks about .NET clients but the concept of the server side entity set page size is the highlight there. Rest of the article is based on those understandings.

 

Server Side Set Entity Page Size:

The producers of an Odata service can put a page size limit on the server side itself. Following is the code which sets the page size limit on a data set

   1:  config.SetEntitySetPageSize("Products", 20); 

The DataServiceConfiguration object provides SetEntitySetPageSize() method which takes the Entity Set name and the page size. In the above code Products Entity Set is set to a page size of 20. This means anytime a request for Products is made the server will only send  20 records at a time. That’s good we are not sending too many payloads now and there is a check in place. But what really happens to the entity set feed when we put a page size limit on the server. For this example I will be using the following OData service which is available for anybody to consume.

http://services.odata.org/Northwind/Northwind.svc/

This OData service is of the classic Northwind Database we all are familiar with. The service exposes Products Entity Set and it has a page size limit set to 20. Here is the URI for the Products Feed:

http://services.odata.org/Northwind/Northwind.svc/Products

When we request for the Products entity set we will get the feed for the Products Entity Set. Now if we observer closely the feed, we will see the following information at the end of the feed:

   1:   <link rel="next" href="http://services.odata.org/Northwind/Northwind.svc/Products?$skiptoken=20" /> 
This is a familiar <link> tag and pay attention to the “rel” attribute. It has a value of “next”. And notice the “href” attribute. It has a url pointing to the Products entity set but with an additional odata construct called “$skiptoken”. This is how Producers let the Consumer know that here is the URL you need to navigate to get the next page. All it means to the server is skip the 20 records and then return the rest. If I navigate to the above URL and we still have more records, I would get back a <link> tag as follows:
   1:  <link rel="next" href="http://services.odata.org/Northwind/Northwind.svc/Products?$skiptoken=40" /> 
Notice the skip token has a value of 40. That means 3rd page will be served by skipping 40 records and returning the rest from there.
So this is all fine from human eye perspective. But how to achieve these from lets say a client side application. Lets see that in the next section.
Looping through pages from Client Side:
In previous section we understood how the server page size limit is represented in the odata feeds. Now lets see how a client side app can make use of the same. For this example we will be doing the following:
- Access Products Entity Set from http://services.odata.org/Northwind/Northwind.svc/
- Create a HTML page and create a Tab per Page of data feed read
- List out products in each tab
Here is the screen shot of the finished solution:
image
As usual I will be using the dataJS library for reading the OData. This example we will focus on dataJS. I will follow up with other examples such as – using JQuery to page, within .NET client apps etc.

So lets walkthrough the example:

HTML Markup:

   1:  <div>
   2:     <button id="btnGetData">
   3:        List Products</button>
   4:        <span id="loading">Fetching data...</span>
   5:  </div>
   6:  <div id="tabs">
   7:     <ul>
   8:     </ul>
   9:  </div>

Nothing fancy, I have a button to list the products. Then I have a div to create the tabs. I will be using the JQuery tabs widget to achieve this.

Product Listing Template:

   1:   <script type="x-jquery-tmpl" id="resultsTemplate">
   2:          <ul>
   3:              {{each results}}
   4:                  <li style="margin:.5em">
   5:                      {{if $value.Discontinued }}
   6:                          <span class='discontinued'><b>${$value.ProductName}<i> ($${$value.UnitPrice})</i></b></span>
   7:                      {{else}}
   8:                          <span class='productName'><b>${$value.ProductName}<i> ($${$value.UnitPrice})</i></b></span>
   9:                      {{/if}}
  10:                  </li>
  11:              {{/each}}
  12:          </ul>
  13:      </script>

Again nothing fancy, I have a unordered list <ul> and output each product as a <li>. If the product is discontinued I use a style to show it in red.

Scripts:

On DOM ready, I wire up the button click to a handler:

   1:  $(getDataButton).button().click(OnGetDataClick);

Now on click of the button “List Products”, use dataJS to make the service call to Products entity set as follows:

   1:  function MakeRequest(url) 
   2:  {
   3:      OData.read(url, successHandler, errorHandler);
   4:  };

We make use of dataJS read API to read the data. successHandler is the callback for success and errorHandler is the callback if in case of any errors.

Now all the magic happens in the successHandler. Here is the code for the successHandler:

   1:  function successHandler(data) 
   2:  {
   3:      var tab_title = "Catalog Page " + tab_counter;
   4:      $(tabsDiv).tabs("add", "#tabs-" + tab_counter, tab_title);
   5:      $(resultsTemplate).tmpl(data).appendTo("#tabs-" + tab_counter);
   6:      $(tabsDiv).show();
   7:      if (data.__next) {
   8:         tab_counter++;
   9:         MakeRequest(data.__next);
  10:      }
  11:      else {
  12:         $(getDataButton).button("enable");
  13:         $(loadingDiv).hide();
  14:      }
  15:  }

Line 4 creates a new tab. The meat of the work is on line 7. we are checking if there is a “next” link available for the feed. If yes we know that we have one more page of data to read. Remember from previous section that “next” is actually a URI to the next page. So we now make use of that URI and we give a request again. We increase the tab counter and a new tab is created on the success. This continues till all the pages are read. And we have a nice looking app which loops through all the pages and creates a nice lookin tabs for each page.

With OData + dataJS + JQuery its super simple to create a data centric applications without much effort.



Conclusion:

- If you are an OData producer don’t forget to set Entity Set Page Size limit

- If you are a Consumer, do look out for “next” link in the feed. Make use of this and loop accordingly through all the pages

- Recommend using dataJS for all OData service related reads and writes.

A live version of this example can be found at the following URL:

http://odataplayground.99on.com/clientsidepaging/index.html

Just view the source and you will get the source code :).

Hope I was able to convey the idea of setting page size limit and looping through those pages.


As usual, till next time – Happy Coding. Code with passion, Decode with patience

  • Vivek

    Nice post