Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Tutorial: How a Web Application Can Consume Data From a Web API

DZone's Guide to

Tutorial: How a Web Application Can Consume Data From a Web API

Zone Leader John Vester takes on his first bounty - using a DZone API and a JavaScript web framework to consume, manipulate and present data regarding his statistics writing for DZone.

· Web Dev Zone ·
Free Resource

Deploy code to production now. Release to users when ready. Learn how to separate code deployment from user-facing feature releases with LaunchDarkly.

This November will mark my second anniversary as a Zone Leader for DZone.  Writing six to eight articles a month has been a thrilling experience - allowing me to continue learning in our ever-changing world of Information Technology.

A few weeks ago on our Slack channel, one of the other Zone Leaders mentioned finding a RESTful end-point that provides details behind an author's posts on DZone.  This piqued my interest, as I have always wondered what additional statistics I could locate for my articles.

Currently, on each author's profile page, there is a list of articles, along with the publish date and the number of page views.  I wondered if I could use the RESTful API in order to build a page that provides additional insights into my posts.

AngularJS to the Rescue

Since I have been working on an AngularJS project, I thought I would use AngularJS to create a simple application to present my DZone post analytics.  The first thing I did was create a simple service to make a request to the following RESTful service:

https://dzone.com/services/widget/article-listV2/list?author=author_id&page=1&portal=all&sort=newest

In the URI  above, the author_id  would be the unique ID for a DZone writer.  Calling the URI  returns quite a bit of data, below is the information limited to my most-recent post using Postman:

{
    "success": true,
    "result": {
        "data": {
            "nodes": [
                {
                    "id": 1713813,
                    "title": "The Dark Web - When Did This Happen?",
                    "imageUrl": 6621036,
                    "authors": [
                        {
                            "id": 1224939,
                            "name": "johnjvester",
                            "realName": "John Vester",
                            "aboutAuthor": "Information Technology professional with 25+ years expertise in application development, project management, system administration and team supervision. Currently focusing on enterprise architecture/application design utilizing object-oriented programming languages and frameworks. Prior expertise building Java-based APIs against React and Angular client frameworks. CRM design, customization and integration with SalesForce. Prior experience using both C# (.NET Framework) and J2EE (including Spring MVC, JBoss Seam, Struts Tiles, JBoss Hibernate, Spring JDBC).",
                            "url": "/users/1224939/johnjvester.html",
                            "avatar": "//dz2cdn3.dzone.com/storage/user-avatar/6554902-thumb.jpg",
                            "tagline": "Sr. Architect at CleanSlate Technology Group",
                            "isMVB": true
                        }
                    ],
                    "articleDate": 1505840460000,
                    "articleLink": "/articles/the-dark-web-when-did-this-happen",
                    "nComments": 1,
                    "tags": [
                        "dark web",
                        "security",
                        "network security",
                        "vpn"
                    ],
                    "acl": {
                        "edit": false,
                        "delete": false
                    },
                    "editUrl": "/content/1713813/edit.html",
                    "saveStatus": {
                        "saved": false,
                        "canSave": false,
                        "count": 3
                    },
                    "views": 10123,
                    "portal": "Security",
                    "shortTitle": "application-web-network-security",
                    "articleContent": "There's been so much hype about "the dark web" lately, I felt like I have missed something. Turns out, this buzzword was created for a concept that has been around for decades."
                }
            ]
        }
    }
}

Using Angular, I created a very simple service to call the RESTful service and obtain my entire list of published articles, which was 139 articles at the time this post was written:

angular.module('dZoneStatistics.stats')
    .service("statsService", ['GLOBAL', '$http', '$q',
        function (GLOBAL, $http, $q) {
            'use strict';

            var service = {};

            service.getAllDzonePages = function (id, pages) {
                var deferred = $q.defer();
                var returnArray = [];

                $q.all(pages.map(function (page) {
                    return $http.get(GLOBAL.URL_BASE + id + GLOBAL.URL_PAGE + page + GLOBAL.URL_PORTAL_SORT)
                        .then(function (response) {
                            if (response.data.result.data.nodes.length > 0) {
                                for (var i = 0; i < response.data.result.data.nodes.length; i++) {
                                    returnArray.push(response.data.result.data.nodes[i]);
                                }
                            }
                        });
                })).then(function () {
                    deferred.resolve(returnArray);
                });

                return deferred.promise;
            };

            return service;
        }]);

Since the RESTful API only returns 25 articles at a time, I created a simple loop process to call the API multiple times until all 139 articles were retrieved.  Keep in mind, this is just a simple example.

Processing the Results

With the results from the RESTful API in memory, I was able to perform some analysis on the results.  My goals were as follows:

  • Understand high-level metrics regarding my posts:

    • Total page views

    • Average page views per article

    • Average page views per day

  • Understand my page views across the various DZone portals

Keeping things simple, I decided to put my code in the controller.  If I had more time, I would have created factory/service classes to better house the programming logic and data.

The following declarations were made at the start of the Angular controller:

var vm = this;
vm.data = [];
vm.articleCount = 0;
vm.totalPageViews = 0;
vm.averagePageViewsPerArticle = 0;
vm.portalList = [];
vm.views = [];
vm.pieOptions = {};
vm.names = [];
vm.realName = "";
vm.avatarURL = "";
vm.averageViewsPerDay = 0;
vm.firstArticlePubDate = moment();
vm.baseURL = GLOBAL.URL_VIEW_BASE;

These vm  elements will be exposed by the controller for use in the HTML view.

Upon loading the page, I called a function from the init()  within the AngularJS controller to connect to the DZone RESTful service:

function getDzoneAllPageInfo(id, maxPageNum) {
  vm.data = [];
  var pages = [];

  for (var i = 1; i <= maxPageNum; i++) {
  pages[i-1] = i;
  }

  statsService.getAllDzonePages(id, pages).then(function (response) {
  if (response) {
    vm.data = response;
        vm.articleCount = vm.data.length;
        getPageViewStats(vm.data);
        computeAverages();
        computeFinalStats();
        getNameAndAvatar(vm.data[0]);
    }
  });
}

The function first determines the maxPageNum to load and builds a pages[]  array for processing by the service.  Additionally, using the  QueryString, I decided to pass in an id   parameter to avoid hard-coding the ID I wanted to process.

When the results are returned, an array of articles are used for processing.  Since the data will be used on the html page, the results will be stored as  vm.data.  The length of the articles will make up the article count and will be stored as vm.articleCount  for the html page used to present the stats.

The remainder of the stats are computed from a few different methods.

getPageViewStats()

The getPageViewStats()  function has three functions. 

First, the overall pageView stats are computed.  Second, the oldest article date is located (for average views per day).  Third, the list of different portals used by the author are identified.

function getPageViewStats(data) {
  var viewTotal = 0;

  for (var i = 0; i < data.length; i++) {
    var thisData = data[i];
    determineOldestArticle(thisData);
    buildPortalList(thisData);
    viewTotal = viewTotal + thisData.views;
  }

  vm.totalPageViews = viewTotal;

  if (viewTotal > 0) {
    vm.averagePageViewsPerArticle = viewTotal / data.length;
  }
}

The logic for the determineOldestArticle()  function, which uses moment.js, is displayed below:

function determineOldestArticle(thisData) {
  var thisArticleDate = moment(thisData.articleDate);

  if (moment(thisArticleDate).isBefore(vm.firstArticlePubDate)) {
    vm.firstArticlePubDate = thisArticleDate;
  }
}

Finally, the buildPortalList()  function is described below:

function buildPortalList(thisData) {
  var portal = [];
  if (vm.portalList && vm.portalList.length > 0) {
    var found = false;
    for (var i = 0; i < vm.portalList.length; i++) {
      var thisPortal = vm.portalList[i];

      if (thisPortal.name.toUpperCase().trim() === thisData.portal.toUpperCase().trim()) {
        found = true;
        thisPortal.views = thisPortal.views + thisData.views;
        thisPortal.count = thisPortal.count + 1;
        break;
      }
    }

    if (!found) {
      portal.name = thisData.portal;
      portal.count = 1;
      portal.views = thisData.views;
      portal.avgViews = 0;
      vm.portalList.push(portal);
    }
  } else {
    portal.name = thisData.portal;
    portal.count = 1;
    portal.views = thisData.views;
    portal.avgViews = 0;
    vm.portalList.push(portal);
  }
}

TODO  in the code is to resolve the duplicate logic in lines 17 - 21 and 24 - 28.

computeAverages()

With the portalList array created (using  vm.portalList), the next step is to compute the averages.  This is handled through the following code:

function computeAverages() {
  for (var i = 0; i < vm.portalList.length; i++) {
    var thisPortal = vm.portalList[i];
    vm.views.push(thisPortal.views);
    vm.names.push(thisPortal.name);
    thisPortal.avgViews = thisPortal.views / thisPortal.count;
  }
}

computeFinalStats()

With the averages computed, the final function computes the final statistics for the selected DZone author's page:

function computeFinalStats() {
  vm.daysSinceFirstPost = today.diff(vm.firstArticlePubDate, "days");
  vm.averageViewsPerDay = vm.totalPageViews / vm.daysSinceFirstPost;
}

getNameAndAvatar()

To make things look a little nicer, I thought I would grab the author's avatar image and real name from the JSON  payload:

function getNameAndAvatar(data) {
  vm.realName = data.authors[0].realName;
  vm.avatarURL = "https://" + data.authors[0].avatar;
}

Creating the view

With the RESTful API data consumed and analyzed, the next step was to define the view for the data.  Using Twitter Bootstrap and the jumbotron class, plus Angular smart-table, the following HTML was configured:

<div class="jumbotron">
    <div class="container">
        <h1 class="display-3">DZone.com Article Information</h1>
        <div class="col-12 col-sm-12 col-lg-12">
            <div class="col-sm-5">
                <h3>{{ctrl.realName}}</h3>
                <img data-ng-src="{{ctrl.avatarURL}}" style="height: 230px; margin-bottom: 15px;">
                <div class="table-responsive">
                    <table class="table table-striped">
                        <thead>
                        <tr>
                            <td colspan="2"><h4>Overall Statistics</h4></td>
                        </tr>
                        </thead>
                        <tbody>
                        <tr>
                            <td style="text-align: left;">First Post: {{ctrl.firstArticlePubDate | moment:'M/D/YYYY'}}</td>
                            <td style="text-align: right;">{{ctrl.daysSinceFirstPost | number:0}} days ago</td>
                        </tr>
                        <tr>
                            <td style="text-align: left;">Total Articles Published</td>
                            <td style="text-align: right;">{{ctrl.articleCount | number:0}}</td>
                        </tr>
                        <tr>
                            <td style="text-align: left;">Total Page Views</td>
                            <td style="text-align: right;">{{ctrl.totalPageViews | number:0}}</td>
                        </tr>
                        <tr>
                            <td style="text-align: left;">Average Views Per Article</td>
                            <td style="text-align: right;">{{ctrl.averagePageViewsPerArticle | number:2}}</td>
                        </tr>
                        <tr>
                            <td style="text-align: left;">Average Views Per Day</td>
                            <td style="text-align: right;">{{ctrl.averageViewsPerDay | number:2}}</td>
                        </tr>
                        </tbody>
                    </table>
                </div>
            </div>
            <div class="col-sm-7">
                <div class="table-responsive">
                    <table st-table="displayedList" st-safe-src="ctrl.portalList" class="table table-striped">
                        <thead>
                        <tr>
                            <th style="text-align: left;" st-sort="name" st-sort-default="true">Portal</th>
                            <th style="text-align: right;" st-sort="count">Articles</th>
                            <th style="text-align: right;" st-sort="views">Views</th>
                            <th style="text-align: right;" st-sort="avgViews">Average Views</th>
                        </tr>
                        </thead>
                        <tbody>
                        <tr ng-repeat="row in displayedList">
                            <td style="text-align: left;">{{row.name}}</td>
                            <td style="text-align: right;">{{row.count | number:0}}</td>
                            <td style="text-align: right;">{{row.views | number:0}}</td>
                            <td style="text-align: right;">{{row.avgViews | number:2}}</td>
                        </tr>
                        </tbody>
                    </table>
                </div>
            </div>
        </div>
    </div>
</div>

Once loaded, the results appeared as shown below, using my DZone ID:

Image title

Based up the information displayed above, I was able to recognize my goals:

  • Understand high-level metrics regarding my posts:

    • Total page views = 2.26 million

    • Average page views per article = 16,287

    • Average page views per day = 3,305

  • Understand my page views across the various DZone portals = data in right-hand table

Not Shown

Not shown, is the table below the screenshot above, where I basically show the following information for each article:

  • publish date

  • article title

  • portal where article lives

  • page views

Angular smart-table allows simple filtering and sorting on this table, making it easy to find information quickly in the list of articles.

Below, is a screenshot on this part of the page - sorted by the views column:

Image title

Conclusion

Using the public RESTful API provided by DZone, I was able to create a quick AngularJS application to obtain all of my articles, then generate the statistics I was interested in without my effort. 

Use of other open source tools, like Angular smart-table and Twitter Bootstrap, made it easy to make my results look far better than I could have writing the HTML myself.

Have a really great day!

Deploy code to production now. Release to users when ready. Learn how to separate code deployment from user-facing feature releases with LaunchDarkly.

Topics:
web dev ,restful api ,angularjs ,statistics

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}