DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

SBOMs are essential to circumventing software supply chain attacks, and they provide visibility into various software components.

Related

  • How To Obtain IP Geolocation Data in Next.js
  • CORS Misconfigurations: The Simple API Header That Took Down Our Frontend
  • Exploring Intercooler.js: Simplify AJAX With HTML Attributes
  • How Spring Boot Starters Integrate With Your Project

Trending

  • Deploying LLMs Across Hybrid Cloud-Fog Topologies Using Progressive Model Pruning
  • Vibe Coding: Conversational Software Development — Part 1 Introduction
  • Why Tailwind CSS Can Be Used Instead of Bootstrap CSS
  • Serverless Machine Learning: Running AI Models Without Managing Infrastructure
  1. DZone
  2. Data Engineering
  3. Data
  4. How to Use Wikipedia API With Node.js

How to Use Wikipedia API With Node.js

In this article, I will show you how to access the statistics data or the page view statistics using Wikipedia API and NodeJS.

By 
Mustapha Mekhatria user avatar
Mustapha Mekhatria
·
May. 25, 18 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
15.1K Views

Join the DZone community and get the full member experience.

Join For Free

Wikipedia has an enormous data set such as page view statistics, tables, etc. In this article, I will show you how to access the statistics data or the page view statistics using Wikipedia API and NodeJS.

Remark: You can find the final project in this GitHub link.

I will use Wikipedia API to get the dates and the users views. You can read more about Wikipedia API here. The data that I am interested in are the pages view statistics for Africa, from 4/27/2017 to 5/17/2018. The request to fetch the data is as follow https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/user/Africa/daily/2017042700/2018051700.

Let’s Get Started 

Go to Wikipedia.com, search for Africa, click on the View history tab, and then click on Page view statistics (see pictures below).

The picture above displays the views statistics of Africa from 4/27/2018 to 5/17/2018, but I would like to get the data from 4/27/2017. So, go to the dates field on the left side of the chart, then type the right dates:

Now it is time to write some code to get the data in a nice table.

Code Time

As I mentioned above, I will use NodeJS to extract the data. I will also use the request-promise package as it simplifies HTTP request, or in other words, it helps me easily access and fetch any web page.

The first thing to do is to create a folder to save all the code. Next, open the terminal, browse to the folder created earlier, and create a package.json file by running this command line:

npm init

Install also the request-promise package:

npm install --save request
npm install --save request-promise

Now, let’s create a file such as readPagesViews.js, and copy/paste the code below:


var rp= require('request-promise');


var options={

    methode: 'GET',    
    uri:'https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/user/Africa/daily/2017042700/2018051700',
    json:true    
};

rp(options)
    .then(function(parseBody){

    var data=[];
    for(i=0 ;i<parseBody.items.length;i++){
        data.push([parseBody.items[i].timestamp,parseBody.items[i].views]);
    }

    console.log(data);
    })
    .catch(function (err){
});

The first line requires the package request-promise and passe it to the variable rp. The options object gathers the Get method parameters such as the link to the Wikipedia API, and the way to receive the data in JSON (json:true)

The options object is then passed as a parameter to rp; once the Get is executed, the result (data from Wikipedia) is saved into the object parseBody.
Basically, parseBody is a set of data received from Wikipedia:

...
    { project: 'en.wikipedia',
       article: 'Africa',
       granularity: 'daily',
       timestamp: '2018051100',
       access: 'all-access',
       agent: 'user',
       views: 6727 },
     { project: 'en.wikipedia',
       article: 'Africa',
       granularity: 'daily',
       timestamp: '2018051200',
       access: 'all-access',
       agent: 'user',
       views: 5450 },
...

Don’t forget that the goal is to get the dates and the users views. To get those two variables from the parseBody object, I use the following loop to save the timestamp (dates) and the views (users views) in the object data:

var data=[];
for(i=0 ;i<parseBody.items.length;i++){
     data.push([parseBody.items[i].timestamp,parseBody.items[i].views]);
}

Go back to the terminal and execute the code:

node readPagesViews.js

And voila! The data from the African Pageviews Analysis is stored in the object data:

...
  [ '2017090300', 5433 ],
  [ '2017090400', 5767 ],
  [ '2017090500', 6016 ],
  [ '2017090600', 6025 ],
  [ '2017090700', 6164 ],
  [ '2017090800', 5468 ],
  [ '2017090900', 4710 ],
  [ '2017091000', 5314 ],
...

Feel free to share your experience with this code or if you have a better way to get the data.

API Node.js Data (computing)

Published at DZone with permission of Mustapha Mekhatria. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • How To Obtain IP Geolocation Data in Next.js
  • CORS Misconfigurations: The Simple API Header That Took Down Our Frontend
  • Exploring Intercooler.js: Simplify AJAX With HTML Attributes
  • How Spring Boot Starters Integrate With Your Project

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • [email protected]

Let's be friends: