DZone
Web Dev Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > Web Dev Zone > Easily parallelize jobs using web workers and a threadpool with HTML5

Easily parallelize jobs using web workers and a threadpool with HTML5

Jos Dirksen user avatar by
Jos Dirksen
·
May. 20, 12 · Web Dev Zone · Interview
Like (0)
Save
Tweet
6.55K Views

Join the DZone community and get the full member experience.

Join For Free

i've been experimenting with web workers and the various browser implementations. most of the articles i've seen show an example where a single worker thread is started in the background to execute some heavy task. this frees up the main thread to render the rest of the webpage and respons to user input. in a previous article i showed how you can off-load cpu heavy tasks to a seperate web worker thread. in that example we used a couple of libraries to get the following effect:

sophie web worker

sinc almost everyone nowadays has multiple cores it's a waste not to use them. in this article i'll show how we can use a simple threadpool to parallelize this even further and increase the rendering time by +/- 300%. you can run this example from the following location: http://www.smartjava.org/examples/webworkers2/

the threadpool code

to test multiple threads with web workers i wrote a simple (and very naive) threadpool / taskqueue. you can configure the maximum number of concurrent web workers when you create this pool, and any 'task' you submit will be executed using one of the available threads from the pool. note that we aren't really pooling threads, we're just using this pool to control the number of concurrently executing web workers.

function pool(size) {
    var _this = this;
 
    // set some defaults
    this.taskqueue = [];
    this.workerqueue = [];
    this.poolsize = size;
 
    this.addworkertask = function(workertask) {
        if (_this.workerqueue.length > 0) {
            // get the worker from the front of the queue
            var workerthread = _this.workerqueue.shift();
            workerthread.run(workertask);
        } else {
            // no free workers,
            _this.taskqueue.push(workertask);
        }
    }
 
    this.init = function() {
        // create 'size' number of worker threads
        for (var i = 0 ; i < size ; i++) {
            _this.workerqueue.push(new workerthread(_this));
        }
    }
 
    this.freeworkerthread = function(workerthread) {
        if (_this.taskqueue.length > 0) {
            // don't put back in queue, but execute next task
            var workertask = _this.taskqueue.shift();
            workerthread.run(workertask);
        } else {
            _this.taskqueue.push(workerthread);
        }
    }
}
 
// runner work tasks in the pool
function workerthread(parentpool) {
 
    var _this = this;
 
    this.parentpool = parentpool;
    this.workertask = {};
 
    this.run = function(workertask) {
        this.workertask = workertask;
        // create a new web worker
        if (this.workertask.script!= null) {
            var worker = new worker(workertask.script);
            worker.addeventlistener('message', dummycallback, false);
            worker.postmessage(workertask.startmessage);
        }
    }
 
    // for now assume we only get a single callback from a worker
    // which also indicates the end of this worker.
    function dummycallback(event) {
        // pass to original callback
        _this.workertask.callback(event);
 
        // we should use a seperate thread to add the worker
        _this.parentpool.freeworkerthread(_this);
    }
 
}
 
// task to run
function workertask(script, callback, msg) {
 
    this.script = script;
    this.callback = callback;
    this.startmessage = msg;
};

using the threadpool

to use this threadpool we now just have to do this:

    var pool = new pool(6);
    pool.init();

this will create a pool that will allow a maximum number of 8 threads running concurrently. if we want to create a task to be executed by this pool we just create a workertask and submit it like this:

      var workertask = new workertask('extractmaincolor.js',callback,wp);
      pool.addworkertask(workertask);

this will create a web worker from 'extractmaincolor.js' and register the supplied function as callback. once the worker is ready to be run, the last argument will be used to send a message to the worker. a caveat on this implementation. i now assume that the when the web worker sends a message back it will close itself after sending this message. as you can see in the following example:

importscripts('quantize.js' , 'color-thief.js');
 
self.onmessage = function(event) {
    var wp = event.data;
    var foundcolor = createpalettefromcanvas(wp.data,wp.pixelcount, wp.colors);
    wp.result = foundcolor;
    self.postmessage(wp);
 
    // close this worker
    self.close();
};

results

i've tested this a couple of times with different settings for number of concurrent threads. the results are shown in the following table:

chrome:

number of threads total rendering time
1 14213
2 9956
3 8778
4 7846
5 6924
6 6309
7 5912
8 5468
9 5201
10 5193
11 5133
12 5208

the result for firefox are less impressive, but you can still see a big gain:

firefox:

number of threads total rendering time
1 17909
2 11273
3 10422
4 10154
5 10115
6 10052
7 10000
8 9997

as you can see, both for firefox and chrome it's useful to not just use a single web worker, but further split the tasks. for firefox we can see a big gain if we use two web workers, and for chrome we keep on getting better results to 8 or 9 parallel web workers!

HTML

Published at DZone with permission of Jos Dirksen, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • How to Perform Visual Regression Testing Using Cypress
  • How to Handle Early Startup Technical Debt (Or Just Avoid it Entirely)
  • Making Machine Learning More Accessible for Application Developers
  • A Guide to Understanding Vue Lifecycle Hooks

Comments

Web Dev Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo