Easily parallelize jobs using web workers and a threadpool with HTML5
Join the DZone community and get the full member experience.
Join For Freei've been experimenting with web workers and the various browser implementations. most of the articles i've seen show an example where a single worker thread is started in the background to execute some heavy task. this frees up the main thread to render the rest of the webpage and respons to user input. in a previous article i showed how you can off-load cpu heavy tasks to a seperate web worker thread. in that example we used a couple of libraries to get the following effect:
sinc almost everyone nowadays has multiple cores it's a waste not to use them. in this article i'll show how we can use a simple threadpool to parallelize this even further and increase the rendering time by +/- 300%. you can run this example from the following location: http://www.smartjava.org/examples/webworkers2/
the threadpool code
to test multiple threads with web workers i wrote a simple (and very naive) threadpool / taskqueue. you can configure the maximum number of concurrent web workers when you create this pool, and any 'task' you submit will be executed using one of the available threads from the pool. note that we aren't really pooling threads, we're just using this pool to control the number of concurrently executing web workers.
function pool(size) { var _this = this; // set some defaults this.taskqueue = []; this.workerqueue = []; this.poolsize = size; this.addworkertask = function(workertask) { if (_this.workerqueue.length > 0) { // get the worker from the front of the queue var workerthread = _this.workerqueue.shift(); workerthread.run(workertask); } else { // no free workers, _this.taskqueue.push(workertask); } } this.init = function() { // create 'size' number of worker threads for (var i = 0 ; i < size ; i++) { _this.workerqueue.push(new workerthread(_this)); } } this.freeworkerthread = function(workerthread) { if (_this.taskqueue.length > 0) { // don't put back in queue, but execute next task var workertask = _this.taskqueue.shift(); workerthread.run(workertask); } else { _this.taskqueue.push(workerthread); } } } // runner work tasks in the pool function workerthread(parentpool) { var _this = this; this.parentpool = parentpool; this.workertask = {}; this.run = function(workertask) { this.workertask = workertask; // create a new web worker if (this.workertask.script!= null) { var worker = new worker(workertask.script); worker.addeventlistener('message', dummycallback, false); worker.postmessage(workertask.startmessage); } } // for now assume we only get a single callback from a worker // which also indicates the end of this worker. function dummycallback(event) { // pass to original callback _this.workertask.callback(event); // we should use a seperate thread to add the worker _this.parentpool.freeworkerthread(_this); } } // task to run function workertask(script, callback, msg) { this.script = script; this.callback = callback; this.startmessage = msg; };
using the threadpool
to use this threadpool we now just have to do this:
var pool = new pool(6); pool.init();
this will create a pool that will allow a maximum number of 8 threads running concurrently. if we want to create a task to be executed by this pool we just create a workertask and submit it like this:
var workertask = new workertask('extractmaincolor.js',callback,wp); pool.addworkertask(workertask);
this will create a web worker from 'extractmaincolor.js' and register the supplied function as callback. once the worker is ready to be run, the last argument will be used to send a message to the worker. a caveat on this implementation. i now assume that the when the web worker sends a message back it will close itself after sending this message. as you can see in the following example:
importscripts('quantize.js' , 'color-thief.js'); self.onmessage = function(event) { var wp = event.data; var foundcolor = createpalettefromcanvas(wp.data,wp.pixelcount, wp.colors); wp.result = foundcolor; self.postmessage(wp); // close this worker self.close(); };
results
i've tested this a couple of times with different settings for number of concurrent threads. the results are shown in the following table:
chrome:
number of threads | total rendering time |
---|---|
1 | 14213 |
2 | 9956 |
3 | 8778 |
4 | 7846 |
5 | 6924 |
6 | 6309 |
7 | 5912 |
8 | 5468 |
9 | 5201 |
10 | 5193 |
11 | 5133 |
12 | 5208 |
the result for firefox are less impressive, but you can still see a big gain:
firefox:
number of threads | total rendering time |
---|---|
1 | 17909 |
2 | 11273 |
3 | 10422 |
4 | 10154 |
5 | 10115 |
6 | 10052 |
7 | 10000 |
8 | 9997 |
as you can see, both for firefox and chrome it's useful to not just use a single web worker, but further split the tasks. for firefox we can see a big gain if we use two web workers, and for chrome we keep on getting better results to 8 or 9 parallel web workers!
Published at DZone with permission of Jos Dirksen, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Trending
-
A Data-Driven Approach to Application Modernization
-
Effective Java Collection Framework: Best Practices and Tips
-
RAML vs. OAS: Which Is the Best API Specification for Your Project?
-
DevOps Midwest: A Community Event Full of DevSecOps Best Practices
Comments