Partial Image Manipulation With Canvas and Webworkers

in my previous article i showed how to use box2d-web and deviceorientation to rol around a set of circles using standard html5 apis. my initial goal of that article was to load in an image, translate it to a series of seperate circles, and let you play around with that. but that's for later. for now i'll show you how you can use canvas together with web workers to offload heavy computing functions to a background thread.

for this example we'll take an input image and manipulate the image in blocks of 10x10 pixels. for each block we'll calculate the dominating color and render a rectangle on our target in that color. since an example says more then a thousand words, we're going to create this:

for those wondering, that's my daugther thinking very hard about something. you can see this demo in action here .

getting started

to implement this example we don't need to do that much. we just have to take the following steps:

1. wait until the image is loaded.
2. split the image into seperate parts ready for processing.
3. configure a web worker to start processing when it receives a message
4. calculate the dominating color from our image sample.
5. render a rectangle on our target canvas

let's begin simple, and look at the image loading code.

wait for image to be loaded

before we start processing the image we have to make sure it is completely loaded. for this we use the following piece of code:

// start processing when the document is loaded.
    $(document).ready(function () {
 
        // handles rendering the elements
        setupworker();
 
        // wait for the image to be loaded, before we start processing it.
        $("#source").load(function () {
 
            // determine size of image
            var imgwidth = $(this).width();
            var imgheight = $(this).height();
 
            // create a canvas and make context available
            var targetcanvas = createtargetcanvas(imgwidth, imgheight);
            targetcontext = targetcanvas.getcontext("2d");
 
            // render elements
            renderelements(imgwidth, imgheight, $(this).get()[0]);
        });
    });

as you can see here, we register the jquery ready function first. this will trigger when the complete document is loaded. this however doesn't have to mean that the images have also already been loaded. to make sure the image is ready to be processed, we add the jquery load function to our source image (has id of #source). when the image is loaded we determine the required size of our target canvas, on which we render the result, and fire of the rendering using the renderelements function. the renderelements function splits the image and fires of the webworkers.

split the image into seperate parts ready for processing

the goal of this example is to create a kind of low pixel effect on our source image. we do this by selecting part of the image, calculate the dominating color, and render a square on the target canvas. the following code shows how you can use a temporary canvas to select part of the image.

    // process the image by splitting it in parts and sending it to the worker
    function renderelements(imgwidth, imgheight, image) {
        // determine image grid size
        var nrx = math.round(imgwidth / bulletsize);
        var nry = math.round(imgheight / bulletsize);
 
        // iterate through all the parts of the image
        for (var x = 0; x < nrx; x++) {
            for (var y = 0; y < nrx; y++) {
                // create a canvas element we use for temporary rendering
                var canvas2 = document.createelement('canvas');
                canvas2.width = bulletsize;
                canvas2.height = bulletsize;
                var context2 = canvas2.getcontext('2d');
                // render part of the image for which we want to determine the dominant color
                context2.drawimage(image, x * bulletsize, y * bulletsize, bulletsize, bulletsize, 0, 0, bulletsize, bulletsize);
 
                // get the data from the image
                var data = context2.getimagedata(0, 0, bulletsize, bulletsize).data
                // convert data, which is a canvas pixel array, to a normal array
                // since we can't send the canvas array to a webworker
                var dataasarray = [];
                for (var i = 0; i < data.length; i++) {
                    dataasarray.push(data[i]);
                }
 
                // create a workpackage
                var wp = new workpackage();
                wp.colors = 5;
                wp.data = dataasarray;
                wp.pixelcount = bulletsize * bulletsize;
                wp.x = x;
                wp.y = y;
 
                // send to our worker.
                worker.postmessage(wp);
            }
        }
    } 

in this function we first determine in how many rows and columns we're going to split up the image. we iterate over each of these elements and render that specific part of the image on a temporary canvas. from that canvas we get the data using the getimagedata function. at this point we've got all the information we need for our worker to calculate the dominating color (this is an expensive operation). we store the info in a 'workpackage':

 function workpackage() {
        this.data = [];
        this.pixelcount = 0;
        this.colors = 0;
        this.x = 0;
        this.y = 0;
 
        this.result = [0, 0, 0];
    }

this is a convience class that serves as the message to and from our webworker. note that we need to convert the result from the getimagedata call to a normal array. information to a webworker is copied, and chrome at least isn't able to copy the resulting array from the getimagedata operation. so far so good. we now have nice workpackages for each part of our screen, which we pass to a webworker using the worker.postmessage operation. but what does this worker look like, and how do we configure it?

   function setupworker() {
        worker = new worker('extractmaincolor.js');
        worker.addeventlistener('message', function (event) {
 
            // the workpackage contains the results
            var wp = event.data;
 
            // get the colors
            var colors = wp.result;
 
            drawrectangle(targetcontext, wp.x, wp.y, bulletsize, colors[0]);
            //drawcircle(targetcontext, wp.x, wp.y, bulletsize, colors[0]);
 
        }, false);
    }

importscripts('quantize.js' , 'color-thief.js');
 
self.onmessage = function(event) {
 
    var wp = event.data;
    var foundcolor = createpalettefromcanvas(wp.data,wp.pixelcount, wp.colors);
    wp.result = foundcolor;
    self.postmessage(wp);
 
};

createpalettefromcanvas(wp.data,wp.pixelcount, wp.colors);

function createpalettefromcanvas(pixels, pixelcount, colorcount) {
 
    // store the rgb values in an array format suitable for quantize function
    var pixelarray = [];
    for (var i = 0, offset, r, g, b, a; i < pixelcount; i++) {
        offset = i * 4;
        r = pixels[offset + 0];
        g = pixels[offset + 1];
        b = pixels[offset + 2];
        a = pixels[offset + 3];
        // if pixel is mostly opaque and not white
        if (a >= 125) {
            if (!(r > 250 && g > 250 && b > 250)) {
                pixelarray.push([r, g, b]);
            }
        }
    }
 
    // send array to quantize function which clusters values
    // using median cut algorithm
 
    var cmap = mmcq.quantize(pixelarray, colorcount);
    var palette = cmap.palette();
 
    return palette;
}

  // draw a rectangle on the supplied context
    function drawrectangle(targetcontext, x, y, bulletsize, colors) {
        targetcontext.beginpath();
        targetcontext.rect(x * bulletsize, y * bulletsize, bulletsize, bulletsize);
        targetcontext.fillstyle = "rgba(" + colors + ",1)";
        targetcontext.fill();
    }

 // draw a circle on the supplied context
    function drawcircle(targetcontext, x, y, bulletsize, colors) {
        var centerx = x * bulletsize + bulletsize / 2;
        var centery = y * bulletsize + bulletsize / 2;
        var radius = bulletsize / 2;
 
        targetcontext.beginpath();
        targetcontext.arc(centerx, centery, radius, 0, 2 * math.pi, false);
        targetcontext.fillstyle = "rgba(" + colors + ",1)";
        targetcontext.fill();
    }