Partial Image Manipulation With Canvas and Webworkers
Join the DZone community and get the full member experience.
Join For Freein my previous article i showed how to use box2d-web and deviceorientation to rol around a set of circles using standard html5 apis. my initial goal of that article was to load in an image, translate it to a series of seperate circles, and let you play around with that. but that's for later. for now i'll show you how you can use canvas together with web workers to offload heavy computing functions to a background thread.
for this example we'll take an input image and manipulate the image in blocks of 10x10 pixels. for each block we'll calculate the dominating color and render a rectangle on our target in that color. since an example says more then a thousand words, we're going to create this:
for those wondering, that's my daugther thinking very hard about something. you can see this demo in action here .
getting started
to implement this example we don't need to do that much. we just have to take the following steps:
- wait until the image is loaded.
- split the image into seperate parts ready for processing.
- configure a web worker to start processing when it receives a message
- calculate the dominating color from our image sample.
- render a rectangle on our target canvas
let's begin simple, and look at the image loading code.
wait for image to be loaded
before we start processing the image we have to make sure it is completely loaded. for this we use the following piece of code:
// start processing when the document is loaded. $(document).ready(function () { // handles rendering the elements setupworker(); // wait for the image to be loaded, before we start processing it. $("#source").load(function () { // determine size of image var imgwidth = $(this).width(); var imgheight = $(this).height(); // create a canvas and make context available var targetcanvas = createtargetcanvas(imgwidth, imgheight); targetcontext = targetcanvas.getcontext("2d"); // render elements renderelements(imgwidth, imgheight, $(this).get()[0]); }); });
as you can see here, we register the jquery ready function first. this will trigger when the complete document is loaded. this however doesn't have to mean that the images have also already been loaded. to make sure the image is ready to be processed, we add the jquery load function to our source image (has id of #source). when the image is loaded we determine the required size of our target canvas, on which we render the result, and fire of the rendering using the renderelements function. the renderelements function splits the image and fires of the webworkers.
split the image into seperate parts ready for processing
the goal of this example is to create a kind of low pixel effect on our source image. we do this by selecting part of the image, calculate the dominating color, and render a square on the target canvas. the following code shows how you can use a temporary canvas to select part of the image.
// process the image by splitting it in parts and sending it to the worker function renderelements(imgwidth, imgheight, image) { // determine image grid size var nrx = math.round(imgwidth / bulletsize); var nry = math.round(imgheight / bulletsize); // iterate through all the parts of the image for (var x = 0; x < nrx; x++) { for (var y = 0; y < nrx; y++) { // create a canvas element we use for temporary rendering var canvas2 = document.createelement('canvas'); canvas2.width = bulletsize; canvas2.height = bulletsize; var context2 = canvas2.getcontext('2d'); // render part of the image for which we want to determine the dominant color context2.drawimage(image, x * bulletsize, y * bulletsize, bulletsize, bulletsize, 0, 0, bulletsize, bulletsize); // get the data from the image var data = context2.getimagedata(0, 0, bulletsize, bulletsize).data // convert data, which is a canvas pixel array, to a normal array // since we can't send the canvas array to a webworker var dataasarray = []; for (var i = 0; i < data.length; i++) { dataasarray.push(data[i]); } // create a workpackage var wp = new workpackage(); wp.colors = 5; wp.data = dataasarray; wp.pixelcount = bulletsize * bulletsize; wp.x = x; wp.y = y; // send to our worker. worker.postmessage(wp); } } }
in this function we first determine in how many rows and columns we're going to split up the image. we iterate over each of these elements and render that specific part of the image on a temporary canvas. from that canvas we get the data using the getimagedata function. at this point we've got all the information we need for our worker to calculate the dominating color (this is an expensive operation). we store the info in a 'workpackage':
function workpackage() { this.data = []; this.pixelcount = 0; this.colors = 0; this.x = 0; this.y = 0; this.result = [0, 0, 0]; }
this is a convience class that serves as the message to and from our webworker. note that we need to convert the result from the getimagedata call to a normal array. information to a webworker is copied, and chrome at least isn't able to copy the resulting array from the getimagedata operation. so far so good. we now have nice workpackages for each part of our screen, which we pass to a webworker using the worker.postmessage operation. but what does this worker look like, and how do we configure it?
we create the worker in the setupworker operation that is called when our document is loaded.
function setupworker() { worker = new worker('extractmaincolor.js'); worker.addeventlistener('message', function (event) { // the workpackage contains the results var wp = event.data; // get the colors var colors = wp.result; drawrectangle(targetcontext, wp.x, wp.y, bulletsize, colors[0]); //drawcircle(targetcontext, wp.x, wp.y, bulletsize, colors[0]); }, false); }
creating a worker, as you can see, is very simple. just point the worker
to the javascript he needs to execute. note that there are all kind of
restrictions with regards to the resources and objects a worker has
access to. a good introduction to what can and what can't be accessed
can be found in this
article
.
once we defined the worker, we add an eventlistener. this listener is
called when the worker uses the postmessage operation. in our example
this is used to pass the result back in the same workpackage. based on
this result we draw a rectangle (or some other figure) on our target
canvas. the worker itself is very basic:
importscripts('quantize.js' , 'color-thief.js'); self.onmessage = function(event) { var wp = event.data; var foundcolor = createpalettefromcanvas(wp.data,wp.pixelcount, wp.colors); wp.result = foundcolor; self.postmessage(wp); };
this worker uses two external scripts to calculate and return the dominating color. it does this by getting the required information from the workpackage, calculate the dominating color, and return the result in the workpackage using the postmessage. calculating the dominating color itself isn't that easy. i ran across a great library named color-thief , that does this for you. apparently you need to take more into account than just the rgb values, if you do that then you just get a set of brown colors.
calculate the dominating color from our image sample.
i mentioned that i used the color-thief library to calculate the dominating color. i do this using this code:
createpalettefromcanvas(wp.data,wp.pixelcount, wp.colors);
this, however, isn't directly provided by color-thief. color-thief assumes you want to use it directly on an image element on your page. i had to extend the color-thief library with the following simple operation so that it can work directly with binary data.
function createpalettefromcanvas(pixels, pixelcount, colorcount) { // store the rgb values in an array format suitable for quantize function var pixelarray = []; for (var i = 0, offset, r, g, b, a; i < pixelcount; i++) { offset = i * 4; r = pixels[offset + 0]; g = pixels[offset + 1]; b = pixels[offset + 2]; a = pixels[offset + 3]; // if pixel is mostly opaque and not white if (a >= 125) { if (!(r > 250 && g > 250 && b > 250)) { pixelarray.push([r, g, b]); } } } // send array to quantize function which clusters values // using median cut algorithm var cmap = mmcq.quantize(pixelarray, colorcount); var palette = cmap.palette(); return palette; }
this returns an array of most dominating colors (just as the normal color-thief functions do) but can work directly on the data from our worker.
render a rectangle on our target canvas
and that's pretty much it. at this point we've split our image into an array of subimages. each part is sent to a webworker for processing. the webworker processes the image and passes the result back to our eventhandler. in the eventhandler we take the most dominating color and we can use that to draw on the canvas. in the figure at the beginning of this article i used rectangles:
using this javascript (and with a bulletsize of 15):
// draw a rectangle on the supplied context function drawrectangle(targetcontext, x, y, bulletsize, colors) { targetcontext.beginpath(); targetcontext.rect(x * bulletsize, y * bulletsize, bulletsize, bulletsize); targetcontext.fillstyle = "rgba(" + colors + ",1)"; targetcontext.fill(); }
but we could just as easily render circles:
using this:
// draw a circle on the supplied context function drawcircle(targetcontext, x, y, bulletsize, colors) { var centerx = x * bulletsize + bulletsize / 2; var centery = y * bulletsize + bulletsize / 2; var radius = bulletsize / 2; targetcontext.beginpath(); targetcontext.arc(centerx, centery, radius, 0, 2 * math.pi, false); targetcontext.fillstyle = "rgba(" + colors + ",1)"; targetcontext.fill(); }
as you can see web workers are really easy to use, and canvas allows us much options to work with imagedata. in this example i only used a single web worker, more interesting would be to add a queue on which multiple workers would listen to really process elements in parallel. the demo and complete code for this article can be found here .
Published at DZone with permission of Jos Dirksen, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments