Over a million developers have joined DZone.

Video Manipulation with the Canvas Element

· Web Dev Zone

Start coding today to experience the powerful engine that drives data application’s development, brought to you in partnership with Qlik.

Video Manipulation with the Canvas ElementOne of the designers in my company asked me to help him with the creation of a transparent video which run on top of an image in HTML. So I did some digging and found a very good resource to do that - Manipulating video using canvas. This post will explain how to combine the video and canvas elements in order to create transparency inside a video during runtime using JavaScript. I’ve based the code on the previous link so they deserve all the credit. You can download the demo code from here.

The Demands

The designer has created a simple movie file with a red square that is floating on top of a black background. He wanted that during runtime the black background will be transparent so the image that he want to show underneath will be visible. Here is a figure that shows how the movie looks like:
MoviePicture

The HTML File

<!DOCTYPE HTML>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <title>Working with Canvas</title>
    <script type="text/javascript" src="main.js"></script>
    <link href="style.css" rel="stylesheet" />
</head>
<body onload="handlePageLoad()">
    <video id="video" controls="controls" width="997px" height="587px" autoplay>
        <source src="vid.mp4" type="video/mp4" />
    </video>
    <div>
        <canvas id="c1" width="997px" height="587px" />
        <canvas id="c2" width="997px" height="587px" />
    </div>
</body>
</html>

The HTML include two canvas elements and the video element. I use two canvas elements because one will hold the buffer that I’m working on (and making the black background transparent) and the other will show the movie after the adjustments finished. I also use the following style.css file:

#c1
{
    background-image: url(Bg.png);
    background-repeat: no-repeat;
}
#c2
{
    display: none;
}
#video
{
    display: none;
}

In the CSS I’m making sure not to display the buffer canvas and the video element and to show the background image in the first canvas element.

The Real Thing – The JavaScript Code

Here is the implementation of the JavaScript file that will help you to do the job:

var video;
var ctx1;
var ctx2;
var width;
var height;
 
function timerCallback() {
    if (video.paused || video.ended) {
        return;
    }
    generateFrame();
    setTimeout(function () {
        timerCallback();
    }, 0);
}
 
function handlePageLoad() {
    video = document.getElementById("video");
    var c1 = document.getElementById("c1");
    ctx1 = this.c1.getContext("2d");
    var c2 = document.getElementById("c2");
    ctx2 = this.c2.getContext("2d");
    video.addEventListener("play", function () {
        width = this.videoWidth;
        height = this.videoHeight;
        timerCallback();
    }, false);
} 

function generateFrame() {
    ctx2.drawImage(video, 0, 0, width, height);
    var frame = ctx2.getImageData(0, 0, width, height);
    var len = frame.data.length / 4;
 
    for (var i = 0; i < len; i++) {
        var r = frame.data[i * 4 + 0];
        var g = frame.data[i * 4 + 1];
        var b = frame.data[i * 4 + 2];
        if (g < 10 && r < 10 && b < 10) {
            frame.data[i * 4 + 3] = 0;
        }
    }
    ctx1.putImageData(frame, 0, 0);
    return;
}

Lets deep dive into the things that the functions are doing:

  • The handlePageLoad function is responsible to initialize all the canvas contexts and to add a listener to the play event of the video. When it occurs it will start the timerCallback function.
  • The timerCallback function is responsible to stop the generations of frames when the video stops and to generate the current frame using the generateFrame function.
  • The generateFrame function is the function that is doing the heavy lifting of drawing on the canvas elements the transparent video. It draw the video frame to the buffer canvas and then extracts that frame. After extracting the frame it is passing on every pixel and check whether it is black. If so it set its opacity to transparent.

The effect will look like:
TheFullExample

Summary

In the post I showed how to use a canvas in order to make changes to a running video on the fly. This is only an experiment that I made to show that it is possible to do that. The performance of this implementation isn’t so good and I would have thought about other solutions such a video that includes the background image and the movie. 

Create data driven applications in Qlik’s free and easy to use coding environment, brought to you in partnership with Qlik.

Topics:

Published at DZone with permission of Gil Fink, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}