Should I Render Three.js to Video on the Client or Server?

DZone 's Guide to

Should I Render Three.js to Video on the Client or Server?

Have you encountered this dilemma? Never fear, Cliff Hall may just have an answer for us!

· Web Dev Zone ·
Free Resource

I’m working on an exciting new geegaw with a business partner and old friend in Seattle. It’s a 3D music visualizer built in HTML5 / WebGL using PureMVC, React, and Three.js. We already have a solid, extendable client framework, and some great stuff going on in the rendered scene, which the user can customize and then save.

Now we want to generate a video so we can upload it to YouTube or share it on Facebook.

Our original plan was to render the saved project on the server with Node.js, render-farm stylee. That would allow us to ensure it played smooth and didn’t stutter due to unknown client load and/or capabilities. Should be a snap, right? Er, no.

If you’ve researched this topic to any degree, you probably already know that there’s currently no out-of-box, miracle solution to make this happen, and certainly no example lash-togethers that will absolutely give you the same rendering on the server as you see on the client. If there is, and I’ve missed it (or if one comes along months from now), please let me know in the comments. For anyone else who may be trodding this path, her face sullen, set, and grim.

What's Wrong With Rendering on the Server?

There is an npm package for Three.js, however, there are a lot of document.createElement calls, and so it needs a DOM. The example on the npm page won’t work in node.js at all for this reason. I assume it is there for use with browserify and not specifically intended for server-side rendering.

You easily can add a DOM to your project with a package like jsdom, however, you won’t be able to do more than get a CanvasRenderer. For a WebGLRenderer, you need a WebGL implementation, which jsdom doesn’t give you.

Assuming it’s possible to get a WebGLRenderer, you’ll most likely need to implement a requestAnimationFrame() method depending on what library you chose for adding your DOM. No biggie, it’s just in charge of calling the passed in callback when a new frame is needed. You can just use setInterval() to call it at the desired frame rate.

There are several options for out-of-browser WebGL, including:

The above link for headless-gl takes you straight to the list of reasons why you might choose it over node-webgl or electron. I was moved to choose headless-gl.

Finally, here is the beginning of several hero posts in an issue thread about getting Three.js and WebGLRenderer to work in Node.js. The gist is in CoffeeScript. But Texturing and AntiAliasing both required special attention after getting a basic PNG of a cube drawn.

Wait a Minute. Am I Certain This Is the Only Way to Go About This?

It is at this point in my research that I begin to question all of the dependencies and potential hacks required to pull this off, and whether it truly is The Right Way(tm). I don’t take on project dependencies lightly. My little test project for figuring this part out had only the following top-level dependencies:

"dependencies": {
  "gl": "^4.0.2",
  "jsdom": "^9.5.0",
  "three": "^0.81.2"

But a quick peek into node_modules shows (…OMG…):

Destiny:node_modules cliff$ ls -al |wc -l

Yikes! A hundred and eighty seven packages already? That’s a lot of independently maintained places for this to fail. A lot of roads to dependency hell.

What Are the Pros and Cons of Trying to Do this Client-Side??

  • One obvious win is that we don’t pay our cloud provider for the CPU cycles necessary to render the frames.
  • We were worried about potential load or capabilities issues on the client, but actually, our particular target demographic (music producers) probably have pretty hefty machines anyway. And if they can get a scene they want to share working on their screen, then why not just hit record right then?
  • The frame rates we’re currently achieving on relatively ordinary hardware (my 2011 MacBook Air, for instance) are around 60 FPS. We’d only need about 30 FPS for video anyway, so we can probably spare a few cycles to grab each frame as an image. But can we turn those images into a movie?
  • Amazingly, the popular ffmpeg has been translated to JS and runs in the browser. However, the discussion thread where I discovered that indicates that it may not be ready for production. Hmmm. It’s an avenue for exploration at least.
  • Here’s a different approach: Render on the client, send the rendered frames to the server, and create the movie there.
  • Creating a movie from a sequence of files is going to be a much more well-trodden server-side path than rendering WebGL. Probably a less dodgy proposition.
  • In the above-linked example, the developer is A) writing frames to a node server on his local machine, B) doing it with synchronous XHR requests, and C) making those requests within the render loop. I don’t think this is feasible over the Interwebz, even given the fact that we’d be doing 30 FPS instead of 60.
  • I believe the concept is still sound, if we 1) write the frames to the browser’s local storage as a buffer within the render loop, 2) have a separate web worker process that sends those frames to the server, and 3) keeps a socket open and dumps in frame after frame until we’re done, eliminating the chatty overhead associated with all the individual XHR requests.
  • The audio should be on the server already, and should be trivial to include when creating the videos from the images.

Alright. I’m convinced that exploring the path of client-side frame rendering with server-side video encoding is worth giving a try. I’ll get back to you in another post with what I find. In the meantime, if you have anything to weigh in with, please hit me up in the comments section below!

Author’s Note: This article is part of a series, wherein my partner and I are trying out the development of our a product ‘out in the open’. Check out the next article here: Creating Video on the Server with Node.js.

client side, node js, rendering, server side, three.js, video, webgl

Published at DZone with permission of Cliff Hall , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}