VRT - Realtime HTML5 Data Visualization Application
An introduction to DZone user's Odd Marthon Lende's hobby project available on GitHub.
Join the DZone community and get the full member experience.Join For Free
Hello, my name is Odd Marthon Lende. I am employed as a Technical Support Engineer in Stavanger, Norway.
I started working on this project 3-4 years ago, it started out as a hobby project at work. The mission was to create a monitoring application. An application that receives data on a web service and reflects it back to browser clients.
This project has taken too much of my time and energy, it has been abandoned several times for many months. There has been periods (months,weeks) with a lot of new ideas and writing. To complete my work project i have written a similar application using only Microsoft technology and a few 3rd party libraries in 3-4 months.
Is/Should be a versatile visualization application in the browser using standard web technologies that enables the user/developer to create visualizations of data inside containers called widgets. These widgets has a standard interface that allows you to read data in a standard way and create your own visualization using the web technology of your choice (HTML, Canvas, WebGL, 3rd pary libraries) inside this container. These programs can then be grouped together into a group called a dashboard. The widgets can then be positioned and scaled inside the body of the browser window. A standard interface for accessing and reading data into a common format (table/matrix), so it can easily be extended with different data sources. I have been working on reading JSON, CSV and also implemented a data source using the Web Audio API.
cd vrt # Change the directory to where you cloned the repository npm install # Installs the required nodejs modules node standalone.js # This starts the nodejs process # If the process started successfully, # you should be able to open the browser (only Google Chrome currently) to # http://localhost:23485
VRT is still in early development and i will not spend any more time developing it if there is no interest for it. I've released the source on github and was hoping the get some developers/designers interested.
The Struggle For Speed
The first serious attemt was to create a simple table as a widget, i found a library called D3.
Creating a table using HTML elements with not many columns and rows worked fine, try creating 500 000 - 1 000 000 elements and look at the browser memory / cpu and you'll know what i mean. D3 is a library that allows you to create SVG graphics, so for every data point there will be created a new DOM element or more in memory, it will bring the browser to a halt eventually.
D3 is a great library, but i was not very impressed with the speed and memory usage when loading a large amount of data.
I do not use it for any of the visualizations in the latest update, but i've used it in the UI alot.
I also experimented with canvas, the code has been deleted in the current revision, but it is still there in the repository. I tried optimizing, so a big difference here from D3 would be having the possiblity to not redraw all the data points when there is an update. To add a new line to the canvas when a new datapoint arrived, i would copy the image data and shift the whole image position horizontally and using the previous point from memory and then draw a line from the previous point to the new point. When pumping up the data thoughtput, it would start lagging alot.
It's about being able to draw the data using with as few programming steps as possible.
Receive Data - Upload to graphics card - Render.
Please visit the github page for more info and fork or star it. If people want to develop it, i think the next step for me would be to start writing some documentation.
Opinions expressed by DZone contributors are their own.