Over a million developers have joined DZone.

Serverless Data Sync in Web Apps With Bit Torrent

Paul Kinlan shows how to retrieve data from your apps and share it on an instance of your app on another device; all without the hassle of a third-party service.

· Mobile Zone

Visually compose APIs with easy-to-use tooling. Learn how IBM API Connect provides near-universal access to data and services both on-premises and in the cloud, brought to you in partnership with IBM.

Our team has built a lot of Progressive Web Apps recently to demonstrate how we think they can be built: Airhorner, Voice Memos, Guitar Tuner, SVG-OMG are a few that spring to mind. One thing that is common across all of these sites is that they have no server component to store and synchronize data. We built these sites as examples and reference implementations of the types of experiences that we can deliver on the web, they were never intended to be full “Apps” that you would build as a business.

I recently had an idea for another Web App that was inspired by Ben Thompson’s Future of Podcasting podcast. I wanted to make podcasting as simple as visiting your site and pressing a record button on the page (I did something similar years ago called FriendBoo but that had to use the PSTN telephone system).

Technically, we have all the parts of the web platform to help us: Service Worker to make it work offline, Blob and IndexedDB to store chunks of data locally, and Media Stream Recorder to take Microphone input and record it to a file. Finally, we have the power of the URL to allow us to access the web app from anywhere there is a browser and an internet connection.

Once you have access to the web, you would expect that the data you recorded from one device would be stored on a server somewhere and then be available again later from any machine that you access the site. Traditionally, you create a web service that will store the data on a server somewhere and you would manage all that infrastructure to support all your users.

However, I’m not in the mood to create a service that requires storage and retrieval of data from a service (getting that through our legal teams will be a nightmare) so everything would have to be stored locally inside the web app. The big question is how do you get your data out and shared to another instance of your web app on another device?

Voice Memos created by Paul Lewis is another example of this.

All the Data Is Local to the App

The data is recorded and stored locally which means if you want to sync it with your other devices or share it with a friend, you can’t.

There are solutions, for example, we could dynamically encode the audio file as base64 and create a custom URL — but that’s not scalable.

It was a bit of a conundrum and one that I wanted to solve. I set up some simple requirements for what I would like to see:

  • The user should be able to share a simple URL that would point to their local data
  • The user should not have to manually save the data outside of their browser or web app
  • There should be no “backend” that stores the data
  • Synchronization should ideally happen peer to peer

Now, step back 6 months. I was at ColdFront conference last year and I saw a talk by Feross Aboukhadijeh about Web RTC Data Channel, BitTorrent and how he started a project called WebTorrent. It was a great talk, but I didn’t hook things up in my head until recently about the potential of what he was talking about.

Now I think I have the start of a solution through the use of WebTorrent.io.

Web Torrent

I won’t explain WebTorrent too much other than to say that it combines BitTorrent style distributed data delivery with WebRTC and it is rather spectacular. I do encourage you all to check it out, though.

The theory I had was that if the user’s client could act as a peer in the Torrent network, it would be then able to seed some of the data that is local to the website, the user could generate a torrent link that could be shared either with another one of their devices or with another user and then the “remote” instance of the web app will fetch the data from the “client” instance.

  • User vists page
  • User does some work
  • User saves it to IndexedDB
  • User clicks “Share” and it generates a “magnet:” URL and then starts to seed the Audio Blob
  • User shares or opens the URL in another browser
  • Site parses magnet URL and connects to tracker
  • Site finds peers from the tracker, connects to peer(s) and downloads data

Rough Flow of Data

For 1:1 connections this might be a bit of an overkill, I could just create a WebRTC signaling service and deliver the messages to the other instance to get the data from one client to another.

The interesting thing with this approach is that it scales nicely when sharing data to more than one person.

Applying This to a Real World Sample

I briefly mentioned Voice Memos earlier. It was a great reference application for me to try and integrate my theory into a working app. It is close to my idea of a podcasting app and it is also in need of a way to synchronize data between clients because it has no server-based backing store for the recordings.

Voice Memos:

I didn’t make too many changes other than adding in a “Share” button. You can check out the demo on BitTorrent Voice Memos. Record a simple audio file, save it, then share it - the “Share” will generate a URL that you can send to another device or send to another person.

I’m quite pleased with the output. In all this demo took only a couple of hours to get ready.

Here is a quick rundown of some of the major things that I had to do:

Add in the webtorrent.min.js script used to power all the magic.

<script src="https://cdn.jsdelivr.net/webtorrent/latest/webtorrent.min.js"></script>

Create a singleton instance of the WebTorrent API that can be used across all of my other classes (inspired by Paul Lewis).

export default function TorrentInstance () {

    if (typeof window.TorrentInstance_ !== 'undefined')
    return Promise.resolve(window.TorrentInstance_);

    window.TorrentInstance_ = new WebTorrent();

    return Promise.resolve(window.TorrentInstance_);

Seed all the memos—this is currently a little overkill, but it got the demos working. The this.memos is an Array Instance of VoiceMemos that were stored in our IndexedDB.

seed () {
    this.memos.forEach((memo) => {
        // Seed the memo.
        TorrentInstance().then(torrentClient => {
            memo.audio, // Audio is a blob, but that is ok.
                name: `${memo.title}.webm`,
                comment: memo.description || '',
                creationDate: memo.time || Date.now()
            torrent => {
            memo.torrentURL = torrent.magnetURI;

When the user clicks the Share button, create a custom URL that will contain all the details needed to get access to the torrent and stream the data into another client.

onShareButtonClick(e) {
    MemoModel.get(this.memoId).then( (memo) => {
        return `/share?seeds=${encodeURIComponent(memo.torrentURL)}`;
    }).then(uri => {
        history.pushState({}, "Share seed", uri)

We need to run some custom logic to fetch the torrent when we detect that the user has entered a URL that contains the torrent information. The Voice Memos app has a custom router which looks for a URL that starts with ‘/share’.

          (url) => this.fetchTorrent(location.search.slice(1)),
          (out) => console.log(out)

Now we need to parse out the torrent information, and then using the WebTorrent API fetch the file from the BitTorrent network.

Once it has been fetched we have to then get the file data out of the Blob using a rather hacky XMLHttpRequest.

    const seeds = new URLSearchParams(url);
    let seedURLs = seeds.getAll('seeds');

    for(let seedURL of seedURLs) {
        TorrentInstance().then(torrentClient => {

            torrentClient.add(seedURL, {}, torrent => {

                var file = torrent.files[0];

                file.getBlobURL((err, url) => {

                    var xhr = new XMLHttpRequest();
                    xhr.open('GET', url, true);
                    xhr.responseType = 'blob';
                    xhr.onload = function(e) {
                        if (this.status == 200) {
                            var audioData = this.response;

                            const newMemo = new MemoModel({
                                audio: audioData,
                                volumeData: audioData, // Assuming pre-normalised
                                title: torrent.name.replace(/\.webm$/, ""),
                                torrentURL: torrent.magnetURI,
                                description: ""




Wrapping Up

I will end this all by saying that this is a massive hack and it shouldn’t be used for private data—right now if goes out on the network if people have the URL to the torrent then it will be accessible.

I do think the concept is important, the ability to synchronize data and have no middle-man or server logic in our web apps is an important concept and we should actively consider how we build such experiences.

I’m off to keep playing with this…

The Mobile Zone is brought to you in partnership with Strongloop and IBM.  Visually compose APIs with easy-to-use tooling. Learn how IBM API Connect provides near-universal access to data and services both on-premises and in the cloud.

mobile app,information flow,hack,torrent

Published at DZone with permission of Paul Kinlan, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}