Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Downloading Audio From Azure for MR Apps

DZone's Guide to

Downloading Audio From Azure for MR Apps

To bring audio files to your mixed reality apps, here is how to download them from Azure Blob Storage and implement them in a UWP application.

· IoT Zone ·
Free Resource

All but the most trivial apps have some kind of backend. In the past, I have written about accessing all kinds of services from HoloLens and Mixed Reality apps, but apart from mere data, you can download all kinds of media from external sources and use those in your mixed reality apps.

This is highly useful if you want to change media or use different files depending on some factor, or heck, have some random background music. In fact, what I will be describing is not even mixed reality-specific – the principle can be used in any Unity app, although the code that sits around it to demonstrate the workings definitely only works for mixed reality.

Oh, and by the way, for us Microsoft geeks, “backend” equals “Microsoft Azure” – if not for the competitive pricing, then for the way Microsoft makes it easy for developers to get going (heaven knows this was quite different in the early days). But to be clear: This will work with any backend that hosts files.

This post will be two parts: The first part concentrates on the actual technique of downloading and playing audio files, and the second part will explain the ‘floating audio player’ I built to make this easily demonstrable. The floating part in ‘floating player’ should be taken in the most literal way possible:

image

imageYou can find it in the demo project. You can just jump there if you want to skip all my mumbling.

Global Overview

If you open the project in Unity, you will not see one, but three floating players next to each other. They will all attempt to play one of my ringtones — an excerpt from the Doctor Who theme song (one is a nerd or one is not). But it will attempt to use one of three different audio formats – the well-known MP3, Ogg Vorbis, and WAVE (AKA ye good ole WAV).

image

To hear them, either build the app or just hit the play button. If you choose the second option, you will need to attach an XBox One controller to your PC to steer the cursor and click on the buttons. You will notice the Ogg and the WAV file playing nicely. The MP3 one won’t. In fact, the player just disappears before you even can get to it. We will get to that later.

Using UnityWebRequest

In previous posts, I have shown you either how to use the Unity WWW class, or how to resort to pure UWP code and use HttpClient/HttpRequestMessage. It seems like the WWW class is being deprecated, although I have no official information on that. However, the new (third) kid on the block seems to be UnityWebRequest.

This is a bit of an oddball. It uses a handler. Generally, a UnityWebRequest looks like this:

var request = UnityWebRequest.Get(url);
request.downloadHandler = handler;
yield return request.SendWebRequest();


The second line is optional – if you don’t set the handler, it’s the default DownloadHandler class, which sports a “text” property you can query. This is very useful for accessing data services like an API app on Azure App Service. If you want to download audio files, you will need a DownloadHandlerAudioClip handler.

Putting It in Code

To make this all work easily, I have created the following base class for download loading media:

using System.Collections;
using UnityEngine;
using UnityEngine.Networking;

public abstract class BaseMediaLoader : MonoBehaviour
{
    public string MediaUrl;

    private string _currentMediaUrl;

    protected virtual void Update()
    {
        if (_currentMediaUrl != MediaUrl)
        {
            _currentMediaUrl = MediaUrl;
            StartCoroutine(StartLoadMedia());
        }
    }

    protected abstract IEnumerator StartLoadMedia();

    protected IEnumerator ExecuteRequest(string url, DownloadHandler handler)
    {
        var request = UnityWebRequest.Get(url);
        request.downloadHandler = handler;
        yield return request.SendWebRequest();
    }
}


MediaUrl is a URL that points to a place where the actual audio file can be downloaded – in this case, as SAS link to a file in my own Azure Blob Storage. Every time Update is called, it checks if MediaUrl has been changed, and if so, it starts the StartLoadMedia in the background. ExecuteRequest is a helper method that can be used from a StartLoadMedia. Like this:

using System.Collections;
using UnityEngine;
using UnityEngine.Networking;

public class SoundPlaybackController : BaseMediaLoader
{
    public AudioSource Audio;

    public AudioType TypeAudio = AudioType.OGGVORBIS;

    protected override IEnumerator StartLoadMedia()
    {
        yield return LoadMediaFromUrl(MediaUrl);
    }

    private IEnumerator LoadMediaFromUrl(string url)
    {
        var handler = new DownloadHandlerAudioClip(url, TypeAudio);
        yield return ExecuteRequest(url, handler);
        if (handler.audioClip.length > 0)
        {
            Audio.clip = handler.audioClip;
        }
    }
}


This is a very reduced version of the SoundPlaybackController that is in the demo project, concentrating only on the actual downloading and playing of data.

The override of StartLoadMedia simply calls StartLoadMedia, which proceeds to create a DownloadHandlerAudioClip handler. Now the odd thing is that this handler wants a URL as well as the request. Why this is so, I have no idea. Also notice the fact you need to supply the handler with the type of audio you are going to download – there’s an AudioType enumeration for that.

If the audio has been downloaded successfully, you only have to set the “clip” property of an AudioSource to the handler’s “audioClip” property and call the AudioSource’s “Play” method. Then, you are good to go.

Audio Types Are Important

Since the TypeAudio property is public, the Unity editor makes a nice dropdown for us to select the type of audio:

image

I’ll be the first one to admit, I haven’t heard of most of these file types. Actually, before I started, I only knew MP3 and WAV. I learned to know Ogg Vorbis. And for a good reason, too. I already mentioned the fact the left (MP3) player disappears when you start the code. The SoundPlaybackController that I actually created (not the simple version above) hides the UI of the whole player while actually downloading the audio file. You might have noticed an error in the Unity status bar when you run the code. That actually says:

Streaming of 'mp3?st=2018-02-02t15%3a51%3a00z&se=2020-02-03t17%3a51%3a00z&sp=rl&sv=2017-04-17&sr=b&sig=kymvql5q1yyr%2bqilcxhfc3popwo56vd0ejibennldzw%3d' on this platform is not supported
 UnityEngine.Networking.DownloadHandlerAudioClip:get_audioClip()
 <LoadMediaFromUrl>c__Iterator0:MoveNext() (at Assets/App/Scripts/SoundPlaybackController.cs:34)
 UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)


imageThe code in that player crashes and never gets around to showing the UI again. Well, that’s a bummer. Unity simply does not seem to support MP3 on ‘this platform’, which is apparently UWP. You can convert it to WAV, but unless you like to burn a lot of battery downloading stuff, you must be pretty much out of your mind doing so – Ogg Vorbis is a much better option, as this simple list of files shows.

And Audacity, that good old workhorse of audio-artists and podcasters all around the globe, makes conversion easy, so why not use it, right?

image

Conclusion

The very short version of this blog post: Use UnityWebRequest and DownloadHandlerAudioClip to download an audio clip from Azure Blob Storage, and if you value your users’ bandwidth, their devices’ battery life, and your own sanity – use Ogg Vorbis audio files.

I have not tried all the other audio file types simply because I did not have to do so – Ogg Vorbis works fine. WAV, too, but is absurdly big compared to MP3 and Ogg Vorbis. In addition, I have no idea what those audio types are, how I should create/convert them, and why I should use them. To paraphrase Star Trek’s (TOS) medical officer Dr. Leonard “Bones” McCoy – I am a developer, not an audio engineer. 

Next time, I will explain the workings of the floating audio player around this code.

Topics:
iot ,mixed reality ,mp3 ,ogg vorbis ,wav ,audio ,hololens ,azure blob storage ,tutorial ,unity

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}