Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Building a Floating Audio Player in Mixed Reality

DZone's Guide to

Building a Floating Audio Player in Mixed Reality

Learn how to build a cool floating audio player for mixed reality projects while dodging a few pitfalls during Unity development.

· IoT Zone ·
Free Resource

imageAs I promised in my previous blog post, I would write about how I created the floating audio player designed to easily demonstrate how to download and play audio files in mixed reality (or actually, just Unity, because the code is not MR-specific). I kind of skipped over the UI side. In this post, I am going to talk a little more about the floating audio player itself. This code is using the Mixed Reality Toolkit and so actually is mixed reality-specific.

Dissecting the AudioPlayer Prefab

The Main Game Object

imageThe AudioPlayer consists of two other prefabs, a SquareButton and a Slider. I have talked about this button before, so I won’t go over that one in detail again. The main game object of the AudioPlayer has an AudioSource and two extra scripts. The simple version of the Sound Playback Controller was already described in the previous blog post and will be handled in more detail here. The other script is a standard Billboard script from the Mixed Reality Toolkit. It essentially keeps the object rotated toward the camera, so you will never see it from the side or back, where it’s hard to read and operate. Note that I have restricted the pivot axis to Y, so it only rotates over a vertical axis.

The Button

imageIt’s a fairly standard SquareButton, and I have set the text and icon as I described here. Now that button only shows in the editor, the runtime text and the icon are set by a simple script that toggles icon and text, so that the button cycles between being a “Play” and a “Pause” button. That script is pretty easy:

using HoloToolkit.Unity.InputModule;
using UnityEngine;

public class IconToggler : MonoBehaviour, IInputClickHandler
{
    public Texture2D Icon1;

    public Texture2D Icon2;

    public string Text1;

    public string Text2;

    private TextMesh _textMesh;

    private GameObject _buttonFace;

    void Awake ()
    {
        _buttonFace = gameObject.transform.
           Find("UIButtonSquare/UIButtonSquareIcon").gameObject;
        var text = gameObject.transform.Find("UIButtonSquare/Text").gameObject;
        _textMesh = text.GetComponent<TextMesh>();
        SetBaseState();
    }

    public void SetBaseState()
    {
       _textMesh.text = Text1;
       _buttonFace.GetComponent<Renderer>().sharedMaterial.mainTexture = Icon1;
    }

    private float _lastClick;

    public void OnInputClicked(InputClickedEventData eventData)
    {
        if (Time.time - _lastClick > 0.1)
        {
            _lastClick = Time.time;
            Toggle();
        }
    }

    public void Toggle()
    {
        var material = _buttonFace.GetComponent<Renderer>().sharedMaterial;
        material.mainTexture = material.mainTexture == Icon1 ? Icon2 : Icon1;
       _textMesh.text = _textMesh.text == Text1 ? Text2 : Text1;
    }
}


It has four public properties, as already is visible in the image: Image1 and Text1 for the default image and text (“Play”), Image 2 and Text 2 for the alternate image and text (“Pause”). The Awake method grabs some objects within the button itself, then sets the base state – the default icon and text.

It also implements IInputClickHandler, so the user can tap it. In OnInputClicked, it calls the Toggle method. That then toggles both text and image. Notice there’s simple time-based guard OnInputClicked. This is to prevent the button from sending a burst of click events. In the Unity editor, I mostly get two clicks every time I press the XBox controller's A button, and then nothing happens. Annoying, but easily mitigated this way.

The Slider

I can be short about that one. I did not create that, but simply nicked it from the Mixed Reality Toolkit Examples. It sits in HoloToolkit-Examples\UX\Prefabs. I like making stuff, but I like  stealing  reusing stuff even more.

The extended Sound Playback Controller

Let’s start at Start! Note: The BaseMediaLoader was handled in the previous blog post:

public class SoundPlaybackController : BaseMediaLoader
{
    public AudioSource Audio;

    public GameObject Slider;

    public GameObject Button;

    private SliderGestureControl _sliderControl;

    private IconToggler _iconToggler;

    public AudioType TypeAudio = AudioType.OGGVORBIS;

    void Start()
    {
        _sliderControl = Slider.GetComponent<SliderGestureControl>();
        _sliderControl.OnUpdateEvent.AddListener(ValueUpdated);
        Slider.SetActive(false);
        Button.SetActive(false);
        _iconToggler = Button.GetComponent<IconToggler>();
    }
}


In the Start method, we first grab a bunch of stuff. Note the fact that we not only turn off the slider control but also actually attach an event handler to that.

We continue with StartLoadMedia and LoadMediaFromUrl:

protected override IEnumerator StartLoadMedia()
{
    Slider.SetActive(false);
    Button.SetActive(false);
    yield return LoadMediaFromUrl(MediaUrl);
}

private IEnumerator LoadMediaFromUrl(string url)
{
    var handler = new DownloadHandlerAudioClip(url, TypeAudio);
    yield return ExecuteRequest(url, handler);
    if (handler.audioClip.length > 0)
    {
        Audio.clip = handler.audioClip;
        _sliderControl.SetSpan(0, Audio.clip.length);
        Slider.SetActive(true);
        Button.SetActive(true);
        _iconToggler.SetBaseState();
    }
}


The override from StartLoadMedia in this version turns off the whole UI while we are actually loading data, and turns it on when we are done loading. Since that fails when we load MP3s, the MP3 player in the demo project disappears on startup. The others one disappear too, in fact, but immediately appear again since we are loading small clips. This happens so fast you can’t even see it.

LoadMediaFromUrl not only executes the request and sets the downloaded clip to the Audio Source, as we saw before, but we also set the span of the Slider Control between 0 and the length of the AudioClip in seconds. Easy, right?

Now for the Update method, which, as you know, is called 60 times per second. That is the trick to keeping the slider equal to the current time of the clips that are now playing:

protected override void Update()
{
    base.Update();
    if (Audio.isPlaying)
    {
        _sliderControl.SetSliderValue(Audio.time);
    }
    if (Mathf.Abs(Audio.time - _sliderControl.MaxSliderValue) < 0.1f)
    {
        Audio.Stop();
        Audio.time = 0;
        _iconToggler.SetBaseState();
        _sliderControl.SetSliderValue(0);
    }
}


Thus if the audio clip plays, the slider moves along. It’s not quite rocket science. If the clip has nearly finished playing, it is stopped and everything is set to the base state: the icon, the time of the audio clip, and the slider are set to 0 again.

And finally – remember that event handler we added to the OnValueUpdated event of the slider? Guess what:

private void ValueUpdated()
{
    Audio.time = _sliderControl.SliderValue;
}


It’s the opposite of the third line of Update – now we set the Audio time to the Slider value.

Conclusion

And that’s it. You can simply use some out-of-the-box components in the Mixed Reality Toolkit and/or its examples to build a simple but effective control to play audio. You can grab the demo project (it’s still the same) from here.

Topics:
iot ,mixed reality ,floating ,audio player ,unity ,tutorial

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}