Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Measuring User Movement in Mixed Reality Apps

DZone's Guide to

Measuring User Movement in Mixed Reality Apps

How to measure user movement, specifically speed, movement, and rotation, in mixed reality apps.

· IoT Zone ·
Free Resource

For a business HoloLens app that I am currently developing — as well as for my app Walk the World — I needed a way to see if a user is moving or rotating in excess of a certain speed. I needed to do this to make certain that the control elements are floating when in view and disappearing when he/she is on the move, and, then, come back once the movement stops. I will later describe in greater detail how this is used, but, first, I want to provide an easy helper behavior to sample and measure speed, movement, and rotation.

The Actual Tracker

using HoloToolkit.Unity;
using UnityEngine;

namespace HoloToolkitExtensions.Utilities
{
    public class CameraMovementTracker : Singleton<CameraMovementTracker>
    {
        [SerializeField]
        private readonly float _sampleTime = 1.0f;

        private Vector3 _lastSampleLocation;
        private Quaternion _lastSampleRotation;
        private float _lastSampleTime;

        public float Speed { get; private set; }
        public float RotationDelta { get; private set; }
        public float Distance { get; private set; }

        void Start()
        {
            _lastSampleTime = Time.time;
            _lastSampleLocation = CameraCache.Main.transform.position;
            _lastSampleRotation = CameraCache.Main.transform.rotation;
        }
   }
}


Here, the behavior is implemented as a singleton class. Although this is not necessary, it makes sense to do so because there can only be one mixed reality camera when there is only one user. There is also only one public property — the sample time. The idea of a sample time is simple — if you want to measure speed, rotation, or movement, you have to do so over time. The default samples location and rotation every second. Then, it's up to you to decide to do something with it. At the start, it simply sets the sample time at now, the first sample location to the camera's current location, and the rotation to its current rotation.

In the update method (called every 60th of a second), we simply check whether the sample time period has expired. Then, we get a new sample of location and rotation.

void Update()
{
    if (Time.time - _lastSampleTime > _sampleTime)
    {
        Speed = CalculateSpeed();
        RotationDelta = CalculateRotation();
        Distance = CalculateDistanceCovered();
        _lastSampleTime = Time.time;
        _lastSampleLocation = CameraCache.Main.transform.position;
        _lastSampleRotation = CameraCache.Main.transform.rotation;
    }
}


The calculations itself are rather simple:

private float CalculateDistanceCovered()
{
    return Vector3.Distance(_lastSampleLocation, CameraCache.Main.transform.position);
}

private float CalculateSpeed()
{
    // return speed in km/h
    return CalculateDistanceCovered() / (Time.time - _lastSampleTime) * 3.6f;
}

private float CalculateRotation()
{
    return Mathf.Abs(Quaternion.Angle(_lastSampleRotation,                                       CameraCache.Main.transform.rotation));
}


The distance is the difference between the previous and the current camera position.Time.timeis always in seconds, so dividing the speed through the elapsed time results in the speed in meters per second. Multiplying it by 3.6 makes that km/h. I presumed that to be a unit most people have a feeling for, but feel free to adapt this to your needs and have it return miles, yards, feet, furlongs, stadia or your outdated/obscure distance unit of choice!

So, What Is This Good For?

Well, simply put, this is done to take action when some threshold for rotation or movement is crossed. Like I mentioned, it's particularly useful for determining if control elements that should be more or less in the user's field of view should be moved. However, do not use it too often or too brusque, otherwise, it is not possible to properly view or interact with them. In the demo project, I have created a demo behavior that shows speed, distance covered, and rotation in a floating text., It also uses that data to decide whether or not it's time to move the text back into view.

image

This is a picture of the text just after it was moving back in view. It will rapidly go back to showing all zeroes, as it only measures these values over the last second.

Here is the demo behavior in a bit more detail:

using HoloToolkitExtensions.Utilities;
using UnityEngine;

public class ShowCameraActions : MonoBehaviour
{
    private TextMesh _mesh;

    [SerializeField]
    private float _rotationThreshold = 10f;

    [SerializeField]
    private float _moveTreshold = 0.4f;

    [SerializeField]
    private float _moveTime = 0.2f;

    private bool _isBusy;

    void Start()
    {
        _mesh = GetComponentInChildren<TextMesh>();
        MoveText();
    }

    void Update()
    {
        SetText();
        if ((CameraMovementTracker.Instance.RotationDelta > _rotationThreshold ||
            CameraMovementTracker.Instance.Distance > _moveTreshold ) && !_isBusy)
        {
            MoveText();
        }
    }

    private void MoveText()
    {
        _isBusy = true;
        LeanTween.move(gameObject, 
                        LookingDirectionHelpers.CalculatePositionDeadAhead(), _moveTime).
                  setEaseInOutSine().setOnComplete(() => _isBusy = false);
    }

    private void SetText()
    {
        var text = 
            string.Format(             "Speed: {0:00.00} km/h - Rotation: {1:000.0}° - Moved {2:00.0}m",
            CameraMovementTracker.Instance.Speed,
            CameraMovementTracker.Instance.RotationDelta,
            CameraMovementTracker.Instance.Distance);
        if (_mesh.text != text)
        {
            _mesh.text = text;
            Debug.Log(text);
        }
    }
}


Long story short:

  • The text will be updated in every call to update, which is 60 times per second. But, since the CameraMovementTrackerupdates itself only once a second by default, you should see the text change only once a second. I have also included a Debug.Log so you can see the numbers change when the text is still outside of your view. This of course only works in the Unity editor.
  • If the rotation threshold (10 degrees) or movement threshold (0.4 meters) is exceeded, the behavior will attempt to move the text back into view (if it is not already doing so), using good old LeanTween. The "setEaseInOutSine" will make the movement start and stop fluently.

Conclusion

It's not hard to measure these things and the code is not complicated. But, if I need to make something the 3rd time, it's time to make it into a generalized reusable class. And, there you have it. Have fun with the demo project.

Topics:
iot ,tutorial ,mixed reality app development ,mixed reality ,user movement

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}