Force Feedback in a Dynamic Scene Tutorial

Building on the Basic Force-Feedback tutorial, this guide introduces how to simulate dynamic interactions within Unity, allowing users to feel force feedback from a moving object. This scenario highlights the necessity of high-frequency updates for haptic feedback, which significantly exceeds the typical update rates for visual rendering in Unity.

Introduction

For a compelling haptic experience, especially in dynamic scenes, it's crucial to perform calculations at frequencies above 1kHz. This is in stark contrast to the usual game update loop, which operates around 60Hz. The challenge lies in managing these high-frequency updates alongside the main game loop, ensuring thread-safe data exchange to maintain consistent and accurate force feedback.

Extending the Basic Force-Feedback Setup

Start with the scene setup from the Basic Force-Feedback tutorial. To incorporate dynamic behaviour, we will adapt the SphereForceFeedback script, enabling it to respond to the movement of the sphere and simulate interactions with a dynamically moving object.

Key Modifications

• Dynamic Object Movement: Incorporate logic to update the Sphere's position and velocity based on user input or predefined movement patterns.
• Thread-Safe Data Exchange: Use a ReaderWriterLockSlim to manage concurrent access to shared data between the main and haptic threads.
• Force Calculation Adjustments: Modify the SphereForceFeedback.ForceCalculation method to consider the sphere's velocity, providing realistic feedback based on both position and motion.

Dynamic Interaction

To simulate the moving sphere, you can either manually update its position in the Update method or use a separate component to control its movement based on keyboard input or other interactions. In this example we'll rename the Sphere game object to Moving Ball and add the MovingObject component given in the Tutorials sample.

Adjusting ForceCalculation for Movement

The force feedback calculation now needs to account for the Moving Ball's velocity, adjusting the force based on both the interaction's position and velocity. This provides a more nuanced and realistic haptic sensation, reflecting the dynamic nature of the interaction.

• Add a Vector3 otherVelocity method parameter
• Replace force -= cursorVelocity * damping by force -= (cursorVelocity - otherVelocity) * damping

private Vector3 ForceCalculation(Vector3 cursorPosition, Vector3 cursorVelocity, float cursorRadius,
    Vector3 otherPosition, Vector3 otherVelocity, float otherRadius)
{
    var force = Vector3.zero;

    var distanceVector = cursorPosition - otherPosition;
    var distance = distanceVector.magnitude;
    var penetration = otherRadius + cursorRadius - distance;

    if (penetration > 0)
    {
        // Normalize the distance vector to get the direction of the force
        var normal = distanceVector.normalized;

        // Calculate the force based on penetration
        force = normal * penetration * stiffness;

        // Calculate the relative velocity
        var relativeVelocity = cursorVelocity - otherVelocity;

        // Apply damping based on the relative velocity
        force -= relativeVelocity * damping;
    }

    return force;
}

Thread-Safe Data Exchange

In dynamic scenes where objects move and interact in real-time, ensuring that haptic feedback calculations are based on the latest data without causing data corruption due to concurrent access is crucial. This is where thread-safe data exchange becomes essential.

Key Concepts for Thread-Safe Data Exchange

• Thread-Safe Mechanisms: Utilize ReaderWriterLockSlim to manage concurrent data access. This allows multiple reads or a single write operation, ensuring data integrity.
• Data Reading and Writing:
  • Reading: The haptic thread reads object positions and velocities under a read lock, ensuring it does not interfere with data updates.
  • Writing: Updates to object data by the main thread are done under a write lock, preventing simultaneous reads or writes that could lead to inconsistent data states.

Implementing in Unity

• Struct for Scene Data: To facilitate thread-safe operations, we define a structure that holds all necessary data about the scene. This structure includes the position and velocity of both the Moving Ball and the cursor, as well as their radii. This data structure serves as the foundation for our thread-safe data exchange.

  private struct SceneData
  {
      public Vector3 ballPosition;
      public Vector3 ballVelocity;
      public float ballRadius;
      public float cursorRadius;
  }
  
  private SceneData _cachedSceneData;

• Lock Initialization: A ReaderWriterLockSlim instance is initialized to manage access to the scene data. This lock allows multiple threads to read data simultaneously or exclusively lock the data for a single thread to write, ensuring data integrity during concurrent operations.

  private ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();

• Writing to the Cache with a Write Lock: The SaveSceneData method updates the scene data within a write lock. This ensures that while one thread updates the data, no other thread can read or write, preventing data races and ensuring consistency.

  private void SaveSceneData()
  {
      _cacheLock.EnterWriteLock();
      try
      {
          var t = transform;
          _cachedSceneData.ballPosition = t.position;
          _cachedSceneData.ballRadius = t.lossyScale.x / 2f;
          _cachedSceneData.cursorRadius = inverse3.Cursor.Model.transform.lossyScale.x / 2f;
          _cachedSceneData.ballVelocity = _movableObject.CursorVelocity;
      }
      finally
      {
          _cacheLock.ExitWriteLock();
      }
  }

• Reading from the Cache with a Read Lock: The GetSceneData method retrieves the scene data under a read lock. This allows multiple threads to safely read the data concurrently without interfering with write operations, ensuring that the haptic feedback calculations are based on the latest scene data.

  private SceneData GetSceneData()
  {
      _cacheLock.EnterReadLock();
      try
      {
          return _cachedSceneData;
      }
      finally
      {
          _cacheLock.ExitReadLock();
      }
  }

• Main Thread Data Update: The FixedUpdate method is used to periodically update the scene data in the main thread. This ensures that the haptic feedback calculations have access to the most current data, reflecting the dynamic nature of the scene.

  private void FixedUpdate()
  {
      SaveSceneData();
  }

• Applying Force Calculation with Updated Data: In the OnDeviceStateChanged callback, force calculations are performed using the latest scene data obtained through thread-safe methods. This ensures that the force feedback is accurate and responsive to the dynamic interactions within the scene.

  var sceneData = GetSceneData();
  
  var force = ForceCalculation(device.CursorLocalPosition, device.CursorLocalVelocity, sceneData.cursorRadius,
      sceneData.ballPosition, sceneData.ballVelocity, sceneData.ballRadius);

Gameplay Experience

These script enhancements allow you to interact with a sphere that actively moves across the scene. The haptic feedback dynamically adapts to the sphere's trajectory, offering a more immersive and tactilely rich experience.

Source Files

The complete scene and associated files for this example can be imported from the Tutorials sample in the Unity Package Manager.

The Tutorial sample includes the MovableObject script that is used in multiple examples to control the movement of the attached game object with keyboard input.

SphereForceFeedback.cs

/*
 * Copyright 2024 Haply Robotics Inc. All rights reserved.
 */

using System.Threading;
using Haply.Inverse.Unity;
using Haply.Samples.Tutorials.Utils;
using UnityEngine;

namespace Haply.Samples.Tutorials._4A_DynamicForceFeedback
{
    public class SphereForceFeedback : MonoBehaviour
    {
        // must assign in inspector
        public Inverse3 inverse3;

        [Range(0, 800)]
        // Stiffness of the force feedback.
        public float stiffness = 300f;

        [Range(0, 3)]
        public float damping = 1f;

        #region Thread-safe cached data

        /// <summary>
        /// Represents scene data that can be updated in the Update() call.
        /// </summary>
        private struct SceneData
        {
            public Vector3 ballPosition;
            public Vector3 ballVelocity;
            public float ballRadius;
            public float cursorRadius;
        }

        /// <summary>
        /// Cached version of the scene data.
        /// </summary>
        private SceneData _cachedSceneData;

        private MovableObject _movableObject;

        /// <summary>
        /// Lock to ensure thread safety when reading or writing to the cache.
        /// </summary>
        private readonly ReaderWriterLockSlim _cacheLock = new();

        /// <summary>
        /// Safely reads the cached data.
        /// </summary>
        /// <returns>The cached scene data.</returns>
        private SceneData GetSceneData()
        {
            _cacheLock.EnterReadLock();
            try
            {
                return _cachedSceneData;
            }
            finally
            {
                _cacheLock.ExitReadLock();
            }
        }

        /// <summary>
        /// Safely updates the cached data.
        /// </summary>
        private void SaveSceneData()
        {
            _cacheLock.EnterWriteLock();
            try
            {
                var t = transform;
                _cachedSceneData.ballPosition = t.position;
                _cachedSceneData.ballRadius = t.lossyScale.x / 2f;

                _cachedSceneData.cursorRadius = inverse3.Cursor.Model.transform.lossyScale.x / 2f;

                _cachedSceneData.ballVelocity = _movableObject.CursorVelocity;
            }
            finally
            {
                _cacheLock.ExitWriteLock();
            }
        }

        #endregion

        /// <summary>
        /// Saves the initial scene data cache.
        /// </summary>
        private void Start()
        {
            _movableObject = GetComponent<MovableObject>();
            SaveSceneData();
        }

        /// <summary>
        /// Update scene data cache.
        /// </summary>
        private void FixedUpdate()
        {
            SaveSceneData();
        }

        /// <summary>
        /// Subscribes to the DeviceStateChanged event.
        /// </summary>
        private void OnEnable()
        {
            inverse3.DeviceStateChanged += OnDeviceStateChanged;
        }

        /// <summary>
        /// Unsubscribes from the DeviceStateChanged event.
        /// </summary>
        private void OnDisable()
        {
            inverse3.DeviceStateChanged -= OnDeviceStateChanged;
        }

        /// <summary>
        /// Calculates the force based on the cursor's position and another sphere position.
        /// </summary>
        /// <param name="cursorPosition">The position of the cursor.</param>
        /// <param name="cursorVelocity">The velocity of the cursor.</param>
        /// <param name="cursorRadius">The radius of the cursor.</param>
        /// <param name="otherPosition">The position of the other sphere (e.g., ball).</param>
        /// <param name="otherVelocity">The velocity of the other sphere (e.g., ball).</param>
        /// <param name="otherRadius">The radius of the other sphere.</param>
        /// <returns>The calculated force vector.</returns>
        private Vector3 ForceCalculation(Vector3 cursorPosition, Vector3 cursorVelocity, float cursorRadius,
            Vector3 otherPosition, Vector3 otherVelocity, float otherRadius)
        {
            var force = Vector3.zero;

            var distanceVector = cursorPosition - otherPosition;
            var distance = distanceVector.magnitude;
            var penetration = otherRadius + cursorRadius - distance;

            if (penetration > 0)
            {
                // Normalize the distance vector to get the direction of the force
                var normal = distanceVector.normalized;

                // Calculate the force based on penetration
                force = normal * penetration * stiffness;

                // Calculate the relative velocity
                var relativeVelocity = cursorVelocity - otherVelocity;

                // Apply damping based on the relative velocity
                force -= relativeVelocity * damping;
            }

            return force;
        }

        /// <summary>
        /// Event handler that calculates and send the force to the device when the cursor's position changes.
        /// </summary>
        /// <param name="device">The Inverse3 device instance.</param>
        private void OnDeviceStateChanged(Inverse3 device)
        {
            var sceneData = GetSceneData();

            // Calculate the moving ball force.
            var force = ForceCalculation(device.CursorLocalPosition, device.CursorLocalVelocity, sceneData.cursorRadius,
                sceneData.ballPosition, sceneData.ballVelocity, sceneData.ballRadius);

            // Apply the force to the cursor.
            device.CursorSetLocalForce(force);
        }
    }
}