Integrating ODIN Voice Chat with the FMOD Audio Solution in Unity

Introduction

Welcome to this guide on integrating the ODIN Voice Chat Plugin with the FMOD Audio Solution in Unity. The code used in this guide is available on the ODIN-FMOD Git repository.

What You'll Learn:

• How the

  FMODMicrophoneReader

  and FMODPlaybackComponent scripts work and how to use them in your project
• Properly set up ODIN in Unity when using FMOD as an audio solution
• Deal with limitations and potential pitfalls

warning

Note: This guide assumes that your project has disabled Unity's built-in audio.

danger

Disclaimer: Be aware that the implementation shown here uses Programmer Sounds of the FMOD Engine. While this allows real-time audio data, a big disadvantage of this approach is an increased latency by ~500ms.

Getting Started

To follow this guide, you'll need to have some prerequisites:

• Basic knowledge of Unity
• The FMOD Plugin for Unity, which you can get here
• The ODIN Voice Chat Plugin, available here

To set up FMOD in your project, please follow FMOD's in-depth integration tutorial. You can find the tutorial here.

To set up the ODIN Voice Chat Plugin, please take a look at our Getting-Started guide, which you can find here:

Begin ODIN Getting Started Guide

FMODMicrophoneReader

The

FMODMicrophoneReader

script is an essential part of the FMOD integration. It replaces the default ODIN

MicrophoneReader

component, taking over the microphone input responsibilities by using FMOD. This script is crucial for reading microphone data and sending it to the ODIN servers for voice chat.

You can either follow the Usage setup to drop the

FMODMicrophoneReader

directly into your project, or take a look at how it works to adjust the functionality to your requirements.

Usage

1. Add the

   FMODMicrophoneReader

   script to your OdinManager prefab.
2. Disable the original

   MicrophoneReader

   component.

danger

If you're using ODIN Plugin versions older than 1.5.9, do not remove the

MicrophoneReader

component, as doing so may lead to NullpointerExceptions.

info

The script currently doesn't support automatic device switching or allow for programmatically changing devices. If you'd like to see extensions to this script, feel free to join our Discord server and let us know.

How it works

To read data from the microphone using FMOD, we'll need to perform the following steps:

1. Setup and create a FMOD.Sound object, into which FMOD can store the microphone input data.
2. Start the microphone recording.
3. Continually read the FMOD microphone data and push it to the ODIN servers.

1. Setup

The setup is performed in Unity's Start() method.

Retrieve Microphone Info

You need to retrieve details about the microphone, such as the sampling rate and the number of channels. We'll use this info to configure the FMOD recording sound in the next step and the ODIN microphone streams later on.

FMODUnity.RuntimeManager.CoreSystem.getRecordDriverInfo(_currentDeviceId, out _, 0, out _, 
    out _nativeRate, out _, out _nativeChannels, out _);

Configure Recording Sound Info

After obtaining the input device details, the next action is to set up the CREATESOUNDEXINFO object. This object carries essential metadata that FMOD needs for audio capture.

_recordingSoundInfo.cbsize = Marshal.SizeOf(typeof(CREATESOUNDEXINFO));
_recordingSoundInfo.numchannels = _nativeChannels;
_recordingSoundInfo.defaultfrequency = _nativeRate;
_recordingSoundInfo.format = SOUND_FORMAT.PCMFLOAT;
_recordingSoundInfo.length = (uint)(_nativeRate * sizeof(float) * _nativeChannels);

We use SOUND_FORMAT.PCMFLOAT because ODIN requires this format for microphone data. This avoids the need for audio data conversions later on.

The _recordingSoundInfo.length is set to capture one second of audio. To change the recording duration, you can adjust the formula with a different multiplier.

Create Recording Sound

To hold the captured audio, FMOD requires us to create a FMOD Sound object as shown below.

FMODUnity.RuntimeManager.CoreSystem.createSound("", MODE.LOOP_NORMAL | MODE.OPENUSER,
    ref _recordingSoundInfo,
    out _recordingSound);

Here, we use the MODE.LOOP_NORMAL | MODE.OPENUSER flags in combination with the previously configured _recordingSoundInfo to initialize the _recordingSound object.

2. Recording

At this point, we're ready to start capturing audio. To do so, call the recordStart method from FMOD's core system.

FMODUnity.RuntimeManager.CoreSystem.recordStart(_currentDeviceId, _recordingSound, true);
_recordingSound.getLength(out _recordingSoundLength, TIMEUNIT.PCM);

After initiating the recording, we also get the length of the recorded sound in PCM samples by calling getLength. This value will help us manage the recording buffer in later steps.

3. Continually push microphone data

In the Update() method, we manage the ongoing capture of audio data from the FMOD microphone and its transmission to the ODIN servers. This ensures that the audio stream remains both current and continuously active.

Initialization

The method starts by checking if there is an active

OdinHandler

with valid connections and rooms. If not, it returns immediately.

if (!OdinHandler.Instance || !OdinHandler.Instance.HasConnections || OdinHandler.Instance.Rooms.Count == 0)
       return;

The next step is to find out how much audio has been recorded since the last check. This way, we know how much data to read from the buffer.

FMODUnity.RuntimeManager.CoreSystem.getRecordPosition(_currentDeviceId, out uint recordPosition);
uint recordDelta = (recordPosition >= _currentReadPosition)
    ? (recordPosition - _currentReadPosition)
    : (recordPosition + _recordingSoundLength - _currentReadPosition);

// Abort if no data was recorded
if (recordDelta < 1)
    return;

If the read buffer is too short to hold the new audio data, its size is updated.

if(_readBuffer.Length < recordDelta)
    _readBuffer = new float[recordDelta];

Read Microphone Data

Microphone data is read from the FMOD sound object and copied into the read buffer using FMOD's @lock and the System Marshal.Copy functions.

IntPtr micDataPointer, unusedData;
uint readMicDataLength, unusedDataLength;

_recordingSound.@lock(_currentReadPosition * sizeof(float), recordDelta * sizeof(float), out micDataPointer, out unusedData, out readMicDataLength, out unusedDataLength);
uint readArraySize = readMicDataLength / sizeof(float);
Marshal.Copy(micDataPointer, _readBuffer, 0, (int)readArraySize);
_recordingSound.unlock(micDataPointer, unusedData, readMicDataLength, unusedDataLength);

In this implementation, it's crucial to be aware of the unit differences between FMOD, ODIN, and the system's Marshal.Copy function. FMOD expects the read position and read length to be specified in bytes. In contrast, both ODIN and Marshal.Copy require the lengths to be represented as the number of samples being copied. Since we're recording in the SOUND_FORMAT.PCMFLOAT format, we can use sizeof(float) to easily switch between FMOD's byte-sized units and ODIN's sample-sized units.

Push Microphone Data

After reading, if there is any valid data, it is pushed to the ODIN servers and the current microphone read position is updated.

if (readMicDataLength > 0)
{
    foreach (var room in OdinHandler.Instance.Rooms)
    {
        ValidateMicrophoneStream(room);
        if (null != room.MicrophoneMedia)
            room.MicrophoneMedia.AudioPushData(_readBuffer, (int)readArraySize);
    }
}

_currentReadPosition += readArraySize;
if (_currentReadPosition >= _recordingSoundLength)
    _currentReadPosition -= _recordingSoundLength;

The _currentReadPosition is reset back to zero when it reaches the length of the recording buffer to avoid going out of bounds.

The ValidateMicrophoneStream method ensures that an ODIN microphone stream is set up and configured correctly:

private void ValidateMicrophoneStream(Room room)
{
    bool isValidStream = null != room.MicrophoneMedia &&
        _nativeChannels == (int) room.MicrophoneMedia.MediaConfig.Channels &&
        _nativeRate == (int) room.MicrophoneMedia.MediaConfig.SampleRate;
    if (!isValidStream)
    {
        room.MicrophoneMedia?.Dispose();
        room.CreateMicrophoneMedia(new OdinMediaConfig((MediaSampleRate)_nativeRate,
            (MediaChannels)_nativeChannels));
    }
}

By understanding and implementing these steps, you should be able to continually read FMOD microphone data and push it to the ODIN servers, thereby keeping your audio stream up-to-date.