Skip to main content
Version: 1.x

Integrating ODIN Voice Chat with the FMOD Audio Solution in Unity

FMOD and ODIN

Introduction​

Welcome to this guide on integrating the ODIN Voice Chat Plugin with the FMOD Audio Solution in Unity. The code used in this guide is available on the ODIN-FMOD Git repository.

What You'll Learn:

warning

Note: This guide assumes that your project has disabled Unity's built-in audio.

info

The playback uses an FMOD user stream that pulls voice data from ODIN. FMOD's default decode buffer for such streams is 400 ms, which used to add a noticeable delay. The FMODPlaybackComponent shrinks that buffer to 40 ms by default and keeps ODIN's own buffer small, see Latency settings.

Getting Started​

To follow this guide, you'll need to have some prerequisites:

  • Basic knowledge of Unity
  • The FMOD Plugin for Unity, which you can get here
  • The ODIN Voice Chat Plugin, available here

The sample project was created with:

  • Unity 6000.3.27f1
  • ODIN Voice Unity SDK 1.7.9
  • FMOD for Unity 2.03.15 and FMOD Studio 2.03

To set up FMOD in your project, please follow FMOD's in-depth integration tutorial. You can find the tutorial here.

To set up the ODIN Voice Chat Plugin, please take a look at our Getting-Started guide, which you can find here:

Begin ODIN Getting Started Guide

Trying out the sample​

The sample scene Assets/Odin-FMod/OdinFMod.unity joins the ODIN room default on start. You can grab and move both the audio listener and the remote peers with the left mouse button to hear the 3D positional audio. Press M to mute or unmute your microphone.

To test it, create a build and start it alongside the Editor. Alternatively, use the ODIN web app together with the Editor. In that case, enter the same Auth Access Key in the web app and connect to the default room.

Project setup​

Disabling the Unity audio engine​

To inform ODIN that the Unity audio engine is disabled, e.g. when using FMOD exclusively for audio processing, add the ODIN_UNITY_AUDIO_ENGINE_DISABLED flag to the Scripting Define Symbols in Project Settings > Player. This flag ensures that ODIN won't try to use Unity's audio systems.

OdinHandler​

The

OdinHandler

can only create playback automatically with the Unity audio system. Switch its playback setting from Playback auto creation to Manual positional audio and create the

FMODPlaybackComponent

instances yourself. The

OdinAutoJoin

script of the sample shows how to do this, see Spawning playback components.

Overview​

The integration consists of these scripts:

ScriptReplacesJob

FMODMicrophoneReader

MicrophoneReader

Records the microphone with FMOD and pushes the samples to ODIN

FMODPlaybackComponent

PlaybackComponent

Plays one remote ODIN media stream as an FMOD user stream

OdinAutoJoin

Playback auto creation of the OdinHandlerJoins a room and spawns or destroys one playback object per media stream

FMODPlayback3DPosition

-Optional: spatializes a playback component

FMODCoreListener

-Optional: moves the FMOD Core listener with your camera or player

FMODMicrophoneReader​

The

FMODMicrophoneReader

script is an essential part of the FMOD integration. It replaces the default ODIN

MicrophoneReader

component, taking over the microphone input responsibilities by using FMOD. This script is crucial for reading microphone data and sending it to the ODIN servers for voice chat.

You can either follow the Usage setup to drop the

FMODMicrophoneReader

directly into your project, or take a look at how it works to adjust the functionality to your requirements.

Usage​

  1. Add the

    FMODMicrophoneReader

    script to your OdinManager prefab.
  2. Disable the original

    MicrophoneReader

    component.

The OdinManager prefab after adding the FMODMicrophoneReader and disabling the original MicrophoneReader

The script records from FMOD's default recording device. Press the Mute Key (default M) to toggle the microphone, or call SetMuted(bool) from your own code. The Muted property returns the current state.

info

The script currently doesn't support automatic device switching or allow for programmatically changing devices. If you'd like to see extensions to this script, feel free to join our Discord server and let us know.

How it works​

To read data from the microphone using FMOD, we'll need to perform the following steps:

  1. Setup and create a FMOD.Sound object, into which FMOD can store the microphone input data.
  2. Start the microphone recording.
  3. Continually read the FMOD microphone data in 20 ms chunks and push it to the ODIN servers.

1. Setup​

The setup is performed in Unity's Start() method.

Retrieve Microphone Info

First, the script checks that FMOD found at least one recording device. Then it retrieves details about the microphone, such as the sampling rate and the number of channels. We'll use this info to configure the FMOD recording sound in the next step and the ODIN microphone streams later on.

FMODUnity.RuntimeManager.CoreSystem.getRecordNumDrivers(out int numDrivers, out _);
if (numDrivers <= 0)
{
Debug.LogWarning($"{nameof(FMODMicrophoneReader)} found no recording device");
return;
}

FMODUnity.RuntimeManager.CoreSystem.getRecordDriverInfo(_currentDeviceId, out string deviceName, 256, out _,
out _nativeRate, out _, out _nativeChannels, out _);

Prepare the chunk buffers

ODIN encodes mono or stereo audio. Devices with more than two channels have to be downmixed to mono. The script reads and sends the microphone data in chunks of 20 ms and allocates the buffers for these chunks once:

_odinChannels = _nativeChannels > 2 ? 1 : _nativeChannels;
_chunkFrames = (uint)(_nativeRate / 50); // 20 ms
_chunk = new float[_chunkFrames * _nativeChannels];
_odinChunk = _odinChannels == _nativeChannels ? _chunk : new float[_chunkFrames * _odinChannels];

Configure Recording Sound Info

Next, set up the CREATESOUNDEXINFO object. This object carries essential metadata that FMOD needs for audio capture.

_recordingSoundInfo.cbsize = Marshal.SizeOf(typeof(CREATESOUNDEXINFO));
_recordingSoundInfo.numchannels = _nativeChannels;
_recordingSoundInfo.defaultfrequency = _nativeRate;
_recordingSoundInfo.format = SOUND_FORMAT.PCMFLOAT;
_recordingSoundInfo.length = (uint)(_nativeRate * sizeof(float) * _nativeChannels);

We use SOUND_FORMAT.PCMFLOAT because ODIN requires this format for microphone data. This avoids the need for audio data conversions later on.

The _recordingSoundInfo.length is set to capture one second of audio. FMOD records into this sound as a ring buffer, so it only needs to hold enough data to bridge the time between two Update() calls.

Create Recording Sound

To hold the captured audio, FMOD requires us to create a FMOD Sound object as shown below.

FMODUnity.RuntimeManager.CoreSystem.createSound("", MODE.LOOP_NORMAL | MODE.OPENUSER,
ref _recordingSoundInfo,
out _recordingSound);

Here, we use the MODE.LOOP_NORMAL | MODE.OPENUSER flags in combination with the previously configured _recordingSoundInfo to initialize the _recordingSound object.

2. Recording​

At this point, we're ready to start capturing audio. To do so, call the recordStart method from FMOD's core system.

FMODUnity.RuntimeManager.CoreSystem.recordStart(_currentDeviceId, _recordingSound, true);
_recordingSound.getLength(out _recordingSoundLength, TIMEUNIT.PCM);

After initiating the recording, we also get the length of the recorded sound in PCM samples by calling getLength.

When the component is destroyed, the recording is stopped and the sound released:

private void OnDestroy()
{
FMODUnity.RuntimeManager.CoreSystem.recordStop(_currentDeviceId);
_recordingSound.release();
}

3. Continually push microphone data​

In the Update() method, we manage the ongoing capture of audio data from the FMOD microphone and the transmission to the ODIN servers.

Initialization

The method starts by checking if there is an active

OdinHandler

with valid connections and rooms, and whether the setup found a usable recording device.

if (!OdinHandler.Instance || !OdinHandler.Instance.HasConnections || OdinHandler.Instance.Rooms.Count == 0)
return;
if (_chunk == null)
return; // no usable recording device

The next step is to find out how much audio has been recorded since the last check (in PCM frames). This way we know how much data to read from the ring buffer.

FMODUnity.RuntimeManager.CoreSystem.getRecordPosition(_currentDeviceId, out uint recordPosition);
uint available = (recordPosition >= _currentReadPosition)
? (recordPosition - _currentReadPosition)
: (recordPosition + _recordingSoundLength - _currentReadPosition);

Skip stale audio

If the application stalls, e.g. during a scene load or while the Editor is paused, old audio piles up in the ring buffer. Instead of sending it late, the script skips everything but the most recent 200 ms:

uint maxBacklog = (uint)(_nativeRate / 5); // 200 ms
if (available > maxBacklog)
{
_currentReadPosition = (_currentReadPosition + available - maxBacklog) % _recordingSoundLength;
available = maxBacklog;
}

Read Microphone Data

As long as at least one full 20 ms chunk is available, the script copies it from the FMOD sound into the chunk buffer using FMOD's @lock.

uint bytesPerFrame = sizeof(float) * (uint)_nativeChannels;
while (available >= _chunkFrames)
{
_recordingSound.@lock(_currentReadPosition * bytesPerFrame, _chunkFrames * bytesPerFrame,
out IntPtr ptr1, out IntPtr ptr2, out uint len1, out uint len2);
int samples1 = (int)(len1 / sizeof(float));
Marshal.Copy(ptr1, _chunk, 0, samples1);
// the ring buffer wrapped, the rest of the chunk starts at the beginning of the sound
if (ptr2 != IntPtr.Zero && len2 > 0)
Marshal.Copy(ptr2, _chunk, samples1, (int)(len2 / sizeof(float)));
_recordingSound.unlock(ptr1, ptr2, len1, len2);

if (_odinChunk != _chunk)
DownmixToMono(_chunk, _odinChunk, _nativeChannels);

// push to ODIN, see below

_currentReadPosition = (_currentReadPosition + _chunkFrames) % _recordingSoundLength;
available -= _chunkFrames;
}

Here it's important to be aware of the unit differences between FMOD, ODIN, and the system's Marshal.Copy function:

  • FMOD expects the read position and read length to be specified in bytes. One PCM frame holds one float per channel, so bytesPerFrame converts frames to bytes.
  • Marshal.Copy and ODIN expect the number of samples (floats) being copied.
  • The read position and the record position are counted in frames.

When a chunk reaches past the end of the ring buffer, @lock returns two pointers: ptr1 for the part up to the end of the sound and ptr2 for the rest, which starts at the beginning of the sound. Copying both parts ensures that no audio is lost when the buffer wraps around.

Downmix to mono

For devices with more than two channels, DownmixToMono averages the channels of each interleaved frame:

private static void DownmixToMono(float[] input, float[] output, int channels)
{
for (int frame = 0; frame < output.Length; frame++)
{
float sum = 0f;
for (int channel = 0; channel < channels; channel++)
sum += input[frame * channels + channel];
output[frame] = sum / channels;
}
}

Push Microphone Data

Each chunk is pushed to the microphone stream of every joined room:

foreach (var room in OdinHandler.Instance.Rooms)
{
ValidateMicrophoneStream(room);

// a stream re-created by ValidateMicrophoneStream starts unmuted
if (null != room.MicrophoneMedia)
{
if (room.MicrophoneMedia.IsMuted != Muted)
room.MicrophoneMedia.MuteStream(Muted);
room.MicrophoneMedia.AudioPushData(_odinChunk, _odinChunk.Length);
}
}

The ValidateMicrophoneStream method ensures that an ODIN microphone stream is set up and configured with the sample rate of the device and the channel count sent to ODIN:

private void ValidateMicrophoneStream(Room room)
{
bool isValidStream = null != room.MicrophoneMedia &&
_odinChannels == (int)room.MicrophoneMedia.MediaConfig.Channels &&
_nativeRate == (int)room.MicrophoneMedia.MediaConfig.SampleRate;
if (!isValidStream)
{
room.MicrophoneMedia?.Dispose();
room.CreateMicrophoneMedia(new OdinMediaConfig((MediaSampleRate)_nativeRate,
(MediaChannels)_odinChannels));
}
}

Muting

SetMuted calls MuteStream on the microphone stream of every room. Because a stream that was re-created by ValidateMicrophoneStream starts unmuted, the push loop above re-applies the mute state before pushing data.

public void SetMuted(bool muted)
{
Muted = muted;
if (OdinHandler.Instance)
foreach (var room in OdinHandler.Instance.Rooms)
room.MicrophoneMedia?.MuteStream(muted);
}

FMODPlaybackComponent​

The

FMODPlaybackComponent

replaces ODIN's

PlaybackComponent

. After setting the room name, peer id and media stream id, the component automatically connects to the ODIN media stream and plays back the voice data using the FMOD engine.

Usage​

  1. Create a prefab with the

    FMODPlaybackComponent

    . The sample ships with the FModPlaybackComponent prefab in Assets/Odin-FMod.
  2. Optionally add the

    FMODPlayback3DPosition

    script to the prefab for 3D positional audio.
  3. Instantiate one prefab per remote media stream and set RoomName, PeerId and MediaStreamId.

Spawning playback components​

The

OdinAutoJoin

script of the sample joins a room and listens to the OnMediaAdded and OnMediaRemoved events of the

OdinHandler

to spawn and destroy the playback objects:

private void OnMediaAdded(object roomObject, MediaAddedEventArgs mediaAddedEventArgs)
{
if (roomObject is Room room)
{
FMODPlaybackComponent newPlayback = Instantiate(playbackPrefab);
newPlayback.transform.position = transform.position;
newPlayback.RoomName = room.Config.Name;
newPlayback.PeerId = mediaAddedEventArgs.PeerId;
newPlayback.MediaStreamId = mediaAddedEventArgs.Media.Id;
_instantiatedObjects.Add(newPlayback);
}
}

private void OnMediaRemoved(object roomObject, MediaRemovedEventArgs mediaRemovedEventArgs)
{
if (roomObject is Room room)
{
for (int i = _instantiatedObjects.Count - 1; i >= 0; i--)
{
var comp = _instantiatedObjects[i];
if (comp.RoomName == room.Config.Name && comp.PeerId == mediaRemovedEventArgs.Peer.Id &&
comp.MediaStreamId == mediaRemovedEventArgs.MediaStreamId)
{
_instantiatedObjects.RemoveAt(i);
Destroy(comp.gameObject);
}
}
}
}

In a game, you'd usually parent the playback object to the player object that belongs to the peer, so the voice moves with the player.

Latency settings​

The component exposes three settings in the Inspector:

  • Decode Buffer Milliseconds: size of FMOD's decode buffer for the voice stream. FMOD's default is 400 ms, the sample prefab uses 40 ms. Values of 40 to 50 ms play without dropouts, while 20 ms produces crackles. Set it to 0 to keep FMOD's default. You can override the value globally in builds with the -decodebuffer <ms> command line argument.
  • Target Buffer Milliseconds: the amount of voice data ODIN should buffer internally during playback (default 40 ms). FMOD reads the voice at exactly the playback rate, so any delay that builds up in ODIN's buffer (at startup, after a network hiccup or while FMOD isn't reading) would otherwise stay forever. If ODIN buffers more than this value, the voice plays 2% faster, which is practically inaudible, until the buffer is back at the target level.
  • Max Buffer Milliseconds: above this buffer level, the excess is discarded at once instead of played faster, e.g. after the channel was paused or the mixer suspended (default 400 ms). ODIN itself can buffer up to 2 seconds of voice data.

How it works​

The playback is set up in three steps:

  1. Create an FMOD user stream whose PCM read callback pulls voice data from the ODIN media stream.
  2. Fill the stream from the callback whenever FMOD needs more data.
  3. Keep ODIN's buffer small by catching up or discarding excess data.

1. Create the stream​

In Start(), the component configures a CREATESOUNDEXINFO with ODIN's remote sample rate and channel count, the PCMFLOAT format and a PCM read callback. If a decode buffer size is set, it replaces FMOD's 400 ms default:

int playBackRate = (int)OdinHandler.Config.RemoteSampleRate;
int numChannels = (int)OdinHandler.Config.RemoteChannels;

_createSoundInfo.cbsize = Marshal.SizeOf(typeof(FMOD.CREATESOUNDEXINFO));
_createSoundInfo.numchannels = numChannels;
_createSoundInfo.defaultfrequency = playBackRate;
_createSoundInfo.format = SOUND_FORMAT.PCMFLOAT;
_pcmReadCallback = new SOUND_PCMREAD_CALLBACK(PcmReadCallback);
_createSoundInfo.pcmreadcallback = _pcmReadCallback;
_createSoundInfo.length = (uint)(playBackRate * sizeof(float) * numChannels);

int decodeBufferMilliseconds = CommandLineInt("-decodebuffer", DecodeBufferMilliseconds);
if (decodeBufferMilliseconds > 0)
_createSoundInfo.decodebuffersize = (uint)(playBackRate * decodeBufferMilliseconds / 1000);

The delegate is stored in the _pcmReadCallback field, so the garbage collector doesn't collect it while FMOD still calls it.

The 3D mode has to be set when the sound is created, because switching a playing stream to 3D is unreliable. If an

FMODPlayback3DPosition

is on the same GameObject, the stream is created in 3D mode with the selected rolloff. The channel starts paused, so the first mixed block is panned to the right position from the start. Once the position is set, the channel is unpaused:

bool spatialize = TryGetComponent(out FMODPlayback3DPosition spatializer);
MODE mode = MODE.OPENUSER | MODE.LOOP_NORMAL | (spatialize ? MODE._3D | spatializer.RolloffMode : MODE._2D);
FMODUnity.RuntimeManager.CoreSystem.createStream("", mode, ref _createSoundInfo, out _playbackSound);
FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out ChannelGroup masterChannelGroup);
FMODUnity.RuntimeManager.CoreSystem.playSound(_playbackSound, masterChannelGroup, true, out _playbackChannel);
// highest priority, so FMOD virtualizes other channels first when it runs out of real channels
_playbackChannel.setPriority(0);
if (spatialize)
{
ATTRIBUTES_3D attributes3D = FMODUnity.RuntimeUtils.To3DAttributes(transform);
_playbackChannel.set3DAttributes(ref attributes3D.position, ref attributes3D.velocity);
}
_playbackChannel.setPaused(false);

Finally, the sound handle is registered in a static dictionary, so the static callback can find the component:

lock (_soundToPlaybackMap)
{
_soundToPlaybackMap.Add(_playbackSound.handle, this);
}

2. Fill the stream​

FMOD calls PcmReadCallback on its own thread whenever the stream needs more data. The callback has to be static and marked with [AOT.MonoPInvokeCallback] to work on IL2CPP platforms. It looks up the component by the sound handle and reads the requested amount of samples from the ODIN media stream. If ODIN buffers more than Max Buffer Milliseconds, the excess is discarded:

PlaybackStream playbackMedia = playbackComponent.FindOdinMediaStream();
if (null != playbackMedia && OdinHandler.Instance?.HasConnections == true && !playbackMedia.IsInvalid)
{
// after a longer interruption, discard the excess at once instead of catching up slowly
int buffered = playbackMedia.AudioDataLength();
if (buffered > playbackComponent._maxBufferSamples)
{
int discard = buffered - playbackComponent._targetBufferSamples;
if (playbackComponent._readBuffer.Length < discard)
playbackComponent._readBuffer = new float[discard];
playbackMedia.AudioReadData(playbackComponent._readBuffer, discard);
}

uint odinReadResult = playbackMedia.AudioReadData(playbackComponent._readBuffer, requestedDataArrayLength);
if (Utility.IsError(odinReadResult))
Debug.LogWarning($"{nameof(FMODPlaybackComponent)} AudioReadData failed with error code:{odinReadResult}");
else
Marshal.Copy(playbackComponent._readBuffer, 0, data, requestedDataArrayLength);
}

FindOdinMediaStream retrieves the stream from RoomName, PeerId and MediaStreamId on every call, so the component never holds on to a media stream that was already removed.

3. Catch up on buffer buildup​

FMOD reads at exactly the playback rate, so any delay that has built up in ODIN's buffer would never go away on its own. In Update(), the component checks ODIN's buffer level and plays the channel 2% faster (can be adjusted) while it is above Target Buffer Milliseconds:

bool catchUp = playbackMedia.AudioDataLength() > _targetBufferSamples;
if (catchUp != _catchingUp)
{
_catchingUp = catchUp;
_playbackChannel.setFrequency(_sampleRate * (catchUp ? CatchUpSpeed : 1f));
}

When the component is destroyed, it removes its entry from the dictionary first, so a running callback can't find it anymore, and then releases the sound.

3D positional audio​

FMODPlayback3DPosition​

Add the

FMODPlayback3DPosition

script next to an

FMODPlaybackComponent

to spatialize the voice. If present, the playback component creates its FMOD stream in 3D mode. The script then updates the channel's position from the GameObject's transform every frame:

private void LateUpdate()
{
if (_playbackComponent && _playbackComponent.FMODPlaybackChannel.hasHandle())
{
ATTRIBUTES_3D attributes3D = FMODUnity.RuntimeUtils.To3DAttributes(transform);
_playbackComponent.FMODPlaybackChannel.set3DAttributes(ref attributes3D.position, ref attributes3D.velocity);
}
}

Settings:

  • Min Distance: the voice plays at full volume up to this distance.
  • Max Distance: the voice is silent beyond this distance (linear rolloffs), or stops getting quieter (inverse rolloff).
  • Rolloff: how the volume falls off between min and max distance.
    • Inverse: realistic falloff that never reaches silence.
    • Linear: fades evenly to silence from min to max distance.
    • LinearSquare (default): fades to silence at the max distance, dropping faster the closer you are to the speaker.
note

The rolloff setting is read when the playback stream is created. Changing it at runtime doesn't change the playback behaviour.

FMODCoreListener​

The voice plays on FMOD Core channels, which are spatialized against the FMOD Core listener. FMOD's StudioListener only moves the Studio listener used by Studio events, so without further changes the Core listener would stay at the world origin.

Add the

FMODCoreListener

script to the same GameObject as your StudioListener, usually the camera or player object. It copies the position and orientation into the FMOD Core listener every frame:

public class FMODCoreListener : MonoBehaviour
{
private void LateUpdate()
{
ATTRIBUTES_3D attributes3D = FMODUnity.RuntimeUtils.To3DAttributes(transform);
FMODUnity.RuntimeManager.CoreSystem.set3DListenerAttributes(0, ref attributes3D.position,
ref attributes3D.velocity, ref attributes3D.forward, ref attributes3D.up);
}
}

Stereo master bus​

info

When a Studio master bank is loaded, the Studio master bus takes over the FMOD Core master channel group, so its input format decides how many channels the final mix has. If no events are routed to the master bus, FMOD Studio may build it as mono, which mixes left and right together and removes all panning (directional audio).

To fix this, open the Mixer in FMOD Studio, select the Master Bus, right-click the input meter in the deck, set Input Format to Stereo (or a surround format) and rebuild the banks. The sample's FMOD Studio project is already set up this way.

Limitations and pitfalls​