So far, All the reviews are super positive (thank you!). But they have another thing in common: many mention the game’s audio design. In this post, we will look at the game's procedural soundtrack: how it began and works.
Omnibullet's genre mix suggests the most important thing about it is its mechanics/gameplay. Everything else including the audio design was just an afterthought. However, the audio design has gradually become integral to the game, even winning the Audio Award at GDS 2023.
Before delving into the implementation details and music theory, it is important to explain how the idea for the procedural soundtrack even came to life.
Genre Pivot and Removing the Speed Tower
The game was not always supposed to be a Puzzle. Initially, we thought that it would be just an unconventional tower defense and the original mechanics reflected that. Once we realized that the game would be much more interesting if it was a logic game, we had to discard some of the incompatible mechanics, including one of the towers: Speed Tower.
Its only purpose was to accelerate passing bullets so they could travel further before hitting the ground. While this worked for the TD aspect of the game, it was somewhat odd for a logic game and we decided to cut it out. The expected result was that all bullets now had the same speed. The somewhat unexpected byproduct of that was that all interactions in the system suddenly became periodic, creating a sort of rhythm. We have decided to add a specific sound for each type of interaction and since the results were surprisingly good, we have decided to test this further.
Audio Synthesis
We did not want to play the exact same sound for each interaction and instead wanted it to be dependent on the parameters of the interaction (bullet energy, battery charge, tower type, tower state, …). However, it was unfeasible to create that many SFX assets as the number of possible combinations explodes with each parameter. We quickly realized that the only thing that would work would be an audio synthesis.
At first, we tried the Maestro Midi Player Toolkit Unity asset, but unfortunately, it just did not fit our needs. It was huge, had performance issues (latency unsuitable for real-time use), and generally was somewhat pasta. We needed a solution that would be reliable, fast, and could synthesize dozens of sounds in real-time. We have achieved this by heavily rewriting, modifying, and stripping the existing assets to suit our needs better. The final solution is based on FluidSynth and can be found here.
To play the synthesized audio in Unity, it is necessary to utilize one of the following methods:
OnAudioFilterRead
This is how Unity enables you to implement a custom audio filter. It gives you the audio data and lets you modify them. This is usually used to perform a custom operation on the data, but it is also possible to completely overwrite them with synthesized audio:
void OnAudioFilterRead(float[] data, int channels)
{
for(int j = 0; j < data.Length; j++){
data = 0; // generates complete silence, replace with custom audio data
}
}
The problem with this is that while it works, it has high latency and runs outside the main thread preventing it from using many other Unity functions.
AudioClip.Create
This is what we use in Omnibullet. Basically, you create your own audio clip programmatically:
float[] samples = GenerateSamples(Tone.C_3, Instrument.Harmonica, <other config>)
int sampleCount = samples.Length;
int sampleRate = 44100; // edit to your needs
int position = 0; // position in the samples
var audioClip = AudioClip.Create(
"GeneratedSound",
sampleCount,
1, // number of audio channels
sampleRate,
false, // disable streaming
outSamples => {
int len = Math.Min(outSamples.Length, sampleCount - position);
Array.Copy(samples, position, outSamples, 0, len);
position += len;
},
inPosition => {
position = Math.Min(Math.Max(inPosition, 0), sampleCount);
}
); // assign generated clip to some audio source
This has almost zero latency and the created clips can be easily cached for reuse.
Procedural Soundtrack
As was already mentioned, each interaction in the game produces a different sound (uses a different instrument from the soundfont, the default instruments can be changed in the settings).
The interactions are as follows:
- Bullet Tower shoots - this interaction produces “beats”
- The tower is activated by an incoming bullet
- Bullet hits battery
- Bullet pierces through battery
- Bullet hits an obstacle
- Bullet explodes in the air
- Uncharged battery arrives at the Output Gate
To ensure that the game will sound nice no matter what interactions occur, all produced sounds are harmonized using defined scales. To make the game sound less monotonic, the active scale changes from wave to wave:
- C Major - 1st wave
- A Major - 2nd wave
- F Major - 3rd wave
- D Minor - 4th wave
- E Major - 5th wave
If there are more than five waves, the scales repeat. Furthermore, A Minor scale is used during the time between waves, which creates much-needed intermezzos. It also makes the transitions between other scales smooth, because A Minor sounds kind of neutral.
Scale is defined as a sequence of seven tones. Each scale defines three important chords: Tonic, Dominant and Subdominant. The chords are used for picking tones that sound nice together.
For example, the C Major scale is defined as follows:
// C Major Scale definition (starting from tone C_1)
Scale C_MAJ_1 = new Scale(
new Tone[] { Tone.C_1, Tone.D_1, Tone.E_1, Tone.F_1, Tone.G_1, Tone.A_1, Tone.B_1 }
);
and its chords as:
public class Scale
{
public Tone[] Tones { get; private set; } // Tone is just an int
public Tone[][] Chords { get; private set; } public Scale(Tone[] tones)
{
Tones = tones;
Chords = new Tone[3][];
// Example for the C major scale
Chords[0] = new Tone[] { Tones[0], Tones[2], Tones[4], Tones[0] + 12, Tones[2] + 12 }; // Tonic = C major (C1, E1, G1, C2, E2)
Chords[1] = new Tone[] { Tones[0], Tones[3], Tones[5], Tones[0] + 12, Tones[3] + 12 }; // Subdominant = F major (C1, F1, A1, C2, F2)
Chords[2] = new Tone[] { Tones[1], Tones[4], Tones[6], Tones[1] + 12, Tones[4] + 12 }; // Dominant = G major (D1, G1, B1, D2, G2)
}
}
Notice that the chords are defined by five tones. It would be sufficient to define them using only the first three, but adding additional tones allows for more variability because even though both C1, E1, G1 and G1, C2, E2 are C major chords, they sound slightly different (still harmonic though).
With scales and chords defined, it is now possible to examine their usage in specific in-game interactions.
Tower Activation
Tower activations are the most pronounced components of the resulting soundtrack. Each tower type uses a different instrument and the exact tone produced depends on several parameters. The tower is activated whenever a bullet passes through.
Upon activation, the tower produces two sounds:
Ambient tower hum
Uses the same instrument for all towers (air choir by default). The tone at which it plays depends on the active scale and the energy of incoming bullets. Low-energy bullets produce a deeper hum, and high-energy bullets a higher hum. Together with some reverb, the combination of all hums produces a very nice factory-like background ambiance.
Tower-specific sound
This is the main melodic component of the soundtrack. Different towers use different instruments and by default, important towers use recognizable instruments and utility towers something very subtle/rhythmic. The generated tone depends on several parameters:
- position of the tower
- active scale and its defined chords
- the energy of the incoming bullet
- distance the bullet has traveled so far
- beat number
It is somewhat hard to explain in natural text the process by which these parameters are combined, so take the pseudocode instead:
Tone generateToneForTower(int x, int y, Scale activeScale, Bullet bullet, int beat)
{
// Should we use an alternate chord?
bool alt = (beat / 8) % 2 == 0; // Select a chord from the active scale
Tone[] chord = alt ? activeScale.Chords[0] : activeScale.Chords[(x + y) % 3]; // If between waves, do not alternate between chords and always use the tonic
if(activeScale == A_MINOR) chord = activeScale.Chords[0]; // Select a tone from the chord
Tone tone = chord[(x + y + bullet.distance) % tones.Length];
// For bullets with an energy above a threshold, transpose up the tone
// make it higher by one octave (octave has 12 tones)
if (bullet.energy > 6) tone += 12;
return tone;
}
Several points could use further explanation:
- alternating between chords every 8 beats creates an interesting dynamic and breaks the monotony
- the coordinates are used to ensure that two neighboring towers will use different chords
- the coordinates are also used to make the picked tone distributed evenly across the system, which ensures that in larger systems, all harmonic chords are present together, creating a full chord progression
- the chords are not alternating during the intermezzo to make the system sound calmer
Bullet hits battery
When the bullet hits the battery, the generated tone depends solely on the battery charge. The higher the battery charge, the higher the tone from the active scale is played. If the battery is charged gradually, it plays the entire scale (do, re, mi, fa, so, la, si) which both sounds nice and naturally provides feedback about the charge progression.
Bullet pierces through a battery
A bullet pierces through batteries whenever its energy surpasses a threshold. Such bullets are powerful and as such it should be rewarding for the player to use them. The audio design recognizes this by using a very rewarding bell sound whenever the bullet pierces. The generated tone depends on the bullet energy. The higher it is, the higher the tone from the active scale is selected.
Bullet hits a wall/explodes in the air
This is generally undesirable and so the audio feedback produces a specific - and by default not very nice - sound. Systems with problems therefore don't sound as good, which can alert the player and communicate that something is wrong.
Uncharged battery arrives at the Output Gate
Three uncharged batteries are the loose condition for the game and as such, this interaction should really alarm the player. Whenever this happens, a loud error sound is played and the entire generated soundtrack is redirected through an audio filter, which makes it sound distorted. This communicates that something went really wrong and that the current system is in trouble. The strength of distortion depends on the amount of uncharged batteries so far.
Conclusion
In Omnibullet’s development journey, sound stole the show. Initially an afterthought, the audio became the game’s star feature. The decision to prioritize audio design, despite its secondary status in game mechanics, proved instrumental in shaping Omnibullet’s identity. Through a lot of experimentation and many failures, we have managed to create ever-changing soundscapes that sync with the player’s actions. These sounds, from tower beats to bullet hits, made the game come alive.
Thanks for reading.