Refactoring Miku2Tower to use SDL3's Mixer

Estimated reading time 45 minutes

In my last game I experimented with SDL3 and used it for both images and audio. While working on it, I felt that the overall interface of the sound effects and music were a little on the clunky side. So, I wanted to break out a post specifically to poke around and try to get that into a state that I can feel better about.

What we made last time

Before we get started, it's kind of important for context that you all understand what we made last time. I'm not going to ask you to spend the four plus hours it takes to read through the creation of a tower defense game, but I do need to highlight an overall design philosophy and jot down a few notes for how the current implementation of sounds and music work in that game since that's our baseline.

The design philosophy I generally adhere to is one of compartmentalization. If I'm using a library that I don't own and isn't absolutely integral to the shape of what I'm building, I'll do my best to put up a defensive wall between myself and it. This is a "less is more" approach to the way the rest of the program interacts with it.

So, for example, if I was writing up a tool for myself that did something like wrapped ripgrep and just slapped a more intuitive interface for me to use, I'd absolutely let the ripgrep specific nuances leak through and mix with the codebase.

Counter to this, and relevant to today's post, is that if I wrap up something that provides visuals or sound for a game, I'll absolutely create my own interface for how I want the game code to interact with that library because the specifics of using raw OpenGL, SDL3, egor, bevy, raylib, or whatever are absolutely not relevant to how I want my game code to behave. This means that the game code stays "pure" and testable, while the program takes on a snowflake shape where the 3rd party tools that do things like "render it for linux" or "render it for the web" are pushed out to the boundaries to be melted first if the heat turns up and the code becomes bothersome and I want to swap.

Case in point, here's the audio interface I made last time for the game:

use std::time::Duration;

#[derive(PartialEq, Copy, Debug, Clone, Hash, Eq)]
pub struct SfxId(pub usize);

#[derive(PartialEq, Copy, Debug, Clone, Hash, Eq)]
pub struct MusicId(pub usize);


pub trait Audio {
    fn play_sfx(&mut self, id: SfxId);
    fn load_sfx(&mut self, id: SfxId);
    fn play_music(&mut self, id: MusicId);
    fn load_music(&mut self, id: MusicId);
    fn load_bg_music(&mut self) -> Vec<MusicId>;
    fn music_duration_seconds(&self, id: MusicId) -> Duration;
    fn prepare(&mut self);
}

One thing that you should notice here is that there are 0 dependencies here on anything beyond the standard library. This is 100% intentional as loading files is messy, slow, and has quite a few failure cases which are being ignored by this interface. Because this trait hides away the details of audio in SDL3, I can theoretically create a fake version of this that does nothing, or maybe tracks some internal state commands if I wanted to unit test game logic that should play certain sounds at certain times and what have you.

The trait here is simple, and the implementation of the audio is… less so. Here's the struct's definition that is implementing the trait and one of the functions:

struct SDL3Sounds {
    sound_by_id: HashMap<SfxId, SoundData>,
    music_by_id: HashMap<MusicId, SoundData>,
    buckets: Vec<Bucket>,
    poolsize: usize,
    base_path: PathBuf,
    current_track: MusicTrack,
    music_streams: [Option<SfxStream>; 2],
    context: Rc<RefCell<SDL3Context>>,
}

impl Audio for SDL3Sounds {
    fn play_music(&mut self, id: MusicId) {
        let Some(sound_data) = self.music_by_id.get(&id) else {
            return;
        };
        let (play_index, pause_index) = match self.current_track {
            MusicTrack::A => (0, 1),
            MusicTrack::B => (1, 0),
        };
        let now = Instant::now();
        self.music_streams[play_index] = Some({
            let ctx = &mut *self.context.borrow_mut();
            let device = ctx.audio.default_playback_device();
            let mut stream = SfxStream {
                stream: device
                    .open_device_stream(
                        Some(AudioSpec {
                            freq: Some(sound_data.spec.freq),
                            channels: Some(sound_data.spec.channels.into()),
                            format: Some(sound_data.spec.format),
                        })
                        .as_ref(),
                    )
                    .expect("could not open logical device for spec"),
                free_at: Some(now),
            };
            stream.claim(&sound_data, now);
            stream
        });

        match &mut self.music_streams[pause_index] {
            None => {}
            Some(SfxStream { stream, .. }) => {
                let _ = stream.pause();
            }
        }
        self.current_track = match self.current_track {
            MusicTrack::A => MusicTrack::B,
            MusicTrack::B => MusicTrack::A,
        };
    }
    ...

Before I explain more details, there's actually a bit more if you were to count the implementation of what stream.claim is doing, which is relying on an internal struct I made called SoundData

struct SoundData {
    spec: AudioSpecWAV,
    duration: Duration,
}
struct SfxStream {
    stream: AudioStreamOwner,
    free_at: Option<Instant>,
}

impl SfxStream {
    fn is_free(&self, now: Instant) -> bool {
        self.free_at.map_or(true, |t| now >= t)
    }

    fn claim(&mut self, entry: &SoundData, now: Instant) {
        let _ = self.stream.clear();
        let _ = self.stream.put_data(entry.spec.buffer());
        let _ = self.stream.resume();
        self.free_at = Some(now + entry.duration);
    }
}

As you can see. There's a lot of weedy details and terrifying expects floating around in the underlying code. But the game code that consumes the train never had to see that. Which is kind of the point and advantage of the compartmentalization I mentioned. Some people like to mix everything together and that's fine, but I find that splitting things out is better for my own mental clarity about the code, makes collaboration easier if more than one developer were to ever be tinkering at the same time, and long term, allows for easier building of Theseus's ship while paddling through an ocean of upgrades.

Putting aside the benefits of my approach and why I like it, the code above can be explained in pretty simple terms. To play music

  1. One must load the music first (different method call)
  2. We have two active streams to play music out of at any time, figure out which is free for us to use
  3. Begin playing music on the free track
  4. Pause the music that was playing before
  5. Mark the previous playing stream as free

And "play the music" really means transfer the audio data from the SoundData wrapper to the stream and make a note about when that stream should be done playing it.

It's not terribly difficult to describe, but boy does it gloss over some error details. The other thing that might not be readily apparent if this is your first time seeing the code here is that this code is only designed to work with wav files. I'm not 100% sure if we're going to change that in this post or not, but it's something that I'd like to do at some point. Mainly because wav files are massive compared to something like an mp3 and it hurts me a little bit every time I commit a wav file to a git repository.

I'm not sure how beneficial it would be to get into details about SDL3 specifics in this "last time on DBZ" type explanation, so let's hold off on that and I'll explain anything needed as we go. This is going to be a refactoring-type post for at least half of its runtime, so let's jump into it.

Adding errors

The first refactor I'd like to do will break compilation in one fell stroke. As alluded to in the last section, there's a lot of panicking code floating around underneath the hood. While I don't like exposing the details (at the type level) of a library to the game code, it is helpful if we can get a proper error code to help troubleshoot things when stuff does go wrong. However, this doesn't mean we leak error details out everywhere.

We can use the compiler to start to paint the picture for us, starting internally with the code in my SDL3 specific file, let's tweak one place, or well, add one character:

fn claim(&mut self, entry: &SoundData, now: Instant) {
    let _ = self.stream.clear()?;
    let _ = self.stream.put_data(entry.spec.buffer());
    let _ = self.stream.resume();
    self.free_at = Some(now + entry.duration);
}

Do you see it? No? Yes? Now that we've enjoyed a game of Where's Waldo? We can see that adding ? to any of the potentially failing lines results in rust trying to raise the error from the runtime to compilation time, and so we can see the types it's throwing at us:

error[E0277]: the `?` operator can only be used in a method that returns `Result` or `Option` (or another type that implements `FromResidual`)
   --> src/backend_sdl3.rs:143:36
    |
142 |     fn claim(&mut self, entry: &SoundData, now: Instant) {
    |     ---------------------------------------------------- this function should return `Result` or `Option` to accept `?`
143 |         let _ = self.stream.clear()?;
    |                                    ^ cannot use the `?` operator in a method that returns `()`
    |
help: consider adding return type
    |
142 ~     fn claim(&mut self, entry: &SoundData, now: Instant) -> Result<(), Box<dyn std::error::Error>> {
143 |         let _ = self.stream.clear()?;
...
146 |         self.free_at = Some(now + entry.duration);
147 ~         Ok(())
148 ~     }

Updating our method signature in the way it suggests results in the code compiling, but pushes a warning up to the places where I'm calling claim:

warning: unused `Result` that must be used
   --> src/backend_sdl3.rs:170:9
    |
170 |         stream.claim(&sound_data, now);
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: this `Result` may be an `Err` variant, which should be handled
    = note: `#[warn(unused_must_use)]` on by default
help: use `let _ = ...` to ignore the resulting value
    |
170 |         let _ = stream.claim(&sound_data, now);
    |         +++++++

If the linked list in your head you call a train of thought is on the same tracks as mine, you can probably note that this will continue up the call chain for a while and eventually land at the station known as our interface code. What's somewhat interesting to me is that changing the call site within the play_music doesn't populate the same sort of suggestion from the compiler as before:

error[E0277]: the `?` operator can only be used in a method that returns `Result` or `Option` (or another type that implements `FromResidual`)
   --> src/backend_sdl3.rs:211:43
    |
186 |     fn play_music(&mut self, id: MusicId) {
    |     ------------------------------------- this function should return `Result` or `Option` to accept `?`
...
211 |             stream.claim(&sound_data, now)?;
    |                                           ^ cannot use the `?` operator in a method that returns `()`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `mikumikutower` (lib) due to 1 previous error

I mean, yes, it is still E0277 just like before, but the suggestion doesn't include changing the return type to be Ok(()). I assume this is because this is a method that is being implemented as part of a trait and so the rust compiler is smart enough to know that one might not be able to do that because maybe the trait comes from a 3rd party or something. Of course, we are smart enough to know that we own all chips on the table here. So we can tweak this and make it work by modifying the interface:

pub trait Audio {
    fn play_music(&mut self, id: MusicId) -> Result<(), Box<dyn Error>>;
    ...
}

Then the error changes tune and tells us what to do like before. Though, there is one part of the code where we need to make an actual decision before we address the new set of let _ = suggestions:

fn play_music(&mut self, id: MusicId) -> Result<(), Box<dyn std::error::Error>> {
    let Some(sound_data) = self.music_by_id.get(&id) else {
        // TODO: Fix
        return Ok(());
    };

At the moment, when our play_music method attempts to play something that isn't loaded, it just silently returns. This was a result of laziness on my part in the old code, but since we've now swapped to a result type, we can make decisions like "music not playing isn't game breaking, but we should probably let someone know" in a better way. Specifically, we can return an error here and that would be better than pretending everything is hunky dory with an Ok.

let err = format!("Could not play music with id {}, music not loaded.", id.0);
return Err(Box::<dyn std::error::Error>::from(err));

Now, technically speaking, I could define an enum and try to enumerate every potential type of error that might happen here. That would allow me, at the type level, to easily choose if I should recover from something or not. And I do want to do that, but refactoring is a baby step process. Pausing the world to start enumerating everything would result in me reading a bunch of different things in the SDL wiki, which means going on the internet, which means getting distracted. Rather than do that, I'd rather stick with plain strings in our first pass and then circle back once we've added all the results we needed. 1

The code compiles, and I'm feeling particularly grateful that the SDL3 rust binding library I'm using was idiomatic in their error handling and returned a standard library flavored Error as their result type. This lets us focus on all the new warnings I have now saying things like

warning: unused `Result` that must be used
   --> src/scene/title_screen.rs:105:21
    |
105 |                     audio.play_music(MUSIC_ID_PACHEBAL);
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: this `Result` may be an `Err` variant, which should be handled
help: use `let _ = ...` to ignore the resulting value
    |
105 |                     let _ = audio.play_music(MUSIC_ID_PACHEBAL);
    |                     +++++++

Not all these places need to follow suit and use ? to bubble things up. Eventually, I do want to probably do that, and culminate that into a sort of GameError type thing where the core game core can handle things. But for now, adding in a comment and a let _ = at all 6 places the compiler is telling me to is the best choice for me, as that's one trait method down, and… six more to go.

The easiest one to tweak is probably going to be music_duration_seconds considering it's super small and can only fail in one way:

fn music_duration_seconds(&self, id: MusicId) -> Duration {
    let Some(sound_data) = self.music_by_id.get(&id) else {
        return Duration::from_secs_f64(0.0);
    };
    spec_duration(&sound_data.spec)
}

But at the same time, is it actually correct to change this? If a particular music track doesn't exist, or isn't loaded. Technically speaking, it does have a duration of 0 seconds. But, at the same time, it's a bit more flexible on the caller end to know that something doesn't exist versus if something is actually 0 seconds long. So, let's use a result here as well. That said, I find the Box and dyn dance tiring to write 2 so let's declare a less verbose type to use:

type AudioResult<T> = Result<T, Box<dyn Error>>;

pub trait Audio {
    fn play_music(&mut self, id: MusicId) -> AudioResult<()>;
    fn music_duration_seconds(&self, id: MusicId) -> AudioResult<Duration>;
    ...
}

This doesn't actually change anything about play_music of course. It's just an alias. It does break compilation though!

$ cargo check
error[E0599]: no method named `as_secs` found for enum `Result` in the current scope
... elided ...
error[E0599]: no method named `as_secs` found for enum `Result` in the current scope
... elided ...
note: the method `as_secs` exists on the type `Duration`
   --> /rustc/1159e78c4747b02ef996e55082b704c09b970588/library/core/src/time.rs:471:5
help: consider using `Result::expect` to unwrap the `Duration` value, panicking if the value is a `Result::Err`
    |
760 |    let duration = audio.music_duration_seconds(music_id).expect("REASON").as_secs();
    |                                                         +++++++++++++++++

It just pushes things up and rust is like: "Hey why don't you unwrap the value here? It's got what you want!" Whether or not that's a good idea is sort of reliant on the context though. For the first E0599, we're in the code that handles advancing along the background music for the level. Specifically, the initial match state where the known music files exist, but none have started quite yet:

// Initial load, just start playing.
let music_id = self.bg_music_ids[0];
self.time_to_play_next_song = game_context.audio.as_mut().map(|audio| {
    // do this in the map so that if we cant load the audio we dont set the now playing
    self.now_playing = Some(0);
    let duration = audio.music_duration_seconds(music_id).as_secs();
    let _ = audio.play_music(music_id);
    ReadyState::Cooldown {
        wait_for: duration as u32 * 60, // roughly 60 ticks per second.
        ticks_waited: 0,
    }
});

The way the surrounding code works is pretty simple, there's a ReadyState method that ticks down every game tick (which the comments noted a conversion of) and we use that to swap audio tracks as songs finish. We time things so that we use the duration of the song to determine when we should swap in the next song, and that's the broken part here. If we set it to 0, then the next game tick will try to swap to the next song immediately.

Potentially, this would cause the system to then basically do the same code, just advancing the music id along that list of bg_music_ids vector forever if everything was randomly not loaded for some reason. That might sound bad, but I don't think it actually is. I mean, sure less work is better, but we always check a ready state anyway to see if it's time to move along, so we're not really gaining or losing anything by just saying that the duration is 0. But, hey, if something isn't loaded, it probably means something is slightly borked up or someone deleted some game files between the scene load somehow. So, let's toss in a meaningful back off and just not call play_music since that would fail anyway: no music id to compute seconds? no music id to play a song.

let backoff = ReadyState::Cooldown {
    wait_for: 60, // wait a sec
    ticks_waited: 0,
};
if let Ok(duration) = audio.music_duration_seconds(music_id) {
    let duration = duration.as_secs();
    let _ = audio.play_music(music_id);
    ReadyState::Cooldown {
        wait_for: duration as u32 * 60, // roughly 60 ticks per second.
        ticks_waited: 0,
    }
} else {
    backoff
}

That's the music_duration_seconds handled, we can probably apply the same logic to if the music couldn't be played as well, so rather than let _ = we can return backoff if there's a problem there too:

let music_id = self.bg_music_ids[0];
self.time_to_play_next_song = game_context.audio.as_mut().map(|audio| {
    // do this in the map so that if we cant load the audio we dont set the now playing
    self.now_playing = Some(0);
    if let Ok(duration) = audio.music_duration_seconds(music_id) {
        let duration = duration.as_secs();
        if let Ok(_) = audio.play_music(music_id) {
            ReadyState::Cooldown {
                wait_for: duration as u32 * 60, // roughly 60 ticks per second.
                ticks_waited: 0,
            }
        } else {
            backoff
        }
    } else {
        backoff
    }
});

Staring at that "triangle of doom" forming makes my pine for Scala's for comprehensions but I suppose rust has the ? guy to sort of do that. I just can't really use it when I want to actually handle things like this. I suppose I could do a let pattern match and an else…

let music_id = self.bg_music_ids[0];
self.time_to_play_next_song = game_context.audio.as_mut().map(|audio| {
    // do this in the map so that if we cant load the audio we dont set the now playing
    self.now_playing = Some(0);

    let Ok(duration) = audio.music_duration_seconds(music_id) else {
        return backoff;
    };
    let duration = duration.as_secs();
    let Ok(_) = audio.play_music(music_id) else {
        return backoff;
    };
    ReadyState::Cooldown {
        wait_for: duration as u32 * 60, // roughly 60 ticks per second.
        ticks_waited: 0,
    }
});

Yeah! That's nicer. Easier to read I think and pleasantly explicit. While I said there was another place to fix, it's actually the other half of this branch which basically does the exact same thing, just without setting the music id with a hardcoded [0]. And that's another function moved over to returning a result! Technically, this one could have maybe returned an Option, but I like the idea that if you had some kind of streaming going on, where you couldn't know the duration, that being able to return a non OK message to that extent could be nice.

Alright, what's next? Since we've updated two of the four music related methods, let's wrap up with the other two. They're closely related and dragging one out to get Results is going to force the other, just like that one Javascript meme:

We can start with load_music, which helps as a helper for the load_bg_music method that the level struct in the tower defense game uses to load arbitrary wav files from a folder. load_music is used for the predestined music files played on the title, game over, and shutting down screen. Funnily enough, you can see that while I was working on it I was already thinking about the refactoring that we're doing now:

fn load_music(&mut self, id: MusicId) {
    if !self.music_by_id.get(&id).is_none() {
        return;
    }
    // FUTURE ENHANCEMENT I suppose make things turn results and ? it all.
    let path = self.base_path.join(music_id_to_relative_path(id));
    let spec = AudioSpecWAV::load_wav(path).expect("could not load spec from path");
    let data = SoundData {
        duration: spec_duration(&spec),
        spec,
    };
    self.music_by_id.insert(id, data);
}

So, the easy first thing to do is to toss a ? at the expect and break things. It breaks in the expected place, so I won't bore you with the explanation that would be the same as the previous one, but the updated code is now:

fn load_music(&mut self, id: MusicId) -> AudioResult<()> {
    if !self.music_by_id.get(&id).is_none() {
        return Ok(()); // idempotent
    }
    let path = self.base_path.join(music_id_to_relative_path(id));
    let spec = AudioSpecWAV::load_wav(path)?;
    let data = SoundData {
        duration: spec_duration(&spec),
        spec,
    };
    self.music_by_id.insert(id, data);
    Ok(())
}

There are also now five places where the compiler is asking me to handle the new result. Though, only one of them is something I'm planning on fixing as part of this blog post:

warning: unused `Result` that must be used
   --> src/backend_sdl3.rs:261:21
    |
261 |                     self.load_music(music_id);
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: this `Result` may be an `Err` variant, which should be handled
    = note: `#[warn(unused_must_use)]` on by default
help: use `let _ = ...` to ignore the resulting value
    |
261 |                     let _ = self.load_music(music_id);
    |                     +++++++

This suggestion raises a very important question. The method is the aforementioned load_bg_music and that currently looks like this:

fn load_bg_music(&mut self) -> Vec<MusicId> {
    let user_wav_folder = self.base_path.join("audio").join("cc-vocaloid");
    let mut ids = Vec::new();
    if let Ok(results) =
        glob_directory(user_wav_folder, Some("*.wav"), GlobFlags::CASEINSENSITIVE)
    {
        for path in &results {
            let filename = path.file_name();
            if filename.is_none() {
                continue;
            }
            let filename = filename.unwrap().to_str().unwrap();
            let desired_id = filename[0..filename.len() - 4].parse::<usize>();
            if desired_id.is_ok() {
                let music_id = MusicId(desired_id.unwrap());
                ids.push(music_id);
                self.load_music(music_id);
            }
        }
    }
    ids
}

The main thing I'm thinking about here is the return type. At the moment, it returns a vector of music ids. These ids are then used by the level scene to cycle through background music as the gamer plays the game. The question to ponder is how to handle partial failure.

This is really a question of API design and the ergonomics on the other side. On the one hand, if you swallow the errors and just return anything that could be loaded successfully, then the caller gets a list of ids and away they go. If you don't drink and instead vomit up the error the moment it happens, you end up with no music if even one thing breaks. If you go to the bathroom, release the foul contents of the ill-tasting beverage into the toilet, then return to force your friend to buy you a shot of something else, then…

Ahem. Sorry, this is a gross metaphor. The actual things we're considering is:

  • Return Result<Vec> and ignore any errors that happen
  • Return Result<Vec> and let one error fail everything
  • Return Vec<Result> and force the caller to deal with any errors

You might say to yourself: "well just make the caller deal with it", and I'd normally be swift to agree in the name of flexibility and error reporting, but it's that return type of ours that's giving me pause on this. In the other methods, the caller always knows the MusicId that they want, so they've got context around it. In this case though, the user doesn't, we're assigning new MusicIds based on the filename (which the README for this folder instructs the user to do to control the ordering of the playback).

Ah, I suppose there is that context. If the application took these individual warnings, perhaps they could still figure things out from there. The last thing I want is to report some error and it's too opaque for a user to figure out which track is the problem. Then the user spends hours removing and adding in files slowly to test which is the problem and yeah. Not fun. So long as the errors returned can somewhat meaningfully point the direction for the person between the keyboard and the screen, a list of errors will probably be more useful than a single one.

fn load_bg_music(&mut self) -> Vec<AudioResult<MusicId>> {
    let user_wav_folder = self.base_path.join("audio").join("cc-vocaloid");
    let mut ids = Vec::new();
    if let Ok(globbed) =
        glob_directory(user_wav_folder, Some("*.wav"), GlobFlags::CASEINSENSITIVE)
    {
        for path in &globbed {
            let filename = path.file_name();
            if filename.is_none() {
                continue;
            }
            let filename = filename.unwrap().to_str().unwrap();
            let desired_id = filename[0..filename.len() - 4].parse::<usize>();
            if desired_id.is_ok() {
                let music_id = MusicId(desired_id.unwrap());
                ids.push(self.load_music(music_id).and_then(|_| Ok(music_id)));
            } else {
                let msg = format!("cannot load music file {} please name it numerically in the order you want played", filename);
                let e = Box::<dyn Error>::from(msg);
                ids.push(Err(e));
            }
        }
    }
    ids
}

This breaks the call site since it was originally processing a list of ids only, so that has to be tweaked. Since I don't really have any error reporting in this game, we'll just swallow them there:

impl Scene for LevelScene {
    fn init(&mut self, game_context: &mut GameContext) {
        ...

        // Background music is loaded semi-dynamically from the assets folder.
        // so keep track of the ids we care about by reading the file names which
        // should be numeric:
        self.bg_music_ids = audio
            .load_bg_music()
            .into_iter()
            .filter(|r| r.is_ok())
            .map(|r| r.unwrap())
            .collect();
    }

You might look exasperated right now. Why did this weirdo programmer make a big deal about choosing the flexible return type that makes it the callers decision if he's just going to swallow them anyway?

I'm going to swallow them, for now. My goal with this post isn't to refactor the entire game, but rather refactor just the music related parts of it. So, I'm considering updating the overall game loop to get mad and spit out errors in a way that's visible to the users out of scope. Though, I suppose I could maybe eprintln! the errors before I filter them out I guess.

self.bg_music_ids = audio
    .load_bg_music()
    .into_iter()
    .filter(|r| {
        if let Err(error) = r {
            eprintln!("{}", error);
        }
        r.is_ok()
    })
    .map(|r| r.unwrap())
    .collect();

Simple enough. Now at least if someone is troubleshooting, they'll be able to see the error in stderr where they'd expect it to be. The remaining methods in the Audio interface are:

pub trait Audio {
    ...
    fn play_sfx(&mut self, id: SfxId)
    fn load_sfx(&mut self, id: SfxId)
    fn prepare(&mut self);
}

Both play_sfx and load_sfx are basically the same song and dance we did with its musical counterparts. The only thing worth noting about implementing the signature update is that within the init method of the level code, I figured that we could spit out the problem with any loaded sfx on scene load rather than just completely ignoring the problem:

let _ = audio
    .load_sfx(SFX_ID_DESELECT)
    .map_err(|e| eprintln!("{}", e));
let _ = audio
    .load_sfx(SFX_ID_BASE_HIT)
    .map_err(|e| eprintln!("{}", e));
let _ = audio
    .load_sfx(SFX_ID_ENEMY_HIT)
    .map_err(|e| eprintln!("{}", e));
... you get the point...

The prepare method is an interesting one though. I'm once again waffling about if it should be returning an error or silently swallowing any problems. This is a longer method, but one which only has one place that will actually panic if something bad happens that we need to consider for this situation. The method's purpose is to diff the existing sounds and the ones which have been requested for load by a scene. Then, it updates the internals of the struct's "buckets" to ensure that there are a pool of audio streams available to play those sounds on:

fn prepare(&mut self) {
    let mut specs_to_prepare: HashSet<Spec> = self
        .sound_by_id
        .values()
        .map(|v| to_hashable_spec(&v.spec))
        .collect();
    for spec in self.music_by_id.values().map(|v| to_hashable_spec(&v.spec)) {
        specs_to_prepare.insert(spec);
    }

    // Clean out anything we DONT need anymore:
    let mut already_exist = HashSet::new();
    self.buckets.retain_mut(|bucket| {
        let exists = specs_to_prepare.contains(&bucket.spec);
        if exists {
            already_exist.insert(bucket.spec.clone());
        }
        for SfxStream { stream, .. } in &bucket.streams {
            let _ = stream.pause();
        }
        exists
    });

    let ctx = &mut *self.context.borrow_mut();
    for spec_needs_bucket in specs_to_prepare.difference(&already_exist) {
        let mut streams = Vec::with_capacity(self.poolsize);
        for _ in 0..self.poolsize {
            let device = ctx.audio.default_playback_device();
            let stream = SfxStream {
                stream: device
                    .open_device_stream(
                        Some(AudioSpec {
                            freq: Some(spec_needs_bucket.freq),
                            channels: Some(spec_needs_bucket.channels),
                            format: Some(spec_needs_bucket.format),
                        })
                        .as_ref(),
                    )
                    .expect("could not open logical device for spec"),
                free_at: None,
            };
            streams.push(stream);
        }
        self.buckets.push(Bucket {
            spec: *spec_needs_bucket,
            streams,
        })
    }
}

As you can see, the area we expect, is where we open a new stream for any new type of audio input we need to mix. In SDL3, the audio layer handles a lot of the work for you, where it will take input data of whatever format, and then tweak and mix it for you so that it matches whatever the underlying playback device's specs are. But each device stream does have to be one particular type to do so. And, so if you have sound effects of varying formats, you need more than one bucket, and thus the prepare method, prepares the bucket of available streams so that when we eventually call play_sfx, there's a stream in the correct format ready to go:

 // within play_sfx    	        	
let bucket_key = to_hashable_spec(&sound_data.spec);
let Some(bucket) = self.buckets.iter_mut().find(|b| b.spec == bucket_key) else {
    let err = format!(
        "no bucket found for spec {:?}, ensure you called prepare after load_sfx",
        bucket_key
    );
    return Err(Box::<dyn Error>::from(err));
};

So, given that if there's no bucket prepared we already have the play method raise an error, do we need to raise an error during preparation too? I'm swallowing the error from the call to play_sfx, and even if I were to go refactor a bunch of game code, I'd probably still be suppressing it because a single sound effect not playing is not game breaking for most games. 3

I feel like I should still raise an error, because if we start spamming stderr with notes about sound effects not playing if we don't suppress those errors from not having a bucket ready, then it'd be nice to know what the root cause of said bucket failing was. The trouble is, as you can see, it's in a loop. So we're back to the Vec<Result> type situation it feels. But this time, there's not really anything of note in the "Ok" side.

Writing a docstring for this is easy for the error case, "returns a list of errors if any underlying stream cannot be prepared to play sound effects" or something like that. But, a list of units I guess would be the other half? I don't know, it just feels weird. It almost feels like it ought to return a list of failures and then a caller could check that that was empty. But maybe that's weird too. I think the problem I have with it is that the "prepare" step feels like a leak in the abstraction black box.

It's not like calling it "initialize" would be better either. That would imply this only happens once. But prepare has to be called after you've requested new sound effects have been loaded and before you play them. So it's very much a prepare for a scene kind of thing that has mixed in to the audio playing interface. I suppose one could push it out of the public interface if one did the prep during the load, but that sounds incredibly wasteful of the computers time, and so I'd really like to have the general feel of things be like::

"I would like this"

"Ok"

"Are you good to go?"

"Yup"

"Play the thing please"

"Ok"

Hm. Prepare is the middle step there, so returning a Result feels fine. It's just the contents of the result that are causing my agony. Let's just stop overthinking it and return a list of audio results.

let stream = device
        .open_device_stream(
            Some(AudioSpec {
                freq: Some(spec_needs_bucket.freq),
                channels: Some(spec_needs_bucket.channels),
                format: Some(spec_needs_bucket.format),
            })
            .as_ref(),
        );
if let Ok(stream) = stream {
    let stream = SfxStream {
        stream,
        free_at: None,
    };
    streams.push(stream);
} else {
    // "could not open logical device for spec"
    stream_failures.push(stream);
}
error[E0308]: mismatched types
   --> src/backend_sdl3.rs:340:9
    |
286 |     fn prepare(&mut self) -> Vec<AudioResult<()>> {
    |                              -------------------- expected `Vec<Result<(), Box<(dyn std::error::Error + 'static)>>>` because of return type
...
332 |                     stream_failures.push(stream);
    |                     ---------------      ------ this argument has type `Result<AudioStreamOwner, sdl3::Error>`...
    |                     |
    |                     ... which causes `stream_failures` to have type `Vec<Result<AudioStreamOwner, sdl3::Error>>`
...
340 |         stream_failures
    |         ^^^^^^^^^^^^^^^ expected `Vec<Result<(), Box<dyn Error>>>`, found `Vec<Result<AudioStreamOwner, Error>>`
    |
    = note: expected struct `Vec<Result<(), Box<(dyn std::error::Error + 'static)>>>`
               found struct `Vec<Result<AudioStreamOwner, sdl3::Error>>`

Well that certainly throws a bit of a wrench in the plan. The left side is easy enough, just map away the left side:

stream_failures.push(stream.map(|_| ()));

But that sdl3::Error is still hanging out... on a whim, I tried hitting it with an explicit into

stream_failures.push(stream.map(|_| ()).map_err(|e| e.into()));

And shockingly enough, that worked and the program compiled. I wonder why it needed the explicit into… Well. It compiles, it will report its results for any interested parties, and we can move right along. Everything in the Audio trait's surface now returns a potential error so that we can handle things more explicitly and without panicking because someone did something funny with audio.

I want to play mp3s with SDL3

The other thing that that was sort of grinding my gears when it came to the audio related work on the last game is that I could only use wav files. Granted, that was partially due to my own laziness, and also due to the fact that when I was looking at the feature flags for the library, I saw this:

Which, didn't really fill me with too much hope. Sure, I could probably find a different library to figure out how to open up an audio file and decompress or detect the appropriate values to fill in an AudioSpec and all that. But, the mixer library sounded really nice, i just couldn't use it with a high level rusting binding like I wanted to. Well, good news everyone, I was checking the latest update to the rust package the other day for SDL3 and lo and behold:

So it looks like if upload my SDL3 library from 0.17.3 to 0.18.4, then I can use the mixer! My toml file looks like this:

[dependencies]
sdl3 = { version = "0", features = ["image", "build-from-source"] }

Since I'm moving between minor versions here, I should be able to just run the update command:

$ cargo update
Updating crates.io index
 Locking 12 packages to latest Rust 1.90.0 compatible versions
Updating bitflags v2.11.0 -> v2.12.1
Updating cc v1.2.56 -> v1.2.63
Updating cmake v0.1.57 -> v0.1.58
Removing lazy_static v1.5.0
Updating libc v0.2.183 -> v0.2.186
Updating sdl3 v0.17.3 -> v0.18.4
Updating sdl3-image-src v3.4.0 -> v3.4.4
Updating sdl3-image-sys v0.6.1+SDL-image-3.4.0 -> v0.6.3+SDL-image-3.4.4
  Adding sdl3-mixer-src v3.2.2
  Adding sdl3-mixer-sys v0.6.2+SDL-mixer-3.2.2
Updating sdl3-src v3.4.2 -> v3.4.8
Updating sdl3-sys v0.6.1+SDL-3.4.2 -> v0.6.5+SDL-3.4.8
Updating shlex v1.3.0 -> v2.0.1

and then regenerating my local documentation so I don't get lost online thinking that something exists when it actually doesn't:

$ cargo doc
  Downloaded sdl3-image-sys v0.6.3+SDL-image-3.4.4
  Downloaded bitflags v2.12.1
  Downloaded sdl3-sys v0.6.5+SDL-3.4.8
  Downloaded sdl3 v0.18.4
  Downloaded sdl3-mixer-sys v0.6.2+SDL-mixer-3.2.2
  Downloaded sdl3-src v3.4.8
  Downloaded sdl3-image-src v3.4.4
  Downloaded sdl3-mixer-src v3.2.2
  Downloaded 8 crates (23.1MiB) in 1.00s (largest was `sdl3-mixer-src` at 8.5MiB)
   Compiling shlex v2.0.1
   Compiling sdl3-src v3.4.8
   Compiling sdl3-image-src v3.4.4
   Compiling libc v0.2.186
    Checking bitflags v2.12.1
 Documenting bitflags v2.12.1
   Compiling cc v1.2.63
 Documenting libc v0.2.186
   Compiling cmake v0.1.58
   Compiling sdl3-sys v0.6.5+SDL-3.4.8
   Compiling sdl3-image-sys v0.6.3+SDL-image-3.4.4
   Compiling sdl3 v0.18.4
 Documenting sdl3-sys v0.6.5+SDL-3.4.8
 Documenting sdl3-image-sys v0.6.3+SDL-image-3.4.4
 Documenting sdl3 v0.18.4
 Documenting mikumikutower v0.1.0 (/home/peetseater/src/personal/musicplayer)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 14.34s
   Generated target/doc/mikumikutower/index.html

and now my local docs match what I saw online! But, does the game still compile?

Lovely. Ok, so, if I tweak the features to now include the "mixer" then I should get access to that. Looking at the docs, this actually looks like it should be easy to use. The Mixer's entry point is this function in the docs

pub fn open_device(spec: Option<&SDL_AudioSpec>) -> Result<Mixer, Error>

    Create a mixer that plays to the default audio device.

    Pass None for spec to let SDL choose the best format.

I assume that because it's an extension, and not part of the core SDL3 crate, that that's why there's nothing like sdl.mixer() type thing, and instead we do:

let mixer_subsystem = Mixer::open_device(None).expect("failed to create mixer context");

The dreaded expect returns. But I'm currently inside of the create_event_loop of the Backend trait in my game. Which, if we start refactoring now, will probably lead us to refactor a lot more than I scoped for this session. Still, this is simple enough to get the mixer and then I can track it in this backend's SDL context like all the other stuff

let e = EventLoopSDL3 {
    event_pump,
    context: Rc::new(RefCell::new(SDL3Context {
        _video: video_subsystem,
        window_canvas: canvas,
        textures,
        audio: audio_subsystem,
        mixer: mixer_subsystem,
    })),
};

And once it's in the context, it means we can access it during the sound related methods we refactored. Interestingly enough, we're already in decent shape. The audio method from the mixer just wants a path and then will give you back Audio or an error, as noted in the docs:

pub fn load_audio<P: AsRef<Path>>(
    &self,
    path: P,
    predecode: bool,
) -> Result<Audio, Error>

	Load audio from a file path.

	If predecode is true, the audio will be fully decompressed into memory. Otherwise it will be decoded on the fly during playback.

We already have a path when we load audio, so that all lines up. We can just hardcode the predecode, I'll probably set it to true I think. Playing the audio for the mixer just needs the audio handle:

pub fn play_audio(&self, audio: &Audio) -> Result<(), Error>

    Play audio once from start to finish without any management (“fire and forget”).

    Internally, SDL_mixer creates a temporary track, plays it, and cleans up when done.

So honestly, I think we just need to tweak the hash map inside of the SDL3Sounds struct. Right now it tracks the music by id and then the value is our custom struct of sound data:

struct SDL3Sounds {
    music_by_id: HashMap<MusicId, SoundData>,
    buckets: Vec<Bucket>,
    poolsize: usize,
    base_path: PathBuf,
    current_track: MusicTrack,
    music_streams: [Option<SfxStream>; 2],
    context: Rc<RefCell<SDL3Context>>,
}

I think we swap the SoundData out for the Audio, then potentially that enables us to use the mixer functions pretty easily. Though one small trick we need to employ is to rename the SDL3 import because it conflicts with the name of our interface:

use crate::audio::{Audio, AudioResult};
use sdl3::mixer::Audio as MixerAudio;

Swapping out the hashmap value breaks the compiler and guides us to where we need to be next. So,

music_by_id: HashMap<MusicId, MixerAudio>

gives us lot of "unknown field" errors for where we try to use the result of the hashmap like

error[E0609]: no field `spec` on type `&sdl3::mixer::Audio`
   --> src/backend_sdl3.rs:210:51
    |
210 | ...                   freq: Some(sound_data.spec.freq),
    |                                             ^^^^ unknown field

and in a tantalizing way, we also get an error around the claiming of the streams that we were doing

error[E0308]: mismatched types
   --> src/backend_sdl3.rs:219:26
    |
219 |             stream.claim(&sound_data, now)?;
    |                    ----- ^^^^^^^^^^^ expected `&SoundData`, found `&&Audio`
    |                    |
    |                    arguments to this method are incorrect
    |
    = note: expected reference `&SoundData`
               found reference `&&sdl3::mixer::Audio`

The unknown fields are easy, the easiest one is in the prepare method where we were previously combining the results of both hashmaps so that we could make buckets for each. Since the mixer can handle things internally on its own, we don't need the buckets anymore, so we just delete:

-for spec in self.music_by_id.values().map(|v| to_hashable_spec(&v.spec)) {
-    specs_to_prepare.insert(spec);
-}

and there goes one error. We can also avoid needing to call .spec in the music_duration_seconds because the Audio struct from mixer module has a duration method which returns frames, and a frames to milliseconds method. So, easy peasy, one two three and:

-Ok(spec_duration(&sound_data.spec))
+let duration = mixer_audio.frames_to_ms(mixer_audio.duration());
+Ok(Duration::from_millis(duration.try_into()?))

The duration return type is an i64 and the from_millis expects a u64. Thankfully, we updated all the code to return result types! So that's another place we thank ourselves for the refactor from before. It's fun when tidying up clears the way for better code!

Now, the real thing to deal with of course is the play_music method. If you remember, we were keeping track of the tracks, creating a new audio stream and swapping back and forth between them. The mixer creates temporary tracks under the hood, and while there is a Track type, I regret to inform you and myself that the track type has lifetimes. Which uh, makes it hard to use. That said, the mixer handles making tracks internally as well with its convenience method, so the play music code becomes really simple as long as we don't mind a new track each play, which we were doing before anyway, so:

fn play_music(&mut self, id: MusicId) -> Result<(), Box<dyn std::error::Error>> {
    let Some(mixer_audio) = self.music_by_id.get(&id) else {
        let err = format!("Could not play music with id {}, music not loaded.", id.0);
        return Err(Box::<dyn Error>::from(err));
    };

    let ctx = &mut *self.context.borrow_mut();
    ctx.mixer.pause_all()?;
    ctx.mixer.play_audio(&mixer_audio)?;
    Ok(())
}

It's nice when things get simple, because it also means that we can make things more complicated again without feeling like we're in over our head. Specifically, if we were to figure out how to use the tracks and their sticky lifetimes, then we could actually use the mixer to play music and sound effects. The interesting thing about the mixer interface is that it can play "tags". Tags being arbitrary text you can assign to a given track. So, kind of like our buckets, we could have a set of tracks for playing certain sounds and then trigger them easily enough… specifically there's a stop_tag and pause_tag that can be used to stop many things all at once, so the use of mixer.pause_all we're using right now to stop the music could be more targetted to not silence any sounds when the track swaps for the background music.

Maybe most importantly, if we used the mixer for the sound effects, then we could load up mp3s and other formats for all audio in the program. Technically, right now, if I change the globs then I can load mp3s for the background files and that would all work just fine. I should probably tweak the load_bg_music method to list all the files, then filter them by the available formats we can play. But uh…

How do we do that?

If I look at this demo I can see that there is a way to list of available decoders. And it does work:

let n = sdl3::mixer::get_num_audio_decoders();
println!("available audio decoders: {n}");
for i in 0..n {
    if let Some(name) = sdl3::mixer::get_audio_decoder(i) {
        println!("  decoder {i} => {name}");
    }
}

But the incredibly bothersome thing to me is how hard it is to find this dang function in the local documentation.

Searching for one of them finds what appears to be the SDL3 guy from the wiki and it's not until I slapped the search up on github that I found it

/// Get the number of audio decoders available.
#[doc(alias = "MIX_GetNumAudioDecoders")]
pub fn get_num_audio_decoders() -> i32 {
    unsafe { sys::MIX_GetNumAudioDecoders() }
}

I'm sure there's a good reason for why they do the doc alias thing here. But I also can't help but be somewhat bothered that it 1. hides the unsafe aspect of things, and 2. makes it harder to understand what the supposedly more "ergonomic" interface of the rust bindings are. Extra bothersome to me is that I click on the "source" button in the local docs I don't see the mod file, but rather this random mixer extern work:

unsafe extern "C" {
    /// Report the number of audio decoders available for use.
    ///
    /// An audio decoder is what turns specific audio file formats into usable PCM
    /// data. For example, there might be an MP3 decoder, or a WAV decoder, etc.
    /// SDL_mixer probably has several decoders built in.
    ///
    /// The return value can be used to call [`MIX_GetAudioDecoder()`] in a loop.
    ///
    /// The number of decoders available is decided during [`MIX_Init()`] and does not
    /// change until the library is deinitialized.
    ///
    /// ## Return value
    /// Returns the number of decoders available.
    ///
    /// ## Thread safety
    /// It is safe to call this function from any thread.
    ///
    /// ## Availability
    /// This function is available since SDL_mixer 3.0.0.
    ///
    /// ## See also
    /// - [`MIX_GetAudioDecoder`]
    pub fn MIX_GetNumAudioDecoders() -> ::core::ffi::c_int;
}

Which is just the extern keyword and a function declaration for the ffi work. It's all quite confusing to a rust novice like me. At least there are plenty of documentation strings to try to elevate my attention about that of a water fleas though.

Anyway, putting aside the strange and questionable aspects of why I don't have to call this init method that the example has, let's update the globbing to care about the types of file that the mixer can handle. The code I pasted above has this output:

available audio decoders: 9
  decoder 0 => WAV
  decoder 1 => STBVORBIS
  decoder 2 => DRFLAC
  decoder 3 => VOC
  decoder 4 => AIFF
  decoder 5 => AU
  decoder 6 => DRMP3
  decoder 7 => SINEWAVE
  decoder 8 => RAW

The documentation also has some hints that would have told me I'd be disappointed and unable to do what I wanted to:

The names are capital English letters and numbers, low-ASCII. They don’t necessarily map to a specific file format; Some decoders, like “XMP” operate on multiple file types, and more than one decoder might handle the same file type, like “DRMP3” vs “MPG123”. Note that in that last example, neither decoder is called “MP3”.

So, we're looking at a hardcoded list of things to allow. But I like the idea of letting people play what they want, so I think I have a solution. We'll just make a text file in the assets folder called supported-audio-formats.csv and then read that to determine things! Nothing could go wrong with allowing user entered strings from a file to influence behavior in a game, right?

wav,ogg,flac,mp3,aiff,aif,au,snd,voc

I think it would be useful to be able to print out the available decoders if there's a file extension in the list that isn't in the allowed list. So, we can put that printing code into a little helper function:

fn print_available_decoders() {
    let n = sdl3::mixer::get_num_audio_decoders();
    println!("available audio decoders: {n}");
    for i in 0..n {
        if let Some(name) = sdl3::mixer::get_audio_decoder(i) {
            println!("  decoder {i} => {name}");
        }
    }
}

Then we can load up the CSV and have it ready to go. The only slightly awkward thing about the loading that's worth mentioning is that the load_bg_music's return type is a list of results, and not a result with a list in it. So, handling the potential failed file read needs to respect that:

let supported_extensions_file = self.base_path.join("supported-audio-formats.csv");
let read_result = fs::read_to_string(supported_extensions_file);
let Ok(full_csv) = read_result else {
    return vec![Err(read_result.unwrap_err().into())];
};
let allowed_extensions: HashSet<String> = full_csv.split(",").map(|s| s.trim().to_string()).collect();

But, once we've got our hash set ready to go, then we can update the iteration of the globbed files to check the extension:

for path in &globbed {
    let filename = path.file_name();
    if filename.is_none() {
        continue;
    }

    let Some(extension) = path.extension().map(|os_str| os_str.to_str()).flatten() else {
        continue;
    };
    if !allowed_extensions.contains(extension) {
        eprintln!(
            "cannot load {:?} file type not supported by available decoders or extension list, see supported-audio-formats.csv and decoder list below",
            filename
        );
        print_available_decoders();
        continue;
    }

And that all compiles, and now when I drop an mp3 file into the folder and run it…

cannot load Some("README.txt") file type not supported by available decoders or extension list, see supported-audio-formats.csv and decoder list below
available audio decoders: 9
  decoder 0 => WAV
  decoder 1 => STBVORBIS
  decoder 2 => DRFLAC
  decoder 3 => VOC
  decoder 4 => AIFF
  decoder 5 => AU
  decoder 6 => DRMP3
  decoder 7 => SINEWAVE
  decoder 8 => RAW
Couldn't open assets/audio/cc-vocaloid/2.wav: No such file or directory
cannot load music file a1.wav please name it numerically in the order you want played

Oops. Two of those are expected, I renamed "1.wav" to "a1.wav" so that it would fail and load the 2.mp3 file I had put in the folder as a test, the README obviously is just a text file so it fails, but the load error is my bad. It's coming from the fact that the loading code is expecting to only ever load wavs.

pub fn music_id_to_relative_path(id: MusicId) -> PathBuf {
    let base = PathBuf::new().join("audio");
    let wavs = PathBuf::new().join("audio").join("cc-vocaloid");
    match id {
        MUSIC_ID_PACHEBAL => base.join("Miku Pachebal.wav"),
        MUSIC_ID_MOON => base.join("miku fly to moon.wav"),
        MUSIC_ID_QUIT => base.join("selectedQuit.wav"),
        MUSIC_ID_TETO => base.join("tetowins.wav"),
        _ => wavs.join(format!("{}.wav", id.0)),
    }
}

So that's sort of a problem. Extensions don't really mean anything, so I could just rename the mp3 file and make it lie.

And that does work. But it certainly feels silly to expect some random user who wants to use their own music during the game to not only rename the files to be numeric, but also to rename the extension as well to something that it actually isn't. Someone could definitely get confused too and think they have to actually convert the file to the wav format as well, which they don't. Alright, so what do we do about this… the load_bg_music is re-using the load_music function which is what's calling the relative path helper:

 fn load_music(&mut self, id: MusicId) -> AudioResult<()> {
    if !self.music_by_id.get(&id).is_none() {
        return Ok(());
    }
    let path = self.base_path.join(music_id_to_relative_path(id));
    let ctx = &mut *self.context.borrow_mut();
    let audio = ctx.mixer.load_audio(path, true)?;
    self.music_by_id.insert(id, audio);
    Ok(())
}

So then, I suppose if I just load the music directly, then we can avoid this? Why is it that every time I want to re-use a helper function I end up needing to inline some weird custom version of it… Well, anyway, we just remove the call to load_audio and then build up the path using what we're globbing over in the loop:

let music_id = MusicId(desired_id.unwrap());
if !self.music_by_id.get(&music_id).is_none() {
    // Tricky tricky, make sure you put the music id if we're re-loading the level scene
    // and generating the list of music ids again. Just because we don't need to load the
    // music audio into the hashmap, doesn't mean we don't need to return the id here!
    ids.push(Ok(music_id));
}
let ctx = &mut *self.context.borrow_mut();
match ctx.mixer.load_audio(&user_wav_folder.join(path), true) {
    Ok(audio) => {
        self.music_by_id.insert(music_id, audio);
        ids.push(Ok(music_id));
    }
    Err(e) => {
        ids.push(Err(e.into()));
    }
}

As you can see by the comment, there was one interesting bug I experienced when letting the level fail and then reloading back in from the continue page. Since the game loads up the music ids from the call to this load_bg_music function, if we were to have a continue; inside of that hashmap check, we'd accidentally end up not having any music loaded at all! Which meant that there was no call to stop the music playing either. So, the game over music would keep playing in the level and you'd hear fly me to the moon. Fun times.

But with that, we've successfully made the user music folder flexible and able to play whatever SDL3 can load that the user might want to! Woohoo!

Future plans

My main goal in this post was to refactor the audio related code a bit. There's still some other things I can think of that would be fun to do, but also that I don't actually feel like doing right this second. So, we're going to put a pin in this refactoring adventure for a little while. The framework I've made only depends on and uses SDL3 in one place, so theoretically, I could port over everything we just did to a different native library or to something like WASM if I wanted to target the web.

But, before I get ahead of myself, the obvious thing I could add in to the audio interface that don't current exist is volume control. It would be trivial to add in a mute function, or maybe a generic set volume guy who takes in a number between 0.0 and 1.0 or similar. But the reason I'm not doing that is because I know that the moment I do, I'll want to wire it up into the game, and that means I'll need to make some assets and sliders and controls and will have to explain a bit more about the game than I did here, and I was kind of hoping to keep this blog post focused on just the audio side of things and less on the other bits in the game.

Mainly because the last post was really long, and so doing a few short bits here and there feels like a good mental break for me. I really like being the way I am sometimes 4, but that "way" is the way of the stubborn. If I start writing a blog post, it is very very rare for me to give up on it midway through and not post it. These posts are as much a dev log as they are intended to be tutorial-ish/stream of consciousness for someone to follow along with the hope that they might get a little inspired to do similar stuff on their own. They also function as motivation for me. If I'm making a blog, then I'm going to finish the thing, whatever it is. Sometimes that means that I de-scope some features so I can finish, and that's okay, but so far I'm 8/8 on starting and finishing each game in my 20 games challenge series, and I want to keep that streak going.5

So, a new post might pop up again soon relating to this if that's what floats into my head to do. Or, it might be something totally different, like something related to the microgame jam or maybe a new game post. I still haven't decided what the next game in the 20 game challenge will be, but whatever comes next, I hope to see you there. And, if that's not your thing and you just read this because you wanted to see some examples of how to use SDL3's mixer extension in rust, then here we are! There you are! 6 And I'll see you around!

If you'd like to play the Miku Miku Tower game and have more audio options, then you can check it out here on github