There were a lot of changes under the hood to improve web support in the latest version. Read on for technical details. tl;dr - Audio is now streamed, leading to faster load times. Firefox now works.

Part 1 - Streaming Audio

For both cached and un-cached loads, load time was dominated by unpacking the stored mp3s for playback. This used to be true on desktop as well, but things there substantially improved when I switch over to kira’s streaming api. Using streaming on web would make the load as fast as downloading the the file itself (almost instant when cached). Unfortunately, Kira did not support streaming on web - for good reasons it turns out.

Since Kira didn’t support the feature I wanted, I decided to write my own small implementation of audio loading and playback. I grabbed the cpal and symphonia libraries which underlie Kira and got to work. My first attempts were very naive and resulted in a lot of playback stuttering and other issues which made for an unpleasant experience.

Eventually though, I had smooth playback working by using a web worker via wasm_thread in the background to perform the unpacking, passing completed frames to a rtrb channel to communicate with the foreground thread writing frames to cpal’s output. The background thread sleeps whenever it detects that the send buffer is full to help reduce cpu usage.

This worked great! Until I tried to run the desktop version. It turned out that one of the tracks I include has a sample rate higher than cpal’s supported sample rate on desktop, causing the sound to fail to play. I was able to fix this using the rubato library to convert between sample rates. I use a simple fixed in/out FFT to convert between the mp3’s sample rate and cpal’s default output sample rate.

Now I had working streaming audio on both web and desktop, and saw significant reductions in web load times as expected. However, I noticed some stuttering when the system was under high load caused by cpal’s audio output being run on the main browser thread using timeouts, meaning it shared timeslices with rendering. This would cause audio to stutter whenever rendering lagged due to the system being under load.

This lead to…

Part 2 - Async Render

In order to fix the audio stutters, I decided to move rendering off the main thread. This was complicated by the fact that wgpu doesn’t allow the device to be sent across threads, and creating the device on a web worker causes surface creation to fail. The initial solution was to move computation of the new cache entries and text compositor inputs onto the background thread, leaving the shader execution and ui layout logic on the main thread. The main thread would push updates to the screen text to the background thread, which would recompute the layout and then push the result into a ringbuffer for the main thread to pickup and actually present.

I decided to change the text compositor slightly as part of this to reduce the amount of data I was pushing to the main thread. Now the compositor uses 3 small images which describe the atlas location, fg color, and bg color of each piece of text on the screen, with the compositor re-rendering all the text in a pass. This had no noticeable affect on render performance, but simplified the inter-thread communication significantly.

This worked great, but had some frame stutters and other issues caused by the separation of ui computation from frame computation. It also meant a lot of duplicate data was being communicated between the threads and there were still occasional stutters when rendering a new scene with a lot of text updates. To fix this, I moved the ui computation into the background thread as well.

Moving the ui computation required working around boa’s lack of support for multi-threading. Since both the main thread (for saving/loading) and the render thread (ui updates, input tracking) need access to the global js state, I had to add logic to sync updates between the frontend and backend. The current state of this isn’t ideal, but it works.

With all of that finished, audio now plays smoothly without stutters even when the system is busy.

Appendix - Firefox Support

I realized I hadn’t tested for Firefox when I originally put the game on up on web, so I decided to go ahead and make sure it worked. It didn’t due to a few minor issues with webgl defaults on Firefox (group bindings must be 16 byte aligned). They were simple to fix though, and now I support both Chrome and Firefox. Chrome and Firefox mobile also sort of work. The experience isn’t great, even with large text turned on and the shader off. But it’s there if you want to play it.