Progress update : new save format, multiple selection, and sample-perfect synchronization

Martin Fouilleul  —  2 months, 3 weeks ago [Edited 0 minutes later]
Hello,

Here's a progress update for Top ! I would like to say that there's a new build available but that's not true : I prefer to keep v0.0.5 for later, and I will generally space out new builds.

The first few builds of Top ! are indeed very unstable and clunky, as they are the product of a work in progress at its early stages. It was important to put them out to give a general idea of what this piece of software will be about, and to build up my motivation and commitment for that project. Now that this first phase is done, and that I'm in the slow process of turning this early prototype into something worth using and shipping, working on it on a regular basis, I feel that I should strive for more stable and usable builds, and focus more on long-term improvements. In this regard, one month can be an amazingly short amount of time, especially with a day job !!

So, I expect Top ! to keep the same set of bare bones features for a while, as I will try to improve its stability and its performance. And I will put new builds online when they reach a higher level of polishing. Growing deeper rather than bigger, or something like that ! After all, we're all here to try to enhance the quality of software, isn't it ?

Anyway, after this brief introspection sequence, what's going on under the hood ?

New save file format

The previous save system was based on a text file format. I had written a quick and dirty grammar and let Yacc generate the parser, which was fine for a first attempt (I had prior experience with Lex/Yacc and the grammar was quite simple), but I felt that it was clunky and not easy to expand. Furthermore, the parser was allocating / initialiazing cues during the parsing, which did not fit very well into the new memory model and caused problems for cues referencing other cues (such as control or mix cues).

So I switched to a much more simple (and robust) binary file format, which has essentially one section for each permanent memory arena of the app and requires minimal conversion between "cold" and "hot" states. You can read the comments from Allen Webster about file formats below this post. Thanks for sharing your thoughts Allen !

The save file begins with a header containing a magic number, a version number and a checksum for the entire file, followed by the audio settings of the session, and then a table of content listing different sections of the file. Each section maps to one of the memory arenas of the app, so there's one for the cuelist, one for the cue-specific states (ie the file reference index for audio cues, the mix curve for mix cues, etc..), one for the mixing matrix, one for the audio files references. Finally there's one big strings table at the end. The other parts of the file can refer to variable-length strings by keeping an index into the strings table, which allows to describe most of the file format with fixed-size structures.

The advantage with this new save file format is that I can perform global consistency checks before loading, and I can then convert the file sections into the application's memory arenas "all at once" because all the data I need is at a known location, and most of it is fixed-size. I also avoid the problem of referencing data that has not yet been parsed (or the need to build and keep a syntax tree representation during the process).

Multiple/range selection and editing

It is now possible select multiple cues in the cuelist and move/delete them at once. It works like the classic multi-selection in a file browser, ie. Cmd-click to add/remove a cue from the selection, and Shift-move to extend or reduce the current selection region.

Currently it is not possible to select cues from different 'levels' in the cuelist hierarchy, because it would not make a lot of sense to move them together (without loosing their relative grouping), or to select a group and some of its child cues at the same time (because moving a group implicitly moves all of its cues). However it can seem a litte counter-intuitive to be restricted to a group when selecting multiple cues, so I might make arbitrary decisions for these corner-cases, and implement an unrestricted multi-selection.

Along the way I fixed a bug in the code that handles moving cues in and out of groups, which previously prevented moving existing cues into an empty group.


New synchronization system

I discovered several bugs in the transport subsystem, which is responsible for synchronizing all playing cues to a common time frame, with the proper offsets depending on their "cueIn" property and their position in the cuelist hierarchy.

Race conditions

One kind of bugs were caused by race conditions between the transport system running in the audio thread, and the playback commands issued from the GUI in the main thread. In short the status of a group cue (eg. playing vs. halted) would be changed as a result of a playback command, while its children cues were being processed as if the group was playing, resulting on a inconsistent state.
It would not cause problem for a non group cue, or if the change happened after or before all children cues of a group were processed (as it would just "delay" the effect of the status change to the next iteration of the audio processing). But the problem arises when some children have been processed according to some state of their parent, and then some other children are processed with another state. In this case, the cue start / stops would be out of sync from a small number of audio samples. This is not noticeable in most cases, and currently it doesn't make a big difference, but it could become a problem when we want to extend the functionnality of control cues : as a simple example, we can imagine two control cue sending MIDI messages to an external device at the same date. It would be possible under that race condition that one is sent but not the other. Generally, any kind of behaviour relying on strict synchronicity would not be feasible.

On the other hand, it would be extremely inefficient to acquire a mutex each time the transport code is run in the audio thread, only to protect us from the occasional playback commands comming from the main thread. So I implemented a simple thread-safe queue, where the playback commands are pushed by the main thread, and from which the audio thread reads commands at the begining of each audio processing iteration. From the point of view of the audio thread, it amounts to saying that playback status changes can only happen atomically between two processing iterations, which ensure the consistency of the synchronization inside each block.

Sample correct synchronicity

The other kind of bug was related to the fact that all the dsp was processed on blocks of a fixed size. That's fine if we don't take control cues into account, because in this case the time position of each cue depends only on its parent group, and with a simple traversal of the cuelist we ensure that the parents have been computed before their children. If a group stops in the middle of a block, its children know where to stop.
But if we add control cues, we can have a situation in which the control cue triggers in the middle of a block, and sends a stop message to an audio cue that has already been processed. Here again, that would delay the effect of the control cue from a small number of samples, causing a unnoticeable lag. But again, the problem arises when two events are supposed to be synchronous but applies to two different cues, one which is higher in the cuelist (so it has already been processed and the event is deferred to the next block), and one which is deeper in the cuelist (so it will correctly be processed). In this case the two cues are desynchronized. Furthermore, repeating this kind of small delay over the course of multiple runs of a loop (I will eventually add a looping feature to group cues), will cause a very noticeable out of sync effect, which is not tolerable.
In order to avoid these corner cases, I modified the dsp code to proceed on a sample-by-sample basis, with each control event taking effect at the next sample. This way there is a strict synchronicity and the transport behaviour is much more predictable. However, this means traversing the cuelist for each sample, which can be quite expensive and may cause performance problems for high sample rates and big cuelists.

One way to retain the block by block model would be to introduce a form of backtracking : traverse the cuelist and compute all samples in the block, reducing the block size each time we hit a control event, then backtrack to the minimum size found, and restart from here. In most cases, it would behave as the simple blockwise solution, only occasionally it would recompute part of the block. But for the time being I will stick with the sample-by-sample solution because it's much more simple to understand and I will see after a while how I can optimize it.


What's next

Since what I first saw as a change in the memory management of Top ! transformed into a (more or less) complete rewrite, I thought I would share my mixed feelings on the subject of rewriting vs. refactoring (spoiler: next time I'll try the second option).
But it's already a very talkative and long post so if you've got this far, you'd rather be rewarded with an image of a baby octopus :




Anyway, what's next ? As I said in the intro, I want to take some time to improve the stability of Top !, to make it handle gracefully all kinds of corner cases and errors, to improve performance where it can already be done, and to enhance the overall confidence that one can put in this piece of software.

I feel that in order to efficiently achieve these goals, I will need some kind of automated testing infrastructure. For now, I just have some basic unit tests that I wrote along the way, and can optionally run as part of my build process, but it doesn't cover nearly as much as it should, and I have no real way of automatically testing the application itself. So, that could be the next step.
If you have an experience with unit or integration testing, if you think it's useful or if you think the opposite, if you use some special technique or have any kind of personal insights on the subject, please write a comment ! Opinions, help and advices are alway much appreciated !



Thanks for reading,

Martin
Log in to comment