For my OpenRCT2-OpenMusic project I wanted to have a build pipeline similar to how software is built. Traditionally, artists work in a DAW and do a lot of tweaking until they get the sound they want. If the score changes it's usually not that easy to just replace it and render the audio again. Also, DAW projects are usually not easily tansferable to other systems.
For OpenRCT2-OpenMusic I wanted something different. I edit the score in the LilyPond language and instantly want to hear the results of my changes. In the beginning I would edit the score, render the sheet music and the MIDI file and later edit them in Ardour, which meant starting JACK and having to live with all the implications that come with it. And after making a minor change in the score, I would have to re-import the MIDI file into Ardour, move it to the correct track, delete the 'new' tracks, ... Also, I had to keep track of all the tweaks and settings I applied in Ardour, because I wanted all the tunes of the fairground organ to sound the same. For some time I even merged all the MIDI files of the different songs together and render them in one big project, which was also pretty impractical.
First, I thought about the process that is needed to convert the LilyPond-files in the repository to the final audio files. Here is a flow chart that describes the general idea:
I thought about writing a program that would just do everything, from running LilyPond to splitting and rendering the MIDI file with different SoundFonts to doing post-processing and the final mixdown. I even started working on it. But soon I realized that this would also become impractical and also would not scale well without concurrency. I realized that I could actually use a software build system that would automatically figure out the dependency graph and allow me to "build" the music. It would also allow me to use CI to just push to GitHub and let Travis build the 'artifacts' and push them to my webspace.
So I wrote a few shell scripts that would generate Ninja build files and run them in an Arch Linux Docker container, which is exactly how the build process works right now:
This graph is built from CI, so it always shows how the project was built. You can see how the process I described above is done for every song currently in the repository ("Render sheet music and MIDI" maps to "midi_pdf" and so on). Ninja allows me to build this in parallel without having to think about dependencies. Of course, this could also have been done with GNU Make.
The scripts also build the
midiprepare tool, the only program I actually had to write for this project.
Most of the scripts is actually just doing template work, but they still might be interesting: