Hi, it's me again! One year ago, I made the video below. I wasn't planning to, but I just rewatched it in its entirety. It's just that good. Near the end of the video, I said However, I do not think I am going to pursue this technique of audio synthesis further. I was wrong. Earlier this year, maybe even before the change of year, I started considering the idea of writing a program that would take user-written data about what notes to play and automatically write sheet music based on that. Such a program would take a lot of the tedium out of making music in this way, and make it much easier to make modifications to the composition. One reason to not be so sure whether to do this was that I knew that the results probably wouldn't be great. MuseScore isn't meant for this, and I'd always got rather inconsistent results, despite the amazing proof of concept above. I also wasn't sure whether it wouldn't be too much of a hassle to write files for MuseScore to read. After looking into it, the latter turned out not to be an issue. MuseScore's default file format is compressed, and thus hard to read or write, but it is possible to export to an uncompressed XML file format. This would be quite doable to write to, and I could start with a template XML file containing all the information that gets generated when you create a new score. The former issue was more concerning. Would it be worth my time to invest into a project that would probably not allow for reliable music creation? Actually, yes. Because I had a switch of mindset: my goal wasn't anymore to acquire a new way to make music, but to invent, to hack, to develop. Inspiration for this comes from the chiptune scene: in particular, in 2021 some major advancements were made in Atari ST chiptune. I grew up with the Atari ST, a home computer from the late 1980's, and always loved its sounds. However, when I was young, people could get less different sounds out of it than is now possible, and back then, they could get much more out of it than when it was initially released. Hackers are still developing new ways of creating sound on this old machine. And one of the recently developed sounds actually is somewhat inconsistent, and it doesn't even work on emulators yet. It seems really cool to me what they're doing, but dissecting a computer like they're doing, and coding programs in assembly (at least I assume that's what they're doing) seems a bridge or two too far for me. However, this project seemed tangible. This might be something that I can wrap my head around. And whatever I get out of it, it would be a fun and interesting project. Last week I got sick, and that seemed like the perfect time to start development. Over two days I worked on it a lot, then I took a one day break, and the day after I worked on it some more until I had it to a somewhat complete first version. I had decided to couple this project with learning to use git, so I set up git, committed, and pushed to github. If you would like, you can check out my project at https://github.com/607GitHub/MuseSynthesis. Since the initial release, I've done several more commits: most trying to implement portamento. Unfortunately, I ended up with an issue that I don't think I can solve. Let me know if you're interested in collaborating. I can explain the gist of the problem in another post, if you're curious (you are under no obligation to then find a solution for me ). There are however two feature that I deem more important: support for rests, rather than only continuous notes, and support for multiple voices. Both of these seem tricky to model well, though, especially because they have to work together nicely (eg. voice 1 might have a rest while voice 2 plays notes). I'm hoping to have a go at it some time soon, though. If you have experience coding, feel free to give me feedback here. Of course you could also fork the project and have your own go at it, but I can't promise that I'll accept a pull request, however good it is, as for now this feels like my personal learning project. I'm not sure yet when or if I will make a proper release, with a readme, compiled binary, et cetera. This is mainly because of the unpredictable results. I wouldn't like to disappoint people by having them compose a tune for MuseSynthesis, only to have MuseScore royally mess up the render of it. But I've been having quite some fun with this, and I thought it would be nice to share it with you. It is fun to explore new territory, develop tools, and test limits!
Thanks for sharing that! I hadn't seen it, and it's very interesting. So interesting that I'm really disappointed that that's apparently all there is. I want to hear (or even read) more information...
battleofthebits.org, a website that organises competitions in digital art, mostly music, currently has a competition going on with a MuseScore category. I would like to submit an entry there with my program, as I'm quite curious how people would react! This should be right up their alley, and yet I don't think it has been done before; so far the MuseScore category uses MuseScore pretty much as it was meant to be used, as far as I'm aware. Before creating an entry with it, I'd like to support rests and a second voice. I started on the former earlier and finished it today, and will get started on the latter soon... I have little idea of how hard it will be, at the moment. The thing is, I am very unsure about whether I will be able to create something alright sounding at all. In the original video it worked really well, but it appears to have been somewhat of a fluke: usually MuseScore just can't cope and will mess up the render. We'll see if I can get something working.
I've managed to get both rests and harmony working. Yesterday, after getting harmony working, the first result was unexpectedly disappointing. I was playing a C major chord, but I was only perceiving a very low C and a high B (which isn't even in C major). Huh? I figured this could be explained by the harmonic series, and yeah, I think it is. Left: what is played. Right: what you perceive. If you haven't learned about the harmonic series before, you'll have no idea what this picture means, but hopefully if you have, you get it. Anyway, whereas this was really cool to figure out, it was also annoying, of course, as the goal was to be able to use major chords, and not have them sound totally different. But I was able to get around the issue by making each voice use a different drum. Before After I then created a sequence of some chords... of course only major chords, as the first time a minor chord occurs in the harmonic series is to high up to use (I did try it! It wouldn't play back). But playback still gets worse and worse as time goes on! :/ I - IV - V - I (warning: this clip is a lot louder than the two above) I found out yesterday that you can turn off reverb in MuseScore, which I was amazed with (you can also edit the reverb... this might be quite useful for other MuseScore projects, actually!), and I was hoping that turning off reverb would greatly improve the result... but it didn't seem to help at all. If any geeks would like to help look for a way to get the result more consistent, let me know. ^.^
I submitted. https://battleofthebits.org/arena/Entry/Uncharted Territory/53117/ I am indeed very curious how this will be received!
There are releases on GitHub now! Here's the link again: https://github.com/607GitHub/MuseSynthesis Not sure yet if I'll work on more features... I have a lot of other projects I'd like to spend time on! But maybe if people show interest...
MuseScore 4 was released recently, and unfortunately this way of synthesizing sound doesn't work in it. There is some kind of internal limit to the speed at which notes can be played, which is at an audible pitch, but not high enough. This is quite unfortunate; I was hoping it to work better in MuseScore 4, and thought it very possible that it would work worse, but to be completely impossible like this was disappointing! I might still develop the program further at some point, because it is an interesting way to make sound. I'll look into how it behaves in MuseScore 2 and MuseScore 1... maybe in one of those it's more stable? I took a look at MuseScore 2 this evening, but it's not as straight-forward to test as I hoped, because the XML format changed between versions, so MuseSynthesis' generated data aren't compatible with it.
