MusicRadar’s got your back
Our team of expert musicians and producers spends hours testing products to help you choose the best music-making gear for you. Find out more about how we test.
For as long as I have been teaching music, people have asked me if there is some way to automatically turn audio recordings into sheet music. Until recently, I told them that it couldn’t be done. But now there are a few AI products on the market that claim to do exactly that.
Klang.io promises to turn audio into notated transcriptions, lead sheets and guitar tablature. This would be a valuable tool for music educators, songwriters and composers alike. But does it work? We recently reviewed an AI transcription tool called Songscription, and we were not very impressed. It detects pitches well, but it does not do so well with rhythm. Can Klang.io do better? Let’s find out.
What is it?
The word “klang” suggests “clang” to English speakers, but it means “musical tone” or “musical note” in German. Klang.io has been in operation as a company for a surprisingly long time; the hype around AI might be recent, but researchers have been working on the underlying machine learning technologies for decades.
Klang.io has been offering computer-assisted transcription tools for specific instruments for several years, but their latest offering is their Transcription Studio, a browser-based audio-to-notation platform. (It’s also available as a VST3/AU plugin, but for this review, we only tested the browser-based tool.)
In theory, you can drop any piece of music into Klang.io and get a notated score. However, these kinds of tools are only as good as their training data, and Klang.io’s transcriptions work better on some kinds of music than others, as we will see below.
There are many reasons for a music educator like me to be concerned about AI. Is this tool going to be an excuse to lay all of us off, even if it doesn’t do our job as well as we do?
One disclosure before we go any further: I came to this review with a strong anti-AI bias. I believe that generative AI is harmful to musicians and educators, and that we should approach it with skepticism. Before I started writing, I spoke to the company’s founder, Sebastian Murgul, along with his public relations representative. I came away liking Sebastian and feeling more open-minded toward the product as a result. (I know that this is the whole reason why companies have PR reps, but it was also an organic reaction on my part.)
Why should you care about my thought process, or my feelings about a tech company founder? In assessing a new technology, it isn’t enough to ask what the tool does now; you also want to think about where it might be headed in the future. There are many reasons for a music educator like me to be concerned about AI. Is this tool going to be an excuse to lay all of us off, even if it doesn’t do our job as well as we do?
Sebastian Murgul shares my concerns. He is a deeply nerdy person who just wants to make a useful product, not to disrupt (or destroy) music education as we know it. He does not actually think AI can replace human music transcription and analysis, and he recognizes that it can’t make sense out of every kind of recording.
Murgul has some specific cases in mind for Klang.io: for DAW-based composers of film and game music to be able to communicate their ideas to human performers, for example, and for pop songwriters to be able to generate lead sheets for copyright registration. (In the US, you only have to submit a sound recording of your song, but in Latin America you have to submit a notated lead sheet too.)
Murgul is clear that users will only get good results from Klang.io if they already have a good understanding of the music they are trying to transcribe, and if they are able to edit its output themselves.
When you enter the Klang.io Transcription Studio, you can choose between uploading an audio file, entering a link to YouTube, Instagram or TikTok (but not Bandcamp or SoundCloud), or recording straight into the computer mic. From there, you select a transcription mode from the options below.
- Multi-Instrument Mode: the default setting.
- Single-Instrument Mode: the most accurate and reliable mode, especially if it’s a transparently recorded acoustic instrument like piano.
- Classic Mode: this setting is optimized for classical music.
- Rock Mode: a beta feature. It differs from multi-instrument mode in that it is optimized more specifically for guitar, especially distorted guitar.
- Arrangement Mode: rather than transcribing each individual instrument or voice, this mode produces a lead sheet, just top-line melody and chord symbols. (Based on my tests, this should probably be labeled as a beta product too.)
After choosing a mode, you select the instruments in the recording. You can have Klang.io detect them for you, but it performs better if you tell it what to listen for. Finally, on the Additional Options screen, you can specify key, time signature, approximate tempo, note duration quantization, and whether you want triplets or not. Again, you can have the software detect all of this, but the results will be less accurate.
Once you have created your transcription, you can view and listen to it as a notated score or as a vertically scrolling piano roll. You can export your score in various formats, including Music XML and MIDI. Klang.io also has Note Editor mode, a rudimentary notation editor, though for any serious overhauls or formatting you will want to work in a separate program like MuseScore, Dorico or Finale.
The most interesting feature of the Note Editor is the Anacrusis (Pickup Bar) setting. Klang’s makers recognize that the software has trouble identifying downbeats, especially if the recording has any silence at the beginning, so they make it easy to move the location of the first beat of the first bar. Unfortunately, you can only move the downbeat in increments of one beat. (This is an issue, as I discuss in the next section.)
Performance
In testing out Klang.io, I repeated some of the same tests I performed on Songscription for the sake of comparison, but I mostly selected pieces of music that were better suited to showing Klang.io’s strengths and weaknesses.
I started with Nina Simone’s recording of the blues standard Nobody’s Fault But Mine. It’s a simple piano-plus-vocal recording, and had I recently assigned my aural skills students to transcribe it. Could they have used Klang.io to cheat on the assignment?
The answer is: sort of. Klang detected the notes and rhythms of the piano part very accurately. However, it gave the key signature incorrectly as D major; the song is a blues in A. In fairness, there is no consensus on the correct way to write key signatures for blues, but A or A minor would be the usual choices.
Klang’s take on Nina Simone’s vocal was less accurate. It simplified her melismas significantly, and it was thrown off by her vocal timbre, because notes that she sang with more inflection appeared in the transcription on the wrong pitches.
For a comparison with Songscription, I tried Crazy by Patsy Cline. Songscription had failed to make head or tail out of the song, presumably because its training data doesn’t account for 1950s recording fidelity or instrumentation.
Klang.io did not do much better when I asked it for a lead sheet. It could not distinguish the lead vocal from the backing vocal or instruments, and the triplet feel threw off its rhythmic detection, so it couldn’t find the tempo or downbeat. It did get the chord symbols mostly right, and it detected the lyrics accurately too.
However, Klang shows lyrics in a strange way. Rather than placing each syllable with its corresponding note, it groups entire phrases together in the middle of their approximate location on the score. I would prefer to have lyrics be on a separate page; it is confusing to see them placed so unconventionally.
PRICING
Klang.io is only available on subscription. The Pro tier, which allows you 50 transcriptions per month, is priced at £6.67 per month billed annually, or £16.99 per month billed monthly. If you’d like to test it out, you can transcribe audio clips of up to 20 seconds in length for free.
I wondered whether Klang.io would do better on a multi-instrument transcription of Crazy. This time, I specified the instruments, the key, the time signature and the approximate tempo, and also told it that Patsy Cline is an alto. Klang performed better this way. The bassline was quite accurate. The vocal melody was still mixed up with other parts, but now the meter and rhythm were right. The guitar and piano parts were more hit and miss, and the drum part was completely chaotic; I guess the training data doesn’t include drums played with brushes.
Next, I tested Single Instrument mode with a solo piano recording of Blue Monk by Thelonious Monk. Klang offered me the choice of pop or classical; jazz was not an option. I chose pop, because that seemed closer. I also entered the key and tempo. The results were good, at least until Monk’s improvised solo started. The software rendered some of his expressive timing strangely, but that is understandable; many human transcribers would struggle to represent those rhythms.
Klang’s chord symbols were mostly on point, but with some odd omissions. The chord library does not include ninth chords, so Klang wrote Bb9 as Fm7. In Monk’s solo, something about his timing threw off Klang’s beat detection, so even though his tempo is steady throughout, the transcription was off by one beat from that point. Unfortunately, the anacrusis correction feature only works for the first downbeat. This is something you could easily correct in an external editor, though.
Klang’s training data includes lots of classical music, so I expected it would have an easy time with Prelude No. 1 from the Well-Tempered Clavier by Johann Sebastian Bach, as performed by Glenn Gould. I was right: the transcription took one tenth as long as any of the others I had done so far. It was quite accurate, aside from the fact that it was confused by the silence at the beginning of the recording, which it rendered as a one-and-a-half-beat-long rest. The anacrusis correction only works in increments of one beat, so you would need to fix this in an external editor.
I was interested to see that Klang added chord symbols to the Bach piece, with about 95% accuracy. I sometimes write jazz-style chord symbols on classical scores for harmonic analysis purposes. It is intriguing to consider the idea of having software do it automatically at scale, though I think people should be doing this kind of thing by hand for the sake of their own understanding.
For a further test of Classic Mode, I used Bach’s Contrapunctus I from the Art of Fugue, performed by the Emerson String Quartet. This chart was mostly accurate too, though Klang missed the first note, along with a few others that were played quietly. Klang also combined the violin and viola onto the same staff. Once again, it placed chord symbols throughout, which is not a feature that most classical performers would want, but that jazz and pop musicians might find extraordinarily useful.
I wanted to see how Klang would do with electric piano, which is not one of the instruments on its list. So I gave it What’d I Say by Ray Charles. I gave Klang the key and tempo, and told it to listen for vocals, piano and drums. It did pretty well on the electric piano, with good pitch accuracy and timing, and mostly accurate chord symbols.
The timbre of the electric piano did confuse it somewhat, because it heard the top notes as a vocal part. Klang got the broad outline of Ray Charles’ actual vocal melody, but the rhythms were squared off and the pitch was sometimes a half-step off. As with Crazy, Klang could not even approximate the drums.
The vocal part was uneven, with David Byrne’s idiosyncratic singing clearly throwing the software off
Maybe the noisy vintage recording was throwing the transcription off? I figured that a cleaner, more contemporary recording would work better, so I tried Everybody Laughs by David Byrne. As I expected, the results were more impressive. Klang misidentified acoustic guitar strums as hi-hats, but the guitar part itself was remarkably good, with clear and accurate tablature along with notation.
The vocal part was uneven, with David Byrne’s idiosyncratic singing clearly throwing the software off. Klang did pretty well with the string arrangement, showing that the training data includes a lot of orchestral music. The only real problem was that it heard harmonics as separate notes in places, but this is a problem that all pitch detection algorithms share.
For a more mainstream pop recording, I used Call Me Maybe by Carly Rae Jepsen, assuming that this kind of loud, bright and clear pop banger would be easy to transcribe. Klang wrote this vocal line extremely accurately. However, it had trouble distinguishing the other instruments, presumably because they are layered and processed so heavily.
For a more authentic transcription challenge, I wanted a song that I didn’t already know how to play. I chose Burnin’ Hell by John Lee Hooker. I wasn’t expecting much for the vocal, but I thought Klang might do well with the guitar, since it is unaccompanied and has clean tone. Sadly, whether it was Hooker’s open tuning or playing style, Klang couldn’t make head or tail of it. I guess there isn’t much blues guitar in the training data.
Some measures were perfectly clear and accurate, while others were chaotic
How about the Beatles? I tried Blackbird, figuring that it is sufficiently classical-informed in its harmony and playing style. However, I forgot that the song has a lot of meter changes, and Klang understandably could not follow them. Also, it had trouble distinguishing Paul McCartney’s voice from the guitar. So some measures were perfectly clear and accurate, while others were chaotic.
My final test was one of my own original compositions, a track I made in Ableton Live using only synthesizers and drum machines. I can’t find my Ableton session anywhere, and I can’t remember exactly what notes or chords I used. I’m sure I could figure it out, but I have never had the time to sit down and do it. This is the exact use case that the company has in mind.
Klang has an option for synthesizers, but there is clearly a limit to the weirdness and inharmonicity that it can cope with, so its transcription of my melody and bassline was not very good. However, it did get the majority of the chords right. The transcription is not usable as a guide for a performer, but it would be quite helpful to me if I were to transcribe the track myself.
Verdict
As with other transcription and audio-to-MIDI tools I have tried, Klang.io does very well at detecting pitches, especially on single instruments that are recorded clearly. It also does a good job with vocals in a conventional pop style. It can identify note onsets accurately, and if the rhythms are straightforward, it can organize those rhythms into a meter fairly well.
However, if the rhythms are too far off the grid, or if they’re too swung and idiosyncratic, Klang struggles. The more complex the recording, the less able Klang is to separate the layers, and older or noisier recordings are a problem too. Also, even when Klang’s output is accurate, you would still need to do extensive editing and formatting before you could hand charts out to an ensemble.
The difference between Klang.io and its competitors is that its designers recognize the limitations of their tool
The difference between Klang.io and its competitors is that its designers recognize the limitations of their tool. They are assuming that you are not a complete novice, but rather that you already have some musical sophistication. The user interface makes clear that the more you can inform the software in advance, the better the results will be, and that you should expect to edit and adjust the output afterwards.
The best way to understand Klang.io is not as a tool for generating sheet music. Even if its scores were perfectly accurate, writing sheet music for human performers requires more than identifying the notes. You need to use your judgment to decide how to lay everything out and how much detail the performers will need. (I expect AI transcription to get more accurate, but I do not expect it to develop editorial or graphic design skills.) I think it’s better to think of Klang.io as a tool that gives you clues for writing your own sheet music, and in that application, it has significant value.
