← All episodes
Brian Siegelwax

Episode · Sep 12, 2022

Eduardo Miranda & Brian Siegelwax, Plymouth

Composing music with quantum circuits

Summary

Yuval Boger interviews Prof. Eduardo Miranda and Brian Siegelwax, authors of “Teaching Qubits to Sing: Mission Impossible?”, a fascinating work about creating music with quantum computers. They talk about how the program works, their plans for future work, finishing Schubert’s Unfinished Symphony, and much more. 

Key takeaways

  • Miranda and Siegelwax built a quantum program that maps qubit measurements onto musical parameters like pitch and rhythm, using superposition to generate melodic variations.
  • One goal of the project is to use quantum computing to generate candidate material for the missing movements of Schubert's Unfinished Symphony.
  • They discuss whether quantum randomness gives composers something genuinely different from the pseudorandom number generators already used in algorithmic music.
  • Future work focuses on scaling the approach beyond simple sequences and improving how quantum circuit outputs get translated into notation a musician can actually play.

Transcript

Yuval Boger: Hello, Professor Miranda. Hello, Brian. Thanks for joining me today.

Eduardo Miranda: Thank you. Hello.

Yuval: So Eduardo, who are you and what do you do?

Eduardo: I am Eduardo Miranda. I am a professor in computer music at the University of Plymouth, and currently I am developing research into using quantum computers to create music.

Yuval: And Brian, who are you and what do you do?

Brian Siegelwax: I am Brian Siegelwax. I write code, I write about code, and I also recruit people who write code.

Yuval: Fantastic. And we are here to talk about singing qubits, right? The Qubit Choir. Eduardo, you've been on The Qubit Guy's Podcast before, and I know you're using quantum computers to compose music.

Could one of you, or both of you, tell me how this Singing Qubits project came about?

Eduardo: Well, I think Brian and I have been communicating on social media, exchanging some ideas and so on. And I think naturally both of us wanted to develop something together. And then the opportunity to develop a generative music system came about.

We exchanged some ideas about how we could find a simple way to program a quantum computer to generate sequences of things. Brian can probably tell you a little bit more about how the technical idea came up. But the inspiration I had was, well, if we can generate sequences of musical notes, what would be an interesting instrument to play the sequences?

And at the time, coincidentally, I was working with vocal synthesis. Then I said, "Okay, so let's hook this quantum system to a vocal synthesizer and let's make the qubits sing."

Yuval: So Brian, if I understand correctly, there's a vocal synthesizer, which means that I could put in any file or collection of notes and it would generate the singing sound, and the quantum part is the part that composes the sequence of the notes.

Is that correct?

Brian: Correct. So we started with a sample file, and we used that to generate the rules, and then the rules get encoded into the amplitudes of the qubits, and then the qubits decide what the sequence will be.

Yuval: So let me get this straight. If I have, you know, do, re, mi, fa, sol, la, si…

Usually the next note would be a do, right? Or in English, A, B, C, D, E, F, G, and then A. So that sounds like a probabilistic state machine, right? You could say, well, if this is the sequence, then the next note, with high probability or with some probability, is this, and that, and so on.

That's it. How is that different from the quantum generation?

Eduardo: Okay. So the system has two phases: what we call a learning phase and a generative phase. The learning phase is pretty classical, so to speak. We input music into the system, then we statistically measure how many times note X follows note Y, or how many times three notes follow four other notes, things like that.

So we build a set of rules, then we put these rules into what would be the equivalent of a transition matrix. Okay? So that is the learning phase. We can input any musical sequences to the system, and the system will work out what these rules are. Then the classical way of doing this would be to program the machine to go through this transition table and pick notes that would follow certain probabilistic, you know, Gaussian or whatever, tendencies.

We found an easier way to do that, which is to use the quantum machine to build quantum states with the amplitudes reflecting those probabilities. So it is a much more elegant way, in my view, and I think we can scale this up in the future much more easily than if we were to continue doing this classically.

Yuval: So Brian, maybe you could talk to me a little bit about what's going on on the inside. How does the quantum circuit look? How do you prepare the state? What are you measuring on the output? Could you tell me a little bit about that, please?

Brian: Yeah, the interesting thing is there's no one circuit to show.

They're all generated on the fly. So let's say we want to use three notes to generate the next note, and let's say we have A, B, C, and we go through this original tune and count how many times A, B, C appears. So let's say, for argument's sake, A, B, C appears 10 times; seven times the next note in the sequence is D, and three times the next note in the sequence is E.

And they're all binary coded. So let's say 000 is A, 001 is B, and so forth. So what we have, and from your standpoint, Yuval, it's the classic state preparation function. We want 0% probability of 000, A; zero for B; zero for C; then 70% for D, 30% for E, and zero for the others. So you have eight probabilities, except that we used Qiskit's initialize function, which requires us to convert those into amplitudes.

So we have to take the square root of the probabilities. We have to take one extra step, and then the initialize function generates the circuit on the fly. So in this case it'll generate a circuit that, when we measure it, will have a 70% probability of measuring… I got ahead of myself.

What's D? One zero? One one? Whatever it is, but 011, I think, which is D, and then 100 or whatever E is. And then let's say we measure D. I should be able to do this in my head faster. But anyway, then we have B, C, D as the sequence. Then we look back in the transition matrix, and we have another round of amplitudes, and the initialize function generates a completely different circuit that, when measured, produces the output with the probabilities, not that we want, but the probabilities that came from that original file.

So you end up with a tune that sounds somewhat like the original, but depending on how many notes we use, we can alter how much it sounds like the original, how much it sounds different, how much variety there is. But the core piece, the quantum piece, is generating those circuits that get measured with the probabilities for the allowed next notes.

Yuval: And what type of music did you use as input? Was it Bach? Was it The Beatles? Was it Beethoven? What was going on there?

Eduardo: We used a number of things. We used classical music melodies, like Bach, Mozart, whatever. It so happened that I had a few MIDI files of film music.

And so we had this quirky idea: let's use, for the paper, the tune from the Mission Impossible film. Because, you know, is it possible to program qubits to sing? Is it a mission impossible? No, it is possible. So the tunes mentioned in the paper are from the Mission Impossible theme, and we also used a Bach cello sequence to show that what we do is generic.

Yuval: Was there a particular type of music that worked better, that was more deterministic or that was easier to sort of follow?

Eduardo: No. I think because we are dealing with probabilities, it's a stochastic model, those inputs that are more repetitive will tend to generate music that is more similar to the input.

But those pieces of music where the notes vary more, where there is no obvious beat or pattern that you can identify immediately, for me as a composer, generate more interesting results. What we are doing now is trying to go beyond generating only sequences of notes.

Because I think now we have the beauty of this core quantum algorithm that generates sequences. What I'm going to do now is provide Brian with codes that would represent other things in the composition. For example, dynamics, speed, maybe more voices.

And because we are dealing with speech synthesis, there are things in speech that are quite interesting, like intonation and prosody, things that relate quite well to speech research. So I think what we are doing can have a nice connection with quantum natural language processing, for example.

I know these works are very symbolic. They are looking just at written words, but I think we have the potential here to add the synthesis part, the actual rendering of the speech, which will then give another dimension to language, which is the inflection of the speech, the emotions, and things like that.

So I think this is where music can fit in quite interestingly in scientific research.

Yuval: You mentioned speed and dynamics, but the current circuit generates the melody. Correct? Are you working on generating the harmony as well?

Eduardo: Yes, we are doing that. Of course, the encoding of these materials will kind of explode the dimensions of what we are dealing with.

At the moment, we have two dimensions, which are pitch and the duration of the note. But if we begin to add other elements, like what instrument plays this, what's the loudness of this note, how do we actually play it? Is it a plucked string? Is it a bowed string? Is it a consonant? Is it a vowel?

So once you begin to add these other dimensions, that's where I think quantum computation will become quite powerful.

Yuval: I assume that, for the foreseeable future, this is still going to need a lot of human supervision. For instance, how would the quantum computer know when the piece ends and how to end it? Usually, you might end it on a dominant note or something else. How would that work, for instance?

Eduardo: I think it's possible to automate this as well. I think these are, as we say, the cherry on the cake, so to speak. After we've got the core thing working, I believe the details can be honed and incorporated into the formalisms. So these are things where I need to liaise very closely with Brian so that we keep the beauty of the model.

But, you know, use what we've got and try to scale it up to a larger number of qubits and so on.

Yuval: There was a book that was quite popular several years ago called *Sleeping with Schubert*, and it was about a lady who became obsessed. Something got into her and she started thinking like Schubert, and then, of course, she also finished the Unfinished Symphony.

How far in the future do you think you'll be able to automatically finish the Unfinished?

Eduardo: I think we more or less know how to do this, because there are examples of composers who programmed classic AI to do this sort of thing.

So we've got the Mahler symphony finished. We've got the Schubert symphony finished. We have a Beethoven symphony finished. Of course, these outputs are not, I would say, very realistic, in the sense that it's easy to spot that they were done by a machine. But the know-how for doing this, I think, is already out there.

What we need to do is perfect this, and because I am so enthusiastic about the quantum computing world, I believe these new ways of thinking about programming AI will lead to the sort of thing you just mentioned. So maybe the dream may be quite realistic pretty soon.

Yuval: Brian, from a quantum perspective, what did you learn doing this project? I'm guessing you learned a lot about music and composition, but how did you become a better quantum circuit designer with this project?

Brian: I have to go back about two years. Well, I like this story. I was reading a paper a day off of arXiv.

I was getting most of the suggestions from Twitter. Somehow I discovered a paper from Professor Miranda called "Quantum Computer: Hello, Music!", and it blew my mind, and we've worked together and it still blows my mind. But the key difference with quantum music, and I would compare it to quantum gaming, where you generate some random numbers and use them during the game, or generate some random numbers and generate terrain…

But the interesting thing about music is it has to sound good, and that's very subjective, and I have no musical talent whatsoever. So when we're playing back these tunes, sometimes it's kind of catchy, it sounds kind of interesting. The original is in there, but there's that little bit of variety that sounds good.

It just blew my mind, and as we were working on it, and still as we chat now, I think about all the other possibilities. We started off very simple because Professor Miranda talked about getting the core working. So we started very simple with that, and then we switched to notes and pitches, and it was seamless, an easy transition.

So it blows my mind further: what can we do next with it? What's the potential? I don't know where I found his paper. I'm thrilled I found it. I put it in my first book, *Dungeons and Qubits*. I had no real expectation that this much adventure would come out of it. But I'd say that's the key thing I learned.

Not only is it possible to create something that's subjective and good, but we haven't even started yet. There's so much more. I'm waiting for more to come out of Professor Miranda's mind just so I can work with it and see what it can do, and then have him play it back and just blow my mind some more.

Yuval: Would a larger quantum computer, a better quantum computer, help you, or are you perfectly fine with just a handful of qubits?

Brian: Well, a handful of qubits is okay at this point because we worked out a compression system. When we switched to notes and pitches, it was a nine-bit binary string. But we don't need nine qubits.

We were able to compress that down to, I think, four qubits. Four qubits, five qubits. The advantage of better-quality qubits, though, would be in the error handling. So if we have A, B, C, we can generate D or E. The problem is if noise makes us generate an F, we don't have a B, C, F in the transition matrix, and that's a problem.

So there was a lot of error handling that we had to throw at it. You've got the simulator, which is pretty precise, but it's not as interesting. Then you've got real qubits singing far better than I could possibly do. But you get that B, C, F, and we just can't work with that. I'd really prefer qubits that, if we want 70% and 30%, are at least pretty close, with some weird quantum fluctuations, some quantum variety.

But it still has to follow the rules or you get error messages. So better qubits are better. But then it's exponential. Five qubits gives us 32 different combinations, and each additional qubit doubles that, so you would need a very long, complex tune to really run out, or to need additional qubits.

Yuval: And how long does the process take? If you take a Bach cello sonata and take it through the learning phase, how long does it take to start generating music from that?

Brian: With the size of the pieces that we used, the learning phase was practically instantaneous. The slow part was running on real quantum hardware, but as soon as we were done with it, I started thinking of new ideas for how we can speed that up.

So I'm really looking forward to the next round and overhauling pieces of it like that. We can speed up the quantum part. But most of the classical parts are practically instantaneous.

Eduardo: It's pretty quick. It depends on what we do with it. For example, I am interested in real-time interaction, right?

Real time is not possible yet because of the connection to the cloud and blah, blah, blah. But in a hypothetical world, what I want to do is be playing my instrument and open a window for the quantum machine to listen to what I'm playing, and then play responses or accompaniments back to me, and so on.

So I would like to jam with a quantum machine. For this purpose, what we've got is pretty instantaneous. We would do that in a second. It is real time. Now, the other vein of research would be to train the system with large amounts of information.

Let's say, take the repertoire of John Coltrane, put it in the system, and see if the system can generate melodies in the style of John Coltrane, for example. That would then depend on how many files we use to train the system and so on and so forth.

But I would say what we are doing is much faster than, let's say, deep neural networks and things like that. That's the beauty of what we've got. It's so neat, so simple, and so smart that it's almost unbelievable that we came up with it.

I think Brian was also very surprised by such a simple, elegant way of encoding these transitions in the amplitudes of the quantum states. It's as natural as it can be.

Yuval: Listening to you guys, it sounds like this is just the start, and there's lots more work and additional ideas going on.

And hopefully we'll be able to speak again in the future with the Qubit Choir, or the Qubit Orchestra, or the Qubit Band.

Eduardo: Most definitely. Yeah, most definitely. So watch this space.

Yuval: How can people get in touch with both of you to learn more about your work?

Eduardo: The best thing to do is to go to arXiv and have a look at the paper we posted there.

It's called "Teaching Qubits to Sing: Mission Impossible." Our emails are on the paper, so just email us, and we'll be happy to interact with people. And most importantly, suggestions and ideas, things people spot in the paper that we had not thought of before, would be very welcome.

Yuval: Excellent. So, Eduardo, Brian, thank you so much for joining me today.

Eduardo: It's a pleasure.