Takahiro Fujikawa Keiichiro Shibuya is a gifted musician who has continued to create fresh sounds by crossing different boundaries and evolving. He has been getting good feedback for ATAK024 Midnight Swan, his latest soundtrack album to the film Midnight Swan, and is ready to show the world his highly anticipated android opera, Super Angels, this summer. This series, “Massive Life Flow,” explores his mindset and what he envisions for the future. In our third installment, we followed him to Kunitachi College of Music, where he went to work with associate professor Shintaro Imai on Alter3’s new voice. What technology and thought process did he implement to meet Shibuya’s demands? We break down the factors that make this endeavor groundbreaking while reporting on their journey.
Increasing Alter3’s information content, so it could sing alongside a professional opera singer
One fall day in 2020, Keiichiro Shibuya visited Kunitachi College of Music. The purpose? To create android Alter3’s new voice for his opera piece, Super Angels, showing this summer. He had been using a machine-made voice, but he felt like it wasn’t enough. I asked him why.
“In Super Angels, Daichi Fujiki, a world-renowned countertenor singer, plays the part of the protagonist. Because Alter3 is an android, using a computer-made voice was the obvious choice. I assumed it would be interesting to contrast it with a professional human singer. But my opinion changed altogether when we had rehearsals at the New National Theatre, Tokyo, on August 23rd, 2020. With [Kazushi] Ono-san [Artistic Director of the New National Theatre, Tokyo] as the conductor, I played the piano as Fujiki-san and Alter3 sang along. That was the first time Alter3 sang with a human being. I realized Alter3 didn’t have the information content that Fujiki-san possessed in his voice. I realized something had to be done to [Alter3’s voice].”
As a result of him confiding in his collaborator, associate professor at Kunitachi College of Music Shintaro Imai, he decided to try out a different method: synthesizing a human voice. To achieve Alter3’s new voice, Imai used a particular program created via Max, a programming language for audio control and synthesis invented by himself. The following is his explanation of its technical traits and aims.
“The most difficult thing about creating a voice from scratch with formant synthesis is expressing ‘instability.’ It’s pretty easy to make something with a solid pitch, but it’s tough to make something with a lot of ‘noise,’ in the broadest sense of the term. I feel like elements of noise play an integral part in Shibuya-san’s music. Many people praise his work for its clear sound, but that clarity exists because it has noise. That’s what I wanted to keep with Alter3’s voice.
“Vocaloid [a popular singing synthesizer software] creates a voice by putting together fragments of sounds. But the thing is, it gets rid of noise right at the beginning; it ultimately creates a voice that’s too clean. This time around, I recorded speaking voices and stretched that, instead of breaking it down into fragments. By changing the pitch, I could then create a melody. This is how I preserve noise and the natural instability of human voices.”
For their first-ever test run, Shibuya recorded his voice. After about an hour, Imai took that data and converted it into a new one that could sing or perform a melody. This voice was just temporary, made so that they could check how it sounded. Nonetheless, the voice Shibuya played with his keyboard convinced me of Alter3’s evolution. It maintained a human-like texture — that “noise” — while having an artificial feel. In other words, it sounded like something right between a human and a machine: an original voice appropriate for an android.
In pursuit of the best quality possible, Imai continues to adjust the program today. I can’t wait to see what sort of voice will come out for the first performance of Super Angels this summer.
How did Alter3’s movements become more melodic?
Incidentally, Imai works not only on developing Alter3’s voice; he also works on its movements. He incorporated a different technological program from the previous Alter series. Here, Shibuya gives some background on how the Alter android became more melodic.
“Alter2 wasn’t initially made for music. So, when I had the first performance of Scary Beauty at the National Museum of Emerging Science and Innovation in 2018, it was hard because the android wasn’t moving smoothly right until showtime. One day, Alter2 broke, perhaps because I had used it too much despite it not being a robot specifically designed for music. When I was thinking about what to do, mixi Inc. showed an interest. That made it possible for us to develop a new android tailored to music: Alter3.
“Later on, the University of Tokyo and Kunitachi College of Music started a collaborative research project regarding androids and music. Imai-san was a part of the research team, and that’s how we started working together closely. He has a distinct approach, so I get very inspired by working with him. Right now, he’s putting so much effort into Alter3’s vocals. Things have dramatically evolved because of his involvement, including improvisations.”
What technology did Imai apply to achieve Alter3’s melodic movements? According to Imai, he got the idea from how synthesizers make sounds.
“At first, I couldn’t even imagine how I could improve the movements of Alter3. I went back and forth in my head, and after I shifted my perspective to ‘combine music and movement,’ I wondered if I could apply the way synthesizers make sound onto Alter3’s movements. Alter3 has around 40 joints, and so I tried to allot three LFOs (often used to create a rhythmic pulse in synthesizers, a Low Frequency Oscillator is an oscillator that generates low-frequency soundwaves) per joint. I synchronized the tempo of the music and set the pace of the oscillator. I adjusted the three LFOs in a way that would allow me to change how it mixes. By working on the music at the same time, I was able to create complex movements.”
The desire to subvert the typical western notion of opera
Thus, Alter3 has successfully unified music and movement. However, there’s more; the android can sing and dance impromptu. Imai emphasizes the innovativeness of this aspect and Super Angels itself:
“I presume operas with robots have existed before, but such robots would only move a certain way for a certain amount of time. In Super Angels, Alter3 can match the beat of the music and perform on the spot. In this manner, I think you can say this is an opera piece like no other.”
So, this is a Robot Opera. In A Brief History of New Music by Hans Ulrich Obrist (Jrp|Ringier Kunstverlag Ag, 2014), Iannis Xenakis talks about how he imagined an opera in which robots perform, but because of various reasons, he couldn’t actualize it. The Greek-French composer left his mark on contemporary music and electronic music and sadly passed away right when the 21st century started. I can’t help but picture an alternate timeline where this maestro could see Alter3 perform on the spot.
In any case, there’s no doubt that Super Angels is going to raise questions against the definition of “opera.” What made Shibuya, a musician who’s been creating innovative electronic music and sound installations, want to produce an opera piece, something that’s traditionally western? Shibuya reflects on the initial catalyst that led him to opera.
“THE END [a Vocaloid opera, shown for the first time in 2012] was my first-ever opera piece. That came about after YCAM asked if I could make something. If you look at my career until that point, you’ll probably picture my installation work. But at the time, there was a sudden increase in installations with a strong marketing and entertainment feel, so I didn’t feel like doing that sort of thing (laughs). Then, I wondered, ‘If I were to make something that represents me fully, what would I make?’ I also thought about what advantages I had compared to other artists. And I concluded that perhaps the opera format might work. Of course, I wasn’t planning on making a normal one. I thought about how I could break something that’s typically regarded as western. It wasn’t about doing something completely different from western music because I’m Japanese. I want to stand on the same ground as them and then strike back. I also feel it’s important not to do the same thing as the preceding generation.
“I’ve always felt it was a bit strange to see people sing Italian opera, like the style of Bel canto, in Japanese. A ‘Japanese music’ approach doesn’t feel right. That’s why I turned my attention to Vocaloid and androids. I think it’s efficient because it’s contemporary, and it shows that European opera doesn’t necessarily always have to have human beings at the center.
“I’m also trying to compose in a way that’s never been done before in orchestra or opera. For instance, I sliced the songs from the first half of the act into 0.5 seconds or one second. I then changed the order around and reconstructed them into new songs in the latter half of the act. This remix-ish method is very detailed. And once a more linear countertenor voice goes on top of that, it creates this incredible sonic experience. It’s as if it interweaves different passages of time. This style of composition is something that could only be made with computers.”
A sacred experience where a piece of work that belongs to no one unifies everyone
The opera piece is being honed day by day towards its first show in summer. As such, Super Angels continues to evolve and expand. As Imai’s assistant, Kairi Nagashima works on building different systems and operating them. One day in August, when they were having rehearsals at the New National Theatre, Tokyo, he felt convinced of their progress:
“During rehearsals at the New National Theatre, Tokyo, we got Alter3 up and running, and as it started singing with Fujiki-san, the countertenor singer, I felt like something unique was born right there. The ensemble got better and better, as Alter3 and Fujiki-san’s relationship formed. It surpassed the context of music. It was like this spectacular experiment about the relationship between AI/robots and people. The experience was so exciting.”
The only thing that continues to evolve isn’t the performer, however. Shibuya explains that The New National Theatre, Tokyo is more than just a venue. The creative-minded staff there are keeping busy to ensure the quality of Super Angels.
“The staff of the New National Theatre, Tokyo, is amazing. They’re excellent at what they do and have an inquisitive spirit. They’re doing everything they can to make Super Angels as good as it can be. When production is this big, so many people from different backgrounds come together, such as those working in technology, art, sound, and orchestra. This piece doesn’t belong to anyone anymore. I, too, am merely one part of it. The time and experience used to march towards one collective goal are sacred. It’s making me grow in a big way. This sort of experience seldom happens.”
Many professionals have come together to build and refine the unprecedented opera that is Super Angels. Nobody can foresee its final form: not even Shibuya himself.
Keiichiro Shibuya graduated from Tokyo University of the Arts with a B.A. in Music Composition. In 2002, he founded the music label, ATAK. His diverse soundscape covers areas such as cutting-edge electronic music, piano solos, opera, soundtrack music, sound installation, and so forth. He released a Vocaloid opera starring Hatsune Miku, which comprised of no people, called THE END, in 2012. The opera was shown at Théâtre du Châtelet and other places around the world. In 2018, he released Scary Beauty, an android opera conducted by an AI-operated humanoid android that sings along. This was shown in Japan, Europe, and UAE. In September of 2019, Keiichiro then presented “Heavy Requiem,” a marriage between Buddhist music and chants and electronic music, at Ars Electronica in Austria. He explores themes of humanity and technology and the border between life and death with his work. His new opera piece, Super Angels is scheduled to be released in August 2021 at New National Theatre, Tokyo.
http://atak.jp
Shintaro Imai is a “computer” musician. He creates music by trimming and correcting detailed movements in sounds and objects. After his studies at Kunitachi College of Music and IRCAM in Paris, he received a grant from the Japanese Agency for Cultural Affairs and worked as a researcher at ZKM in Germany from 2002 to 2003. In 2004, he was an artist-in-residence at DAAD Berlin and composed music at the Berlin Institute of Technology. From 2008, he worked as the director of music for the Bauhaus Stage Projects, Dessau. He released Figure in Movement in 2015. His awards include: the Residence Prize at International Electroacoustic Music Competition of Bourges, the First Prize and Special Prize for Young Composer at MUSICA NOVA International Electroacoustic Music Competition, Prize at the EARPLAY Composers Competition, and the First Prize at ZKM International Competition for Electroacoustic Music. His works have been performed at various international festivals. He is an associate professor at Kunitachi College of Music and a part-time lecturer at the University of Tokyo.
http://www.shintaroimai.com
AMEYA
Photographer and Director based in Tokyo, Japan.
Instagram: @itsameyab
Photography and Videography by AMEYA
Translation Lena Grace Suda
