728 x 90

Culture Crossover: beatboxing with a digital twin in We Speak Music | Emerging tech & innovation

Culture Crossover: beatboxing with a digital twin in We Speak Music | Emerging tech & innovation

A new film charts the relationship between people and technology through the lens of the human voice, and explores how humans can augment themselves with machine learning to realise hidden creative potential, but also how people can still innovate with the biological tools that we are born with. It’s all anchored by the journey of

A new film charts the relationship between people and technology through the lens of the human voice, and explores how humans can augment themselves with machine learning to realise hidden creative potential, but also how people can still innovate with the biological tools that we are born with.

It’s all anchored by the journey of beatboxer and visual artist Harry Yeff, AKA Reeps One, who travels the world in a series of vignettes, leading up to a final artistic performance in the last act.

Harry Yeff in the Anechoic Chamber, credit: LonelyLeap
Harry Yeff in the Anechoic Chamber, credit: LonelyLeap

We Speak Music is written and presented in collaboration with Bell Labs, the nearly 100-year-old industrial research facility now owned by Nokia and named after Alexander Graham Bell. Its various parts are described by Yeff to Techworld as individual “curiosity pieces” designed to provoke more questions than they answer. They vary from a crowdsourced exploration of the global beatboxing subculture, to a trip to the Lavelle School for the Blind, the Bronx school for students who are visually impaired or have additional disabilities.

The film culminates with a performance filmed at the Bell Labs where Yeff participates in a beatboxing collaboration of sorts with his digital twin, which was trained with a neural network that had been fed hundreds of recordings of his own voice.

To achieve this, he collaborated with generative music and ‘Dadabot’ digital artist CJ Carr. He performs the final piece in the Bell Labs anechoic chamber, an echoless room invented by Murray Hill that’s padded with wedge-shaped absorbers to soak up acoustic energy, and was once cited in the Guinness Book of World Records as the quietest room in the world. 

There are “two sides of the spectrum” for the film, says Yeff, both technological but also “human innovation”.

“How we can still find new things with the bodies that we have, is such a bizarre idea, the fact that we can still innovate with something like the voice, which is as old as humanity himself,” he adds. “You’d think that would have been covered by now, the spectrum of what the human voice can do should have been established, but there are still new things happening.”

The Lavelle school segment took a lot of groundwork, and the film team spent a year to capture the story.

“It’s just teaching young people to speak, and the fact that people, through experimenting in new ways, can find new ways of reconnecting with their bodies and their own voices… that is a really important thing,” Yeff says. “That goes beyond the music or artistic expression I’m interested in: it’s a huge, objectively participatory context that can benefit everyone, people thinking about their voices, and understanding their voices, is such a huge, fundamental part of everyone’s day to day.”

Then there are the “tech-focused musings” on innovation, and “how the two can connect in some way”, such as another segment that sees Yeff going beatboxing during an MRI scan, with an X-ray animation revealing the human anatomy at play.

“Because I’ve been a part of a number of different studies, the last one I did I bartered after I completed the scanning that I could use the MRI as a medium – so I got a self portrait of my head my hands over my eyes, and actually have that 3D data,” Yeff says. “I can zoom in and out of my physical head. I love that cross section of when you take hyper-functional tools and then you’re able to use them in a lateral way.

“It’s not always as easily tangible, because you could argue: what is the purpose of art? Which is a huge debate especially in academia and research, but it’s interesting that the way that experimentation and using artists and engineers and academics leads to this mutation of different things. That is where I kind of thrive, and I enjoy that so much, as it tends to lead to inspiration on both sides of the spectrum.”

What the viewer doesn’t see is the intense amount of work that went into training the neural network, and Yeff experimenting with it to understand how it works. He says this was deliberate, because that segment of the film was about the end result – the final piece of art – but that it is important to understand the processes behind it, to demystify artificial intelligence, and perhaps remove it from the dystopian tropes that these technologies tend to be shrouded in.

“People try and make it this other, but it’s just a tool – a human-guided tool – and you have to learn and augment with it, and that takes time,” he tells Techworld, adding that it would take two days to process all of these recordings on the Amazon cloud and to get the response back.

Watching Yeff react to the first neural network experiment results leads the viewer to witness a pivotal tipping point from the unreality of hearing yourself back, filtered through a machine, into actively wanting to collaborate with it.

“The main point is when I played the audio of the AI voice, which was a sort of second version of myself, the way people reacted to it was a lot like the way people react to me when I use my voice in an extreme way,” he says. “‘I don’t understand this, I don’t know how that’s happening’.

“It’s a fusion of fear and interest … when I first heard the voice I was immediately inspired, I say it in the film, that there’s a thousand pianos out there that a thousand people can play, but every single person’s voice is like a fingerprint, it’s unique to ourselves.

“But for the first time I had this capacity to interact with myself in a brand new way, and I think there’s something really beautiful about that.”

He says that a separate neural network project allowed him to categorise all the different sounds that he verbalises in a single performance.

The result was that there were 150 variations on a kick drum in a ten-minute piece.

“There is no way I would have known that without the intense listening and observation capacity of something like AI,” he adds. “I think that’s a really beautiful thing, but there tends to be science fiction narratives and damaging narratives where people think these tools are more than tools, but they’re not, they’re just things that can empower us to interact with ourselves in new ways and become better human beings.

“That’s how I felt about this voice – I felt, wow this is a new gateway to inspiration and I wonder if people saw it like that more they would engage with it more, and it would actually push the technology in the right directions instead of it being non-accessible or this alien-type feeling people feel.”

Navigating sponsored content in an authentic way can sometimes be tricky, but the series of film is light on branding and the immediate impression is that Yeff had creative control over the documentary, which he confirms, adding that the partnership emerged organically through Bell Labs’ Experiments in Arts and Technology programme.

“They have always been interested in the EAT programme, it’s there for experimentation,” Yeff says. “They have a real fire to break the rules and not do things in a normal way, in the pursuit of new ideas. That was at the very beginning of this process – obviously it’s important that people know about Bell Labs and I’m so proud to work with them, but it’s a sincere connection, it wasn’t like they came in as a sponsor and had to bend it for anything other than the pursuit of new ideas.”

Next, Yeff is working with other technologists to explore how emerging fields might aid his art. With every crest of innovation, he says, there is an equal opposite wave of doubt and fear, but engaging with these technologies as they emerge can help elucidate them.

He is meeting “one of the world leaders in 5G” shortly to learn about it, and see what the next wave of mobile communications can offer artists, to “find out what are the expert ideas in why it’s negative, and why is it positive, instead of listening to the rhetoric of the media”.

“You take something like 5G and I’m really interested in what that offers, what would compositions be like using 5G,” he says. “What does 5G actually offer artists and human beings to connect to each other more?

“I think that’s a good function of an artist in 2019, that they’re not afraid to engage with this type of technology so that people have something to latch onto, and it’s not just in the commercial or academic realm. I’m sure there will be criticisms for engaging with a technology like that, but then, why be afraid?”

He is also meeting with a designer who runs the DNAblock firm, which specialises in “digital human beings”.

“He’s been 3D-scanning actors, and models, and creating a blockchain digital identity to try to protect these people from being copied and incorporated in adverts and TV and things like that,” he says. “And again, that creeps people out, but they’re very exciting and very real aspects of technology that are coming whether people like it or not, so it’s important for artists to engage with them, so people can engage with them.”

The films are available to watch online here, with the final, sixth piece due for release on Saturday 27 April.


Source link

Susan E. Lopez

Posts Carousel

Leave a Comment

Your email address will not be published. Required fields are marked with *

Latest Posts

Top Authors

Most Commented

Featured Videos