Down the VRabbit Hole With Merlin Sunley - Interview with Sam Willis (dBs Institute)

Down the VRabbit Hole With Merlin Sunley By Sam Willis (Originally published April 5, 2022 - https://insider.dbsinstitute.ac.uk/down-the-vrabbit-hole-with-merlin-sunley)

Driven by decades-long passions for sound and gaming, dBs student Merlin Sunley is working on a mind-bending virtual reality experience that explores the philosophical contradictions within the medium. We caught up with him to talk about VR, his ongoing involvement in a transatlantic game development project and the need for a love of learning. Merlin became interested in gaming when he was first exposed to a Sega Master System “many, many years ago,” and his affection for music is equally deep-rooted. Touring the UK and Europe as a drum and bass DJ and psytrance project in the early 2000s and early 2010s respectfully, Merlin became increasingly engrossed by the complexities and versatility of sound design; “digging into stuff like Pure Data and Max MSP and all these really exotic synthesis methods.” It was when he had “hit a wall” with what he could learn on his own that he applied to dBs’ Sound for Game & Apps degree, with a desire to learn skills used at the frontiers between sound and technology, as well as the “tangential knowledge that you need to break into that industry.”

Now, in the final phase of his third year, Merlin has embraced that initial instinct to operate at the cutting edge with an Honours Project that embeds malleable sound into a virtual environment.

Virtual reality: spectacular and sinister

“I'm currently working on a prototype of a VR sound art project,” says Merlin, “It explores themes of legitimacy and VR as an artistic medium. Exploring how algorithms, which govern our daily lives, have real-world ramifications.

“It's an interactive network of sound-generating nodes, which can freely move around inside VR, and be manipulated. You can hold them and move them around your head… It's basically loads of drones made from different data-generated audio; stuff like taking the HTML from a compromised phishing website and importing it into Audacity as a txt file converted into audio. It’s really mad. There are also recordings taken from CCTV cameras that have open IP addresses and are exposed to the internet. You can actually just go and take audio from an open random remote CCTV camera. There are data sets from botnet activity and malicious computer network activity. These are used to drive the parameters of custom-built granular synthesisers. All sorts of stuff.”

As well as being an impressive art piece with a dazzling breadth of imagination, Merlin’s VR project touches on the philosophical questions that surround VR and the ethics of its application. As a comparatively new medium, that has only recently been integrated into our daily lives, the lingering question around virtual reality is whether it will prove to be a boon or a burden for those who use it and, eventually, us all.

“Some of the experiences [in VR] are just completely mind-blowing,” says Merlin, “But it's also really contradictory. For example, Oculus is owned by Facebook. It's constantly tracking your facial movements and eye movements. It's scanning the room you're in and feeding all of that data back to Facebook, which is then being monetized and fed back to us as algorithmically-generated advertising in evermore cryptic and worrying ways. So it's that two-sided thing; the fact that this is a completely mind-blowing experience, but it's also this kind of sinister thing as well... I think that VR perfectly encapsulates those [contradictions]. It's this incredible technology, with incredible potential for both really good things, and really bad things simultaneously. I find that really interesting.”

Once completed, Merlin will seek funding opportunities to expand and improve the project, exploring whether it could be applicable within augmented reality in a “white cube gallery setting.” In the meantime, he’s joined a transatlantic game development team as the lead for the project’s audio design and integration.

In-progress and constantly-iterating

Under development in Texas and Colorado primarily, but with an international network of developers and artists, Netc0de is a game that pays homage to Anime Arena Fighters; designed by a team who are in love with the genre and built to compete with traditional Western fighter games. Merlin’s role in the development of the game is as its audio lead, designing and implementing all of the audio systems and creating the overall direction of the sound.

“It's been a really steep learning curve,” says Merlin, “from working on demo projects, where you kind of have loads of tutorials, which dive deep into how all the systems work, to being given free rein to work on an in-progress and constantly-iterating project. I just had to take the reins with it really and start thinking like more like a game developer first and an audio person second.”

Merlin became involved with the project while networking with developers on game development Reddit subs. Impressed with his showreel and hyper-real sci-fi sound design work, it was the fact that he was “studying for a game audio degree that really sort of sealed it for me.”

“I got invited to their discord and yeah, they're busy bees! Every single day it's just constant. They have a team of about 20 people working on it all the time. It’s shaping up. In the first week of April, there's going to be a meeting with multiple publishers and hopefully, we can get funding. It's been almost two years of pre-production… Making games is really hard and it takes a long time.”

The challenging nature of creating games, combined with an international team and the obstacles that come with operating across borders, crystalised a pre-existing conviction that independent learning is an essential discipline. Just like the project he is working on, Merlin has found that he must also be “in-progress and constantly-iterating.”

“Only once I've exhausted every other option, will I go to the devs and ask them to implement something for me, because those guys are super, super busy. I'm trying to be as self-contained as possible,” says Merlin, “I'm constantly really trying to learn game dev. As I said, you're a game developer first, and an audio person second. I can knock out assets all day, every day. It’s second nature to me because I've been doing music and sound design for so long. This is a whole other thing: ‘how will the moving parts interact?’ It's stuff that, as audio people, we don't really necessarily think about. “The things that you do in the first and second year prepared me really well, for ‘real life’ game dev work. Things like building an asset pipeline; high-level planning stuff, all the way down to getting everything into the game, tracking the assets - we covered all of it. At the time, you think, ‘Why am I building a spreadsheet?’ It helped me to come across as a professional. If I hadn't had the experience through learning with Jasz, the first thing I would have probably been inclined to do is just start making loads of impact sounds, but the first thing you do is, build an asset tracking database.” That lust for learning and motivation to seek out answers to questions yourself helped Merlin settle into a project that requires quick thinking, adaptability and flexibility. The principle of hunting down knowledge is something that he would recommend to any dBs students who want to succeed in the creative industries.

“You have to be able to learn independently,” he says, “ get a taste for learning on your own. Don't let people spoon feed you. You've got to go and search out that information. So often I'll be on Reddit and people come on and they just ask the most basic questions. This sounds really harsh, but if you can't go to the Wwise API or the FMOD API and search for that information, you are not in the right business, basically. It is really very hard. If you don't have a real strong passion for learning, then you're gonna have a tough time because there's not always going to be someone there to hold your hand… Develop a passion for learning and always be constantly updating yourself with new skills and upgrading your knowledge as much as possible. Read as many books as possible…It's harsh, but it's true; especially in this industry, I think. There are a thousand other people who will willingly do the work to get the job.”