How Do You Audio Transcribe an Entire Video Game?

"At night on a river, the Viking longships emerge from within thick fog. They land on a sandy shore, the blonde Viking's eyes shift upwards, the sky begins to glow. A mass of fiery arrows rain down, the Vikings raise wooden shields; a flaming arrow strikes a man's neck."

The quote above is read by a woman during the climax of the audio described reveal trailer for Assassin's Creed: Valhalla. Audio description, as a concept, has been a fixture in movies and TV shows for decades -- the goal is to narrate the subtle visual details, wordless gestures, and brisk action sequences on screen as if they were happening in a book for those who are either blind or visually impaired. But the concept never made serious landfall in gaming, at least until 2020. And so, the woman details the medieval garb, acrobatic gore, and brutal weaponry during a skirmish between the Knights Templar and the unruly band of vikings. The trailer was produced by Descriptive Audio Works, which has been bringing the visually impaired community to the entertainment industry for years. Now, they've turned their attention to video games, ensuring that everyone, no matter how well they can see, gets a feel for that misty morning in the English countryside. [widget path="global/article/imagegallery" parameters="albumSlug=xbox-series-s-and-series-x-comparison-photos&captions=true"] "As a company we've been doing this work for linear media entertainment for years. We've done a lot of work with Netflix and various other broadcasters. So there were some initial conversations about, 'How would this look in a gaming environment?'" says Rhys Lloyd, General Manager of the company. "By and large the feeling was that there was definitely an appetite for it, but nobody knew how to approach it. Trailers were the easiest entry point, conceptually." Lloyd's hands have been all over Ubisoft's 2020 lineup. Alongside Valhalla, Descriptive Video Works has also supplied descriptive narration for the trailers of the forthcoming Far Cry 6, and the Ghost Recon Breakpoint seasonal event, Resistance. The assignments came together quickly. "Out of the blue, they were like, 'Can you do this [for Assassin's Creed?] It's going to be a quick turnaround,'" he remembers. "That's core to our ethos. We were like, 'Yes we can swarm this and get this done.'" Ubisoft sent over the video and some additional context for the specific points of interest the development team wanted to highlight in the trailer, and Descriptive Video Works got to work writing a script that was both uncompromisingly immersive, and smartly condensed. That is the challenge of description copy, explains Lloyd, you want to be as vivid as possible, without overpowering the music, the mood, and the other voice actors. [widget path="global/article/imagegallery" parameters="albumSlug=confirmed-xbox-series-x-games&captions=true"] "It's all about supplementing. The other elements play such a key role in trailers and games in general, and you want to make sure you're not eliminating that. You just want to be working alongside it," he continues. "One of the things that was really refreshing to hear when the trailer came out was that people didn't even realize that it was an audio described trailer. People just thought that that was the trailer. We never want to be jarring." Ubisoft's description initiatives are the latest manifestation of the game's industry long-overdue reckoning with accessibility. It is strange to consider that for decades, publishers scarcely paid mind to the reality that there are many avid gamers who do not see very well, or are hard of hearing, or have difficulty manipulating a joystick. Finally, disabled communities have won some attention. The Last of Us II, undoubtedly the biggest release of 2020, was touted as the most "accessible game of all time," sporting a huge swathe of quality-of-life options like a high-contrast mode and a magnification tool. [poilib element="quoteBox" parameters="excerpt=Ubisoft's%20description%20initiatives%20are%20the%20latest%20manifestation%20of%20the%20game's%20industry%20long-overdue%20reckoning%20with%20accessibility."] Microsoft released its adaptive controller in 2018, which translated the knottiness of the standard gamepad to an approachable, easily mappable tray configuration that can rest easily in a lap or a table. Audio-described video game trailers are the tip of the iceberg for what's truly possible for visually impaired gamers, but the community is already showing appreciation that they're finally being heard. Steve Saylor, a prominent blind YouTuber and an advocate for accessibility options in games, posted a reaction clip to Valhalla's described trailer. "I love this so freaking much, oh my god," he says, giddy and almost out of breath, at the conclusion. "Video games are cool, man." [widget path="global/article/imagegallery" parameters="albumSlug=confirmed-playstation-5-games&captions=true"] "It felt like visually impaired gamers had found some recognition, that we're a part of the gaming community," he says, in an interview with IGN. "It was cool to see that we were acknowledged and that they did it really well." This leads us to Descriptive Video Works' most daring ambition. Annotating marketing material is one thing, but creating a full audio description for a  video game is quite another. Games are dynamic. Players wander around open worlds at their own pace; they pick flowers, smelt ore, and discover Korok seeds. No two combat encounters are exactly the same. It's an onerous job to fill in the details of a two-hour movie or a season of television for visually impaired folks, but it's at least easy to storyboard and conceptualize. Doing the same for say, a 40 hour Assassin's Creed campaign plus sidequests and collectables would require reinventing the wheel entirely. Lloyd does have some experience with this. Descriptive Video Works has worked on live events coverage before, where a narrator describes the action at, say, the Olympics without missing a beat. A similar improvisational dynamism would be necessary to bring their craft to interactive entertainment. The way Lloyd sees it, there are certain elements of gameplay that the company could start working on right now. "There are things that are set in games. Cutscenes, in-game cinematics, established links between scenes. Or even menus screens. Text to speech can tell you some things about what's on-screen, but if you're choosing an outfit, a describer can give you more information," he explains. "We'll see those things sooner rather than later." But for that deep, seamless immersion? The idea that an audio describer could read and react to everything Ellie does as she skulks around Seattle? That would require an inordinate amount of time, effort, and problem-solving. Descriptive Video Works is a company that's used to getting a call from a client at the last minute. If they were to truly imbue a new game with a harmonious A.I. narrator for the blind, that would require their service to be a core priority for the developer -- long before they started generating enough assets for a trailer. [poilib element="quoteBox" parameters="excerpt=The%20idea%20that%20an%20audio%20describer%20could%20read%20and%20react%20to%20everything%20Ellie%20does%20as%20she%20skulks%20around%20Seattle%3F%20That%20would%20require%20an%20inordinate%20amount%20of%20time%2C%20effort%2C%20and%20problem-solving."] "We aren't game programmers. We'd need to work hand-in-hand with game programmers to build something that's truly immersive, and object-based, that'd be a joint effort," says Lloyd. "Normally we're brought on at the very end. You send us the final picture, we describe it, and you add it to your audio drop-down menu." Surprisingly, says Lloyd, he's had some promising conversations about taking on a project like that. If those discussions get off the ground, and Descriptive Video Works embarks on the first comprehensively annotated video game, we'll likely not see the results for another few years. There are still so many questions to answer, so many problems to solve. But the rapid advances in accessibility have been one of the most encouraging trends in the industry. Disabled gamers have every right to feel optimistic. "We're at the point where people are aware of the human impact of accessibility, what we now push for is to have developers improve on what has been created," says Saylor. "If they're able to jump on the ball now, then years down the road, it'll be a lot easier for developers to add those options because they'll already have had so much experience doing it." [poilib element="accentDivider"] Luke Winkie is a writer and former pizza maker in Brooklyn. Follow him on Twitter.

source https://www.ign.com/articles/how-do-you-audio-transcribe-an-entire-video-game