Hello, and welcome to Protocol Entertainment, your guide to the business of the gaming and media industries. This Friday, we’re taking a closer look at the lessons Meta has learned from video calls on mobile and how they’re helping the company with its metaverse plans. Plus: what to read, watch and play this weekend.
What leaf blowers have to do with the metaverse
Millions of people use Meta’s Messenger and Instagram apps for video calls every single day. By doing so, they’re also helping the company build technology that will improve real-time communication in the metaverse, explained Meta’s remote presence product director, Connor Hayes, during our recent conversation.
Linking mobile and AR/VR devices is one aspect of this work. Meta introduced Messenger calls to its Quest headset in November, and Hayes stressed that enabling real-time communication across device classes was a big focus for his group. “For the first few versions of those devices, the social experiences that we build into them will be only as valuable as their ability to communicate outside of that device,” he said.
- That’s something many Quest owners will likely relate to: Even if you do have friends who also have VR headsets, they might not be using them at the same time as you. Having the ability to call someone on their phone’s Messenger app makes it a lot easier to connect.
- The same will also be true for AR hardware, Hayes said. “If I can get AR glasses on Day One and know that everyone I'm connected to on WhatsApp or Messenger is reachable by me on those devices, that just adds a baseline level of social value to that device.”
Improving cross-device communication could be key to success, even as these devices diverge further. This includes animating facial expressions of avatars, something that will be supported by face-tracking hardware on future VR headsets.
- Meta’s remote presence team has been working on ways to port that experience to lower-end devices by letting people select emotes to express themselves, Hayes said.
- “If someone's on a phone that might not have the same amount of computing power to run the face-tracking model that gives them the optimal level of expression, we could give them a version of the product that they can manually tune to say: ‘This is what it looks like when I smile,’” he said.
Cross-device challenges are nothing new for Meta. The company has enabled video calling on desktop and mobile for some time, and added calling to Portal smart displays and TVs in recent years.
- Initially, those efforts were fragmented across the organization. That’s why Hayes’ remote presence team was put in place in the summer of 2020 to build bridges across clients and devices.
- Part of that work has been dedicated to something Meta internally calls Project Halo. “It's effectively a new design system for the in-call experience that makes things consistent across all these surfaces,” Hayes said.
- Some of the non-VR challenges involved making sure that callers look good on a variety of screens, regardless of whether anyone on the call uses a screen with vertical or horizontal orientation — something that has helped the team think about how to represent callers in AR and VR.
- Another big technical challenge has been background noise. Working on ways to suppress that on mobile has also helped improve communication in VR.
- “If a leaf blower is going off next to me [during a traditional video call], you might reasonably assume there's a gardener,” Hayes said. “If we're at a Miley Cyrus concert in VR, and you hear a leaf blower behind me, that might just be a little more strange.”
Social norms can be just as important as those technical challenges, and having a massive mobile user base has helped Meta learn how we might want to talk to each other in the metaverse.
- “[As] knowledge workers that spend eight hours a day on video calls, we form our opinions about real-time communication based on a form of it that we are required to do, and paid to do,” Hayes said. “You're not supposed to talk over each other; it's rude to be unmuted and have background noise.”
- “In consumer video communication, those norms don't really exist in the same way,” he said. Instead, consumers often use video calls as a second screen while they watch TV or do other things.
- This has led to Meta introducing mini games, filters, co-viewing and other things to help people connect in a more playful way — something that’s also applicable to making metaverse interactions a lot less awkward.
“Those are all things that we can learn from [to] optimize the default experience for communication in VR,” Hayes said.
— Janko Roettgers
A MESSAGE FROM WORKPLACE FROM META
100% of C-suite staff surveyed by Workplace by Meta said that frontline workers were a strategic priority for their business in 2022, but nearly two in three of them said that keeping their frontline staff, who bear the brunt of the stresses of the workplace most acutely, had only become a priority since the pandemic hit.
TGIF: How to spend your weekend
“Bubble” — Netflix. Have you ever thought, “Wouldn’t it be neat if there was a parkour anime?” What if this series happened to be directed by the legendary TetsurÅ Araki of “Death Note” and “Attack on Titan” fame, and animated by the masterful Wit Studio? Well, fans of that — perhaps a bit offbeat — recipe are in luck: “Bubble,” on Netflix, is just that. Following this year’s “Belle,” a modern twist on Beauty and the Beast, the parkour-themed “Bubble” tackles Hans Christian Andersen’s “The Little Mermaid” to similarly clumsy but heartfelt effect. The film is best enjoyed as a visual feast of eye-popping color and fluid animation, featuring a post-apocalyptic Tokyo and a cast of characters using gravity-defying tricks to maneuver the cityscape.
“Better Call Saul” — AMC / AMC+. There’s no better time to tune in to a show like “Better Call Saul” than the final stretch. The “Breaking Bad” prequel’s sixth and final season kicked off on AMC last month, setting up the conclusion to the series after seven years. It’s hard to overstate the genius of Vince Gilligan’s crime drama, which is as much an antihero character study and drug cartel narrative as it is a subversive love letter to classic legal dramas. It blends procedural lawsuit plot beats with some of the best prequel storytelling on TV as it fills in the blanks of its predecessor's biggest mysteries, all shouldered by the performance of a lifetime from Bob Odenkirk.
Knotwords: I highlighted Zach Gage’s new word puzzle game Knotwords in our newsletter on Tuesday, and I’m doing it again here because it is simply that good. The game is a cross between anagram word-guessing games like Wordle (Gage said some of his game design was inspired by Josh Wardle’s creation) and classic crossword puzzles, though there’s a catch. In Knotwords, the only clues available to you are the limited selection of letters for any available group of spaces. Filling an empty puzzle grid with answers through logical tinkering and process of elimination is about as satisfying as these games can get, and I can’t recommend Knotwords enough for those looking to tack on a new daily obsession alongside their Wordle habit.
Fortnite — Apple App Store (again). I fell off Fortnite back in 2019, when I found the novelty of the game’s core battle royale mode had worn off, and, more importantly, I was having trouble competing with the scores of players more skilled than me. But I still found time to squeeze in games here and there, and to check out the nonstop flow of new seasonal content and events. I was especially disappointed when Epic and Apple’s legal feud resulted in the game getting the boot from the App Store. Now, thanks to a new deal with Microsoft, Fortnite is available on iOS devices again through Xbox Cloud Gaming. It’s a good opportunity to jump back into the game while also giving Microsoft’s cloud gaming platform a spin.
— Nick Statt
A MESSAGE FROM WORKPLACE FROM META
Businesses are starting to turn to workplace communication tools. Such tools enable frontline workers to feel more connected to the rest of their business, to raise concerns and to provide feedback on potential pain points or points of improvement. By bridging that divide, companies can unlock new savings and efficiencies, and build a business that can last for the long run.
Thoughts, questions, tips? Send them to entertainment@protocol.com. Enjoy your day, see you Tuesday.