Realistic AI avatars revolutionize expression in the metaverse. Industries must prioritize value-driven applications, while combatting the critical challenge of technology misuse.
The concept of avatars originates from Hindu mythology, where human incarnations of gods are referred to as avatars. In the present context, avatars are our digital representatives — an electronic image (static, animated, 2D, or 3D) — commonly used for multiplayer gaming, online communities, web forums, and metaverse environments.
Experiences in virtual reality (VR) and augmented reality (AR) are becoming increasingly immersive, but they don’t match the experience of a real conversation. Avatars can enable near to real interaction by adding facial expressions, eye contact, gestures, and posture that reflect one’s sincerity, mood, and intent. They also allow people and organizations to express themselves innovatively and build personas that represent themselves — or be someone entirely different. That said, avatars will be critical in the success of metaverse spaces.
From 2D to 3D/4D
Traditionally, digital avatars were simple 2D images to identify users on social media or in online chatrooms. The next step came with the arrival of 3D online environments such as Second Life, the early-noughties online world, which was the start of the metaverse in some way. Now, platforms such as VR Chat are letting users create their 3D avatars and experience an ever-expanding, virtual social universe comprising live concerts, fundraising events, fashion shows, auctions, sporting events, and other spaces for users to connect and play games. Further, researchers have already demonstrated the creation of 4D facial avatars that are modeled from a real person and can be made to look and act like their original counterparts.
Advancements in artificial intelligence (AI), spatial technology, and extended reality (XR) sensor tracking peripherals will capture users’ physical movements or gestures and project them realistically in virtual environments with minimal latency. The prevailing criticism of avatars is the absence of legs, as developers struggle to deal with occlusion (hiding or revealing avatars’ legs when there are other objects like a table in the view). However, evolving AI models and tracking sensor technologies are improving, and are expected to resolve these issues.
Realistic, full-body avatars will open the doors to innovative autonomous applications in customer service, social media, and training and learning. Notably, just as avatars can come close to representing people’s real selves, they can equally represent a persona entirely different from their real ones.
Beyond gaming
Avatars are widely used across the web and social platforms. However, gaming, social interactions, and customer support have marked the beginning of avatars. They are leveraging avatars in new spaces such as learning and education, product launches, hosting online events, official unified communications, and shopping.
Some of the current use cases are:
- Customer service: Avatars can either be controlled directly by a person for one-to-one interaction, or by chatbots enabling companies to handle many customer inquiries simultaneously.
- Corporate training: Organizations are replacing unpredictable and expensive human actors with avatars in video-based training content through synthetic videos. These avatars are computer-generated but look and sound realistic.
- Healthcare: Avatars are being used to teach social and interaction skills to children with autism.
- Media: News anchoring, interactions with panelists or public, live streaming, and animated stories are some of the areas where avatars are finding their use.
- Other virtual spaces: Avatars are getting deployed in environments from banks, real estate showrooms, car dealer showrooms, retail shops, and airports. Instead of real actors, companies can employ avatars to provide personalized guidance, walk through tours, and curated shopping experiences. Several businesses are also looking to provide a virtual collaboration environment where participants can discuss their offerings displayed in virtual spaces, provide reviews, and explore and analyze digitally recreated virtual environments and products.
But this is just the start: other applications are yet to be conceptualized, as the metaverse itself is fast evolving. In the future, an online education company could standardize learning by incorporating the best teaching traits in an AI avatar. AI avatars (such as those enabled by NVIDIA Omniverse) are capable of speech recognition, natural language understanding, computer vision, and facial animation to help people curate highly personalized avatars.
As a result, avatars will become extensions of our personal selves in digital worlds, which, when augmented with ethical AI, could enable us to do what seems unimaginable now. Already, people can equip themselves with digital assets in AR-enabled spaces. For instance, users of Ready Player Me (an avatar creation platform) can create realistic AR-based avatars using their real pictures and Geenee AR’s full body tracking software. These avatars can be customized with a user’s wearable digital assets, including nonfungible tokens (NFTs) purchased across other platforms.
Uphill task ahead
The creation of these metaverse worlds is fraught with challenges. The need for online security, privacy, and identity validation will amplify further as balancing creative expression with accountability for the actions of people and organizations will be paramount. In May 2022, a researcher exploring Meta’s Horizon Worlds claimed her avatar faced sexual harassment within an hour of using the platform. Conversational chatbots used by enterprises could get hacked and start misbehaving with customers. All these scenarios pose risks to businesses.
Interoperability between independent metaverse worlds is also a key challenge. While the big tech players would prefer to create their walled spaces and monetize them the way they like, just like social media currently works, it would restrict people’s personas from working seamlessly across different environments. Moreover, authenticating users and their avatars between interoperable environments will be a challenge, due to differing security protocols and standards. This also brings forth the issue of universal standards for security and privacy. The Metaverse Standards Forum, launched in June 2022, is expected to help set some universal standards. But it won’t be easy, given the varying interests, intent, and capabilities of different industries and geographies. Developments in these technologies will, in turn, raise more issues. AI has already created problems with models developing in undesirable and unforeseen ways.
What if the autonomous actions taken by avatars reflect racism, sexism, or other such biases that AI programs have already shown they are capable of in the past?
For instance, in an experiment, Microsoft’s AI chatbot Tay learned from Twitter conversations and started posting racist comments. Similarly, crime prediction algorithms have unfairly targeted people of color.
Pre-empting the issues arising out of technological developments will be a tall order to achieve, as the rules and standards are yet to be laid out. Issues around operating standards, cyber laws, cross-border laws, compensation mechanisms, and other such aspects are expected to emerge as the ecosystem develops further. There is no magic pill to resolve these concerns, but technologies must continue evolving for the better. Businesses will look to strengthen their underlying technology and process architecture and offer solutions adhering to standards and best practices.
Drive value, not fads
Acknowledging the potential of avatars in the metaverse, several players focus on building avatars for various applications. Many of these now emphasize on creating 3D models hosted in the cloud. For example, Ready Player Me, Nvidia Omniverse, BodyGee Boxx, and UneeQ assist in cloud-based 3D model generation of avatars. These models are generated quickly and efficiently using reference pictures or videos. In some cases, using a single, front-facing image is all it takes to generate a 3D model and the necessary blend shapes required for an animated character that can speak, blink, and emote. Some of these are even capable of full body recognition and lip synchronizations, and are applied in the development of conversational AI chatbots, storytelling, web conferencing, and product trials.
In use cases where avatar likeness is of utmost importance, companies like Microsoft are specializing in holographic streaming through their Mixed Reality Capture Studios. Here, they capture 3D volumetric video of live actors against a green screen using an array of special cameras. After post-processing, and intensive optimization, the hologram (in the form of volumetric video) can be ported to a multitude of platforms. It is now possible to conduct live performances using low latency networks and holograms of performers projected on a stage in front of fans. For instance, a posthumous concert tour featuring Whitney Houston’s holograms along with live performers has been underway since February 2020.
To make the metaverse a reality, the ecosystem participants must find innovative use cases, democratize the tools and virtual environments, and enable the confluence of associated technologies. Meta (previously Facebook) pivoted to the metaverse narrative due to a declining user base, and has invested $36 billion in this space between January 2019 and September 2022. However, Meta’s Horizon Worlds is falling short of the company’s expectations, with most users not returning after the first month. Therefore, companies should focus on developing partnerships, eliminating silos, and employing a design-thinking user centric approach to solve real problems and drive value for users rather than running behind buzzwords.
Find your niche
Businesses across industries are experimenting with metaverse applications and finding out their value-driving use cases. We expect avatars to play a critical role here. Industries that traditionally used to adopt a wait-and-watch approach to technology — such as financial services — are no longer holding back amid intensifying competition and customers (particularly Gen Z) demanding better digital experiences. Experiences with digital natives such as e-commerce players are compared in parallel with, say, an automotive company, and the best is expected out of every interaction.
With this need for high digital maturity in perspective, industries including retail, consumer packaged goods, pharmaceuticals, media and entertainment, telecom, automotive, hospitality, and even industrial manufacturing have begun their journey toward delivering innovative immersive experiences. There are multiple ways for organizations to get their feet in the door. Following are some examples and considerations:
- Avatars involve cybersecurity risks, as they contain and represent critical personal information of a user. However, they could potentially be used to establish credentials and validate identities with more realistic designs incorporating face and body recognition and gesture and lip-sync matching features.
- Businesses can incorporate cartoonish, animated, or realistic avatars, depending on the target population, to cater to users’ design preferences. However, it must be noted that even though artificial figures with a human-like appearance and behavior are more familiar for users, that can change quickly if the virtual character fails to truly mimic a human being.
- In virtual spaces where avatars deliver information and assistance, businesses need to align their personas to brand ethos and make them relatable. The look, voice, accent, gestures, or body language must appeal in a way that enhances user experience over whatever is the current scenario.
The approach to metaverse use cases and associated investments can provide unique advantages to a brand. Companies must carefully consider their play based on what their customers value and select credible partners to fulfill those needs. There is still a long road ahead, but each rapid prototype proof of concept, hackathon, internal R&D project, and client engagement brings us closer to defining the metaverse and the role of avatars.