When did you first hear the term augmented reality (AR)? If you're new to the tech space, your first encounter with this visual interactive technology was probably the Apple Vision Pro or the Meta Quest headset. However, it is likely that players would have first interacted with the technology in the mobile game Pokemon Go, which was released in 2016. Movie buffs might also refer to the 2008 movie Iron Man, where JARVIS gave a look at what the future of AR could be.
Interestingly, the origins of this technology predate all of these modern milestones. The first AR device was built in 1968 when American computer scientist Ivan Sutherland invented a head-mounted display that could place two-dimensional images in a 3D environment. It was a very heavy apparatus that was suspended from the ceiling and showed only rudimentary wire-framed rooms. However, this invention became the cornerstone of AR in the following decades.
What is Augmented Reality (AR)
One of the most interesting technologies in the modern world, AR is a conceptually simple mechanism. Create a 2D or 3D overlay of digital information on top of a real-world environment. Basically, it shows you information, objects, animation or an entire visual scene in the middle of your environment. For example, the game Pokemon Go could place moving Pokemon on the top of the table or on the floor whenever you point the camera at them.
The Apple Vision Pro and Meta Quest have taken the capabilities of AR further. While viewing objects or a movie with this effect is a great experience, the real immersion comes when a user can not only see digital information in the real world but also interact with it. And mixed reality headsets enable exactly that. People using these headphones can watch a movie, play a game, or use an app that is essentially a floating screen in the middle of your room. It can be expanded or minimized according to your preferences.
This growing understanding of augmented reality and the development of hardware infrastructure to support this technology has ensured today that these experiences are not only limited to particular applications or high-end devices, but are present all around us to experience- them in an environment without platforms. Today, several Indian companies are creating AR content that can be experienced directly on the smartphone with a single click of a button on a website or by scanning a QR code, without the need to purchase or download anything.
Some top names in the space include Imaginate, Parallax Labs, PlayShifu, Xtendr and more. Here's an example to put it in the context of what exactly they're building. If you're a fan of Maggi, you might remember that the company recently launched its Korean BBQ noodles, where the package had a visible QR code. Scanning it would open an AR session, allowing you to cook the noodles in an interactive 3D space. The experience acted as a “how to” for those unfamiliar with the product and allowed people to play a minigame without having to download or install anything.
Xtendr built this particular AR experience without platforms, and there's a lot that goes into creating these experiences. To understand the technology behind this AR technology and its potential, Gadgets 360 had a conversation with Anurag Sachdeva, the founder and CEO of the company.
Xtendr's foray into platformless AR
The vision of building an AR-centric company is not common. But for Sachdeva, that path was paved by his 23 years of experience with technology-focused companies. Before trying his hand at entrepreneurship, he was heading Rovio Entertainment's Angry Birds business for India, Southeast Asia and the Middle East. The company did some AR involvement; however, everything was driven through the applications.
Being a marketing professional, Sachdeva wanted to take advantage of the engagement that comes with the immersive experience of AR, but without the limitations that come with an app. This idea led to the founding of Xtendr in May 2020. However, the path was not without challenges. Starting a company in the middle of a global pandemic, adjustments needed to be made to remain operational.
“We were the first in the region to be creating AR experiences in a non-apple way. In the early days, one concern was understanding our target demographic at a time when people were also dealing with the COVID-19 pandemic ” said Sachdeva. Despite the initial hiccup, the startup was able to incorporate two brands and the founder says they saw great results in terms of engagement. This development gave Xtendr the boost it needed.
Today, Xtendr has worked with more than 50 brands across India, the Middle East and Southeast Asia. Some of the brands he has worked with include Redbull, Coca-Cola, ISRO, Unilever, Nestle Maggi, ITC Classmate and more. But how exactly do these “platform-less AR experiences” work? We went deeper.
Breaking down Xtendr's AR experiences
Before we jump in, let's take a moment to understand what Xtendr does. The startup works with consumer brands and other organizations and creates branded content in the AR space. These can be anything from a small animation playing in your bed to a fully interactive cricket game that you can play in your garden. Once activated via a URL or QR code, it loads onto your device without the need to download an app. These experiences, also known as web AR, run entirely in the cloud.
However, creating these experiences requires several components to work properly. Latency needs to be low so these experiences don't take forever to load on devices with a poor internet connection. Experiences should be fully AR compatible, create a sense of immersion and be interactive.
Sachdeva told us that Xtendr uses spatial computing and artificial intelligence (AI) to create augmented reality. Immersion has been a major focus for the company, which aims to “augment human vision.”
The actual creation process involves first coming up with an idea and creating a story-driven narrative through characters, objects, and elements. Once the script is finished, the technical work begins on the back end. The content is then converted into a program that can be run smoothly on the end user's device.
The company primarily uses JavaScript to write programs using the Three.js library. AR and VR web frameworks such as A-Frame, SLAM (Simultaneous Localization and Mapping) and React VR are also used in development. All of this goes to the cloud, where the front end comes together, and the AR experience is ready to be distributed.
The role of AI
Interestingly, the startup also incorporates generative AI into some AI experiences. The company uses large language models (LLMs) for characters in cases where they want to give users a conversational experience. AI is also used to generate art and allow users to experience AI-generated visual media.
These tools are used to design a gamified experience where the user can talk to a chatbot about the brand or do quirky things like paint the sky. Some AI models Xtendr has worked with include Inworld, Midjourney, ChatGPT, Gemini, Dall-E and more.
Latency problem resolution
A key part of creating a platformless AR experience is also its distribution. Users would prefer their content to load in seconds, and ensuring that from budget to premium devices and with low to excellent internet speeds is a big challenge. This becomes important, especially as experiences include 3D characters, complex artwork, and interactive elements.
Xtendr has in the past created gamified engagements where users could play a game and challenge another player online or on the leaderboard, where many others are also dropping their scores. With a single AR experience reaching up to two million impressions, it's important to ensure consistency.
The startup uses Amazon Web Services (AWS) for its cloud-based needs, which solves part of the problem since the cloud computing giant runs some of the most traffic-driven platforms, including Netflix, Twitch, LinkedIn, Facebook and more. . The other part is solved by optimizing the 3D renders to make sure none are too heavy for any device. Sachdeva emphasized the need for balance between art and interaction, which contribute equally to engagement.
A look into the future of augmented reality
For a technology that arrived 56 years ago, AR remains in its infancy, largely due to a lack of technology that can support vision. But in recent years, it has really entered the mainstream and become viable and profitable. That's why companies are experiencing it more and more. Today, AR has gone from a party trick to a utility tool. The digital overlay of a human vision is opening up newer avenues to be productive online while still connected to the real world instead of glued to a screen.
Sachdeva maintains that the future of AR is bright as all the key pieces needed to grow the technology exponentially are finally coming together. “5G will become mainstream in the next two years, which will further solve many of the latency issues. AR-enabled hardware, massive processing power, the emergence of generative AI, and the arrival of reality headsets mixed [by Apple and Meta]it all comes together to create the next highway for immersive media,” he added.
