Apple is expected to announced its metaverse platform, named Apple Reality, in the first half of 2023, with Pegatron reportedly serving as the exclusive assembling partner of Apple’s VR/AR headsets, named Apple Reality Pro and Apple Reality One.
According to Patently Apple, “Under a shell company, Apple has filed for four new trademarks in multiple countries relating to their Mixed Reality Headset. Bloomberg published this revelation that Apple made the fillings under the company name ‘Immersive Health Solutions, LLC’ in Delaware.”
Initial production of Apple’s first run of headsets is said to be very limited, along with a steep price point that will largely limit the the gear to industry experts and commercial markets.
Regardless, as soon as Apple delivers its first headset to market, the future of the company–and the world–will forever be changed. This may sound like hyperbole. It isn’t. Between Apple’s dominance in tech and the mixed reality features that the company is set to unveil as part of its Apple Reality platform, nearly every facet of society will be impacted, from commerce to health, entertainment, education, gaming and beyond.
Accordingly, this is everything you need to know about the revolutionary Apple Reality platform, including virtual reality, augmented reality, headsets, and more.
Apple Reality Pro
Apple’s first headset, the Apple Reality Pro, is slimmer and more streamlined than current models on the market, including the Meta Quest 2 or Pico 4. The design more closely resembles ski goggles than traditional bulky VR gear, which is an important distinction considering the model features a “see-through front.”
By combining a high-resolution, full-spectrum screen along with 14 cameras and a multitude of sensors to track face, eye, hand, and body movements in conjunction with the surrounding environment, users will be able to seamlessly transition between virtual reality, augmented reality, and mixed reality.
The benefits of having a screen that presents itself as a window to the real world are manifold. Users will be able to wear the headsets in more environments, encouraging wider adoption of the technology. The headsets will, in turn, become more socially acceptable, with eye contact allowing for more personal and immediate interaction. And Apple’s emphasis on augmented reality and apps, as opposed to Meta’s emphasis on virtual reality, will more readily resonate with the general population who are already conditioned to use laptops/tablets/phones as a natural extension of everyday life.
Apple Reality Pro represents the company’s first headset release, a high-end, prohibitively expensive model that will be marketed towards industry experts and commercial customers with professional productivity in mind. The second release, Apple Reality One, represents the company’s more affordable, mass-market headset made for everyday consumers.
Both models are expected to be powered by the M2 chip, more powerful than the XR2 chips used in various other VR headsets. Furthermore, Apple has trademarked “RealityOS” via the shell company RealityO Systems LLC. The RealityOS operating system is expected to be a custom version of OSX or iOS catering to extended reality spatial interactions.
Input Devices
Apple is pushing the hardware design envelope with the introduction of wearable finger clips for more reliable and accurate hand tracking. Think of them as press-on nails, as envisioned by the above concept art, so that one’s fingertips remain uncovered for better tactile and touchscreen usage. Matter of fact, Apple looks to be doing away with traditional handsets altogether, freeing up users’ hands, further dissolving the barriers between actively using virtual, augmented, and mixed reality.
Further patent applications have indicted that Apple is exploring other input devices such as a stylus pencil of sorts, though with the fingertip clips and advanced hand-tracking capabilities of the platform, it would seem that the technology is already moving beyond such accessories.
Multimodal Interaction
Apple Reality will blend together various interfaces and interactions, making it possible to seamlessly transition between various hardware and software. For now, we use a mouse and pointer to navigate our computers, or our fingertips and voice commands to interact with tablets and phones.
Apple’s multimodal capabilities will allow users to combine voice commands, hand gestures, eye, face, and body movements to dictate commands and navigate screens, making computing feel more like an extension of our physical self than ever before.
Of course, our old friend Siri will be part of this equation as voice commands will be an integral part of the extended reality experience, though Apple R&D has been investigating how to effectively implement commands without the need for the “Hey Siri” keyword.
Room Mapping
Apple is leaps and bounds beyond its competitors in room mapping, which represents a core capability in blending together the extended reality (XR) experience. Apple’s mapping technology is able to vividly and accurately display an environment, label and measure household objects such as couches, desks, refrigerators, chairs, windows, and more.
This puts Apple Reality at the forefront of delivering easy and effective experiences related to virtual reality gaming, mixed reality shopping, digital design and architecture, and more.
Body Tracking
Apple has committed to leading the development of digital body tracking, another essential facet of the overall mixed reality experience. The company is blending its 14 cameras and multitude of sensors with machine learning to accurately map out limbs and overall movements.
Full body tracking takes users beyond the current standard of floating legless avatars, which caused Mark Zuckerberg to receive an avalanche of criticism following his European launch of the Horizon Worlds platform. The key advantage to full body tracking is not only in the overall user experience, but also in fitness, training, rehabilitation, and other such physiological applications.
Spatial Audio
Francis Ford Coppola is famous for his commitment to quality sound, saying that it represents half the theatrical experience (visuals being the other half, of course). Apple’s commitment to quality sound is just as devoted, with the company’s spatial audio taking users’ aural experiences to the next level.
Apple spatial audio applies directional filters to sound, adjusting the frequencies that one hears so that sounds can be placed anywhere within a 3D space. As a result, sounds will realistically appear to be coming from in front of you, from the sides, the rear, above, or below, depending on what you’re watching or interacting with. The result is a fully immersive, surround sound cinema experience no matter the application.
What’s more, the technology tracks your head movement using accelerometers and gyroscopes, in addition to tracking the spatial orientation of the device(s) you are using, making the levels and directional origination of sound seem truly authentic.
AirPods 3, AirPods Pro, AirPods Max and Beats Fit Pro already utilize these features. However, rumors of multiple microphones on the Apple Reality may be used to enhance the ability for selective noise cancellation, allowing users to choose audio sources. This would allow, for example, one to selectively hear the person you’re talking to despite being in a crowded, noisy environment.
The end result allows Apple to not only provide the highest level of visual acuity in the digital realm, but audio acuity as well.
Apps
Apple is already setting the stage to evolve many of their existing apps for mixed reality (MR) usage through their finger tracking capabilities. Collaborative whiteboard tools are just one example of enhancing productivity and interoperability in the digital space, with both individual users and businesses utilizing the tech.
Then there are the apps that we all rely on everyday, like maps and navigation. Nobody should be driving around with VR goggles on their head, of course, but Apple is rumored to be ramping up design specs for augmented reality glasses in 2026 (remember earlier in the article when we mentioned Google Glass was ahead of its time?). As a result, being able to navigate via a heads-up display like an F-35 fighter pilot will eventually make its way to Main Street. Meanwhile, with Apple taking the steps to 3D model cities and towns the world over, it’ll become commonplace for folks to travel the world in virtual reality without ever leaving the comfort of their living room.
Now you may be thinking, “How will I effectively navigate foreign countries without knowing the local language?” Fear not. Just as Google Lens allows users to instantly translate signs, menus, or other text simply by pointing at it with their phones, Apple Reality will do the same in real time no matter the environment or location.
Entertainment
Apple is investing heavily in both legacy and original content to make AppleTV on par with the likes of Prime Video and Netflix. By combining high quality content with next-gen hardware and software, the company is betting on edging out the competition by becoming a 360-degree entertainment provider.
Not stopping there, Apple also acquired virtual reality streaming company NextVR in 2020. NextVR specializes in broadcasting livestreamed events in 3D. Instead of simply watching a concert on YouTube via some janky, bootlegged phone recording, you’ll be able to experience the concert as it happens, from the venue itself. This applies to sports, too. Can’t afford a ticket to the big game? No worries. Virtual reality tech will place you right in the action, not as though you’re simply a spectator, but as one of the players! For a sampling of the concept, be sure to check out the embedded video above.
