Meta’s Reality Labs division has unveiled new prototypes in its roadmap to lightweight, hyper-realistic VR graphics. Breakthroughs remain far from consumer-ready, but the designs – codenamed Butterscotch, Starburst, Holocake 2 and Mirror Lake – could add up to a slim, well-lit helmet that supports finer details than its current Quest 2 screen.
Meta CEO Mark Zuckerberg and Reality Labs Chief Scientist Michael Abrash, along with other Reality Labs members, presented their work during a virtual panel discussion last week. The event focused on designs that Meta calls “time machines”: chunky proofs of concept intended to test a specific feature, like an ultra-bright backlight or a super-high resolution display. “I think we’re in the midst of a big step toward realism right now,” Zuckerberg told reporters. “I don’t think it will be long before we can create scenes with basically perfect fidelity.” Display technology isn’t the only unsolved piece of this puzzle, but it’s an area where Meta’s extensive VR hardware research is giving it a head start.
Zuckerberg has reiterated plans to ship a high-end headset codenamed Project Cambria in 2022, following its initial announcement last year. Cambria supports full virtual reality as well as mixed reality, with high-resolution cameras that can stream video to an internal display. It will also come with eye tracking, a key feature of future Meta headsets. From there, Zuckerberg says Meta is planning two lines of VR headsets: one that will remain cheap and consumer-focused, like today’s Quest 2, and another that will incorporate the latest technology. of the company, intended for a “prosumer or professional” market. . This follows reports that the company is already planning updates to Cambria and the Quest 2, although those prototypes weren’t discussed on the call.
Meta’s VR headsets sit alongside a separate line of augmented reality smart glasses, which are meant to project images onto the real world instead of blocking it out with a screen. Meta recently scaled back the launch of its first-generation AR glasses, and in general, VR displays have reached consumers much faster than AR holograms. But Meta’s prototypes show how far the company thinks it has to go.
Butterscotch is an attempt at a near-retina-quality headset display – something you can find in high-end headsets from companies like Varjo, but not the current Meta line. The design is “far from shippable” and required roughly halving the 110 degree field of view of the Meta Quest 2. But it offers about 2.5 times the resolution of the Quest 2 (sort of) 1832 x 1920 pixels per eye, allowing users to read the 20/20 line of sight on an eye chart. Zuckerberg says it offers around 55 pixels per degree of field of view, slightly lower than Meta’s retina standard of 60 pixels per degree and a bit lower than Varjo’s 64 pixels per degree.
Starburst is even less deliverable than Butterscotch but tests an equally impressive upgrade. The chunky design uses a powerful lamp – requiring handles to support its weight – and produces high dynamic range (HDR) illumination with 20,000 nits of brightness. “This one is extremely impractical to consider as product direction for the first generation, but we’re using it as a test bed for further research and study,” says Zuckerberg. “The goal of all of this work is to help us identify the technical paths that are going to allow us to make improvements that are significant enough that we can begin to approach visual realism.”
Holocake 2 moves in the opposite direction, exploring Meta options to make VR headsets thinner and lighter. It’s the successor to a 2020 design based on holographic optics, a light bending technique that allows a nearly flat panel to replace a thick refracting lens. The result could be as thin as sunglasses, but Meta is still working on developing a self-contained light source that would power them – almost certainly a laser, not the OLEDs commonly used today. “We’ll have to do a lot of engineering to get a consumer-viable laser that meets our specifications: it’s safe, inexpensive, and efficient, and can fit in a thin VR headset,” says Zuckerberg. “Honestly, as of today, the jury is out on a suitable laser source.”
The presentation also touched on Half Dome, a long series of prototypes that can shift focal planes depending on where users are looking. These varifocal optics started out as a clumsy mechanical system in 2017 and later transitioned to a range of liquid crystal lenses, and according to internal Meta research, they can create a more convincing (and physically comfortable) illusion of depth in VR. .
Meta described Half Dome’s technology as “almost ready for prime time” in 2020, but today Zuckerberg was more measured. “This stuff is pretty far away,” he said in response to a question on the “primetime” comment. “We’re working on it, we really want to get it into one of the next helmets, I’m sure we will at some point, but I’m not going to announce anything in advance today.”
Reality Labs will discuss more research, including how to more accurately capture real-world images for mixed reality, at August’s SIGGRAPH show.
The above designs exist as actual hardware that Zuckerberg briefly showcased at the event. But Meta also revealed a prototype, called Mirror Lake, which is essentially ambitious and was never built. The design looks more like a pair of ski goggles than Meta’s chunky Quest hardware, and it’s said to incorporate Holocake 2’s fine optics, Starburst’s HDR capabilities, and Butterscotch’s resolution. “It shows what a complete next-gen display system could look like,” Abrash said.
In addition to these features, Mirror Lake would include an outward-facing screen that projects an image of the user’s eyes, reducing the sense of physical separation for people outside the headset. Meta showed off this slightly odd feature in a prototype last year, and it may not be the only company interested in the concept: Apple is reportedly considering a similar feature for its rumored headset. The idea is suited for a mixed reality world where Meta has staked much of its future – but today the company is emphasizing incremental steps along the way.