Facebook: Expect Oculus Insight tracking in future wearable AR glasses

Ask any Oculus Quest or Rift S user to name the headset’s most impressive new feature, and there’s a good chance they’ll point to Insight, the tracking system that accurately senses your head and body movements without external cameras. In blog posts today, Facebook is revealing some of Insight’s little-known technical details while making an interesting prediction: “the future for this technology is in all-day wearable AR glasses that are spatially aware.”

Engineers already knew that Insight uses computer vision and visual-inertial simultaneous localization and mapping (SLAM) technologies for tracking, relying on multiple wide-angle cameras to track controller locations and your relative position in a room. But making Insight run on a mobile processor for VR purposes was no simple feat, due in equal parts to the heavy computational requirements of real-time 3D space mapping and a headset wearer’s intolerance for tracking imperfections that might not be noticeable on mobile displays — jitter, swimminess, and inaccuracy. Oculus engineers decided that Insight’s “spatial AI” system needed to work to a sub-millimeter level of accuracy, and do so without eating the Quest’s entire processing budget.