A user wearing a headset looks at a mobile robot in a room. From the user’s eyes, a viewing cone spans the robot from top to bottom. Point-based “splats” inside the cone are colored red, showing what is visible, and those outside are gray, for discard. As the headset moves or the robot moves, the cone adapts to always cover the robot. A second cone is used for the arm, and it grows and shrinks to cover the arm as it expands. Feathering is used to blend the cone with the background, acting like an invisibility cloak.
A user in AR lasso-selects a pile of books and boxes on the floor, which generates GhostObjects—virtual twins colocated with the real objects. The user then drags the GhostObjects along a trajectory, snapping them into their default shelf positions. And then the robot enters the scene and physically places them on the shelves afterwards.