A user in AR lasso-selects a pile of books and boxes on the floor, which generates GhostObjects—virtual twins colocated with the real objects. The user then drags the GhostObjects along a trajectory, snapping them into their default shelf positions. And then the robot enters the scene and physically places them on the shelves afterwards.