Parastoo Abtahi
@parastoo@hci.social  ·  activity timestamp 2 weeks ago

Check out Lauren Wang’s #UIST2025 poster on GhostObjects: life-size, world-aligned virtual twins for fast and precise robot instruction, with real-world lasso selection, multi-object manipulation, and snap-to-default placement.

This is the first piece in her ongoing work on #AR for #HRI 🤖👓 @hci

A user in AR lasso-selects a pile of books and boxes on the floor, which generates GhostObjects—virtual twins colocated with the real objects. The user then drags the GhostObjects along a trajectory, snapping them into their default shelf positions. And then the robot enters the scene and physically places them on the shelves afterwards.
A user in AR lasso-selects a pile of books and boxes on the floor, which generates GhostObjects—virtual twins colocated with the real objects. The user then drags the GhostObjects along a trajectory, snapping them into their default shelf positions. And then the robot enters the scene and physically places them on the shelves afterwards.
Parastoo Abtahi
@parastoo@hci.social  ·  activity timestamp last week

Even virtual agents’ actions can have physical effects, with motion paths that divert attention from the hidden robot. 🐝

3/5

a bee-like virtual character flies toward a Pringles can on a table to the right of the scene. As it arrives, the real can is visually masked, and a virtual duplicate appears. The bee carries the virtual can along a curving path toward the left. Near the end, the bee heads straight to a drop point; at the same moment, the physical can arrives there (moved by a hidden mobile robot). The virtual and physical coincide, the bee exists, and the can seems to materialize.
a bee-like virtual character flies toward a Pringles can on a table to the right of the scene. As it arrives, the real can is visually masked, and a virtual duplicate appears. The bee carries the virtual can along a curving path toward the left. Near the end, the bee heads straight to a drop point; at the same moment, the physical can arrives there (moved by a hidden mobile robot). The virtual and physical coincide, the bee exists, and the can seems to materialize.
Parastoo Abtahi
@parastoo@hci.social replied  ·  activity timestamp last week

In #AR, using real-time on-device 3D Gaussian splatting, we create the illusion that physical changes occur instantaneously, while a hidden robot fulfills the “reality promise” moments later, updating the physical world to match what users already perceive visually. 🤖

4/5

A user wearing a headset looks at a mobile robot in a room. From the user’s eyes, a viewing cone spans the robot from top to bottom. Point-based “splats” inside the cone are colored red, showing what is visible, and those outside are gray, for discard. As the headset moves or the robot moves, the cone adapts to always cover the robot. A second cone is used for the arm, and it grows and shrinks to cover the arm as it expands. Feathering is used to blend the cone with the background, acting like an invisibility cloak.
A user wearing a headset looks at a mobile robot in a room. From the user’s eyes, a viewing cone spans the robot from top to bottom. Point-based “splats” inside the cone are colored red, showing what is visible, and those outside are gray, for discard. As the headset moves or the robot moves, the cone adapts to always cover the robot. A second cone is used for the arm, and it grows and shrinks to cover the arm as it expands. Feathering is used to blend the cone with the background, acting like an invisibility cloak.
Parastoo Abtahi
@parastoo@hci.social  ·  activity timestamp 2 weeks ago

Check out Lauren Wang’s #UIST2025 poster on GhostObjects: life-size, world-aligned virtual twins for fast and precise robot instruction, with real-world lasso selection, multi-object manipulation, and snap-to-default placement.

This is the first piece in her ongoing work on #AR for #HRI 🤖👓 @hci

A user in AR lasso-selects a pile of books and boxes on the floor, which generates GhostObjects—virtual twins colocated with the real objects. The user then drags the GhostObjects along a trajectory, snapping them into their default shelf positions. And then the robot enters the scene and physically places them on the shelves afterwards.
A user in AR lasso-selects a pile of books and boxes on the floor, which generates GhostObjects—virtual twins colocated with the real objects. The user then drags the GhostObjects along a trajectory, snapping them into their default shelf positions. And then the robot enters the scene and physically places them on the shelves afterwards.