Vid2RealHRI: Align video-based HRI study designs with real-world settings

Abstract

HRI research using autonomous robots in real-world settings can produce results with the highest ecological validity of any study modality, but many difficulties limit such studies' feasibility and effectiveness. We propose Vid2RealHRI, a research framework to maximize real-world insights offered by video-based studies.

The Vid2RealHRI framework was used to design an online study using first-person videos of robots as real-world encounter surrogates. The online study (n = 385) distinguished the within-subjects effects of four robot behavioral conditions on perceived social intelligence and human willingness to help the robot enter an exterior door. A real-world, between- subjects replication (n = 26) using two conditions confirmed the validity of the online study's findings and the sufficiency of the participant recruitment target (22) based on a power analysis of online study results. The Vid2RealHRI framework offers HRI researchers a principled way to take advantage of the efficiency of video-based study modalities while generating directly transferable knowledge of real-world HRI.

Interventions

Baseline: robot stops at the entrance and waits for the nearby human to push the button, without offering any indications that it needs assistance

Body-Language-only: robot stops at the entrance and turns its head facing the nearby human, then turns its head back to look at the door, then waits for the human to push the button

Verbal-only: robot stops at the entrance and says, "Excuse me, can you open the door for me?" to the nearby human, then waits for the human to push the button

Body-Language + Verbal: robot stops at the entrance and turns its head facing the nearby human and says, "Excuse me, can you open the door for me?", then turns its head back to look at the door and waits for the human to push the button

Video-Based Study

Video recording methodology followed to ensure consistency between different conditions. Speech uses the python libraries (gTTs and pydub) to produce audio from text on the fly. Bodypose uses pointcloud-based object detection to detect the observer and leverages the Clearpath Spot-ROS wrapper (built upon Boston Dynamics API) to modify the pose of the robot in a way that its head is "looking" at the observer. Motreplay uses a recorded ROS-bagfile and republishes the stream of motion commands required to generate identical navigation motion for the robot.