Vid2RealHRI: Align video-based HRI study designs with real-world settings

1School of Information, UT Austin,
2Department of Computer Science, UT Austin

The Vid2RealHRI end-to-end framework for transferring insights from video-based HRI study designs into real-world settings.


HRI research using autonomous robots in real-world settings can produce results with the highest ecological validity of any study modality, but many difficulties limit such studies' feasibility and effectiveness. We propose Vid2RealHRI, a research framework to maximize real-world insights offered by video-based studies.

The Vid2RealHRI framework was used to design an online study using first-person videos of robots as real-world encounter surrogates. The online study (n = 385) distinguished the within-subjects effects of four robot behavioral conditions on perceived social intelligence and human willingness to help the robot enter an exterior door. A real-world, between- subjects replication (n = 26) using two conditions confirmed the validity of the online study's findings and the sufficiency of the participant recruitment target (22) based on a power analysis of online study results. The Vid2RealHRI framework offers HRI researchers a principled way to take advantage of the efficiency of video-based study modalities while generating directly transferable knowledge of real-world HRI.


Video-Based Study

Video recording methodology followed to ensure consistency between different conditions. Speech uses the python libraries (gTTs and pydub) to produce audio from text on the fly. Bodypose uses pointcloud-based object detection to detect the observer and leverages the Clearpath Spot-ROS wrapper (built upon Boston Dynamics API) to modify the pose of the robot in a way that its head is "looking" at the observer. Motreplay uses a recorded ROS-bagfile and republishes the stream of motion commands required to generate identical navigation motion for the robot.

Latest News

Workshop Paper

March 11, 2024

Presented at the Workshop YOUR Study Design! workshop at HRI 2024.

Video Dataset Published

February 14, 2024

Online video-based questionnaire and video data released on the Texas Data Repository.