Robot adapts speech to get your attention

Researchers developed Talking-Ally, the novel robot that dynamically generate appropriate utterance and gestures based on the person's attention as indicated by his or her actions. The experiments show that this new communicational approach significantly enhances the attention engagement with interactive user.

Most existing robots, however, still use monologue mechanisms, even when engaging in dialogue with a person. For example, they continue speaking in the same way, even if the person is not paying attention.

Researchers at the Interactions and Communication Design (ICD) Lab at Toyohashi University of Technology have devised the novel robotic communication approach that takes into account the listener's attention. The robot follows a person's gaze and determines if that person is distracted by, for instance, a sports event in the background or something in their surroundings. The robot bends forward and nods if the person is watching television or it turns its head and looks around if the person is looking elsewhere. These behaviors are accompanied by appropriate utterance intended to regain the person's attention. Experiments have confirmed that these adaptive interactions considerably increase the other party's attention focused toward the robot as compared with the gestures and speech generated without considering the person's gaze.

"We have set up an environment to manipulate the person's attention with an engaging sports program broadcast simultaneously with the human-robot interaction. This allowed us to validate a suite of conversation situations and utterance-generation patterns," said Hitomi Matsushita, first author of the conference paper on the robot.

"Talking-Ally dynamically determines and synchronizes its body language, turn initials, and entrust behaviors within the speech, according to the person's attention coordinates," Professor Michio Okada, head of the ICD Lab, explained. "Our analysis shows that this is significantly more persuasive than generating these behaviors randomly."

The experimental results significantly contribute to the HRI community by confirming that adaptive communication is essential in acquiring and maintaining attention during conversation. Moreover, Talking-Ally demonstrates a specific communication protocol that is shown to successfully re-engage a distracted person. This is instrumental in achieving persuasive communication and convincing interaction with the robot. Such a platform can ultimately be tailored for use with any HRI application.

Talking-Ally currently chooses its responsive gestures at random from a set that suitably corresponds to the person's level of attention. Future work on the project will include further customizing the robot's interaction to individuals by choosing a specific body language to use in each situation based on subtle cues from the other party.