Nonverbal communication in human interaction

Robotics has already improved lives by taking over dull, dirty, and dangerous jobs, freeing people for safer, more skillful pursuits. For instance, autonomous mechanical arms weld cars in factories, and autonomous vacuum cleaners keep floors clean in millions of homes. However, most currently deployed robotic devices operate primarily without human interaction, and are typically incapable of understanding natural human communication.

As robotic hardware costs decrease and computational power increases, robotics research is moving from these autonomous but isolated systems to individualized personal robots. Robots at home can help elderly or disabled users with daily tasks, such as preparing a meal or getting dressed, which can increase independence and quality of life. Robots for manufacturing can act as intelligent third hands, improving efficiency and job safety for workers. Robot tutors can provide students with one-on-one, personalized lessons to augment their classroom time. Robot therapy assistants can act as social conduits between those with social impairments, such as autism, and their caretakers or therapists.

To be good social partners, robots must understand and use existing human communication structures. While verbal communication tends to dominate human interactions, nonverbal communication, such as gestures and eye gaze, augments and extends spoken communication. Nonverbal communication happens bi-directionally in an interaction, so personal robots must be able to both recognize and generate nonverbal behaviors. The behaviors to be selected are extremely dependent on context, with different types of behaviors accomplishing different communicative goals like explaining information or managing conversational turn-taking. To be effective in the real world, this nonverbal awareness must occur in real time in dynamic, unstructured interactions.

My research focuses on developing bidirectional, context aware, real time nonverbal behaviors for personally assistive robots. Developing effective nonverbal communication for robots engages a number of disciplines including autonomous control, machine learning, computer vision, design, and cognitive psychology. All of my work is driven by the vision of designing robots that make people’s lives better through greater independence, education, and workplace efficiency.