Building Language-Agnostic Grounded Language Learning Systems

Author/Creator ORCID

Date

2019

Department

Program

Citation of Original Publication

Pillai, N., Matuszek, C., Ferraro, F., & Kery, C. (2019). Building Language-Agnostic Grounded Language Learning Systems.

Rights

This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
© 2019 IEEE.

Abstract

Learning the meaning of grounded language— language that references a robot’s physical environment and perceptual data—is an important and increasingly widely studied problem in robotics and human-robot interaction. However, with a few exceptions, research in robotics has focused on learning groundings for a single natural language pertaining to rich perceptual data. We present experiments on taking an existing natural language grounding system designed for English and applying it to a novel multilingual corpus of descriptions of objects paired with RGB-D perceptual data. We demonstrate that this specific approach transfers well to different languages, but also present possible design constraints to consider for grounded language learning systems intended for robots that will function in a variety of linguistic settings