Presentation and Analysis of a Multimodal Dataset for Grounded Language Learning

Simple item page
dc.contributor.authorJenkins, Patrick
dc.contributor.authorSachdeva, Rishabh
dc.contributor.authorKebe, Gaoussou Youssouf
dc.contributor.authorHiggins, Padraig
dc.contributor.authorDarvish, Kasra
dc.contributor.authorRaff, Edward
dc.contributor.authorEngel, Don
dc.contributor.authorWinder, John
dc.contributor.authorFerraro, Francisco
dc.contributor.authorMatuszek, Cynthia
dc.date.accessioned2020-09-11T17:40:39Z
dc.date.available2020-09-11T17:40:39Z
dc.date.issued2020-07-09
dc.description.abstractGrounded language acquisition -- learning how language-based interactions refer to the world around them -- is amajor area of research in robotics, NLP, and HCI. In practice the data used for learning consists almost entirely of textual descriptions, which tend to be cleaner, clearer, and more grammatical than actual human interactions. In this work, we present the Grounded Language Dataset (GoLD), a multimodal dataset of common household objects described by people using either spoken or written language. We analyze the differences and present an experiment showing how the different modalities affect language learning from human in-put. This will enable researchers studying the intersection of robotics, NLP, and HCI to better investigate how the multiple modalities of image, text, and speech interact, as well as show differences in the vernacular of these modalities impact results.en_US
dc.description.urihttps://arxiv.org/abs/2007.14987en_US
dc.format.extent11 pagesen_US
dc.genrejournal articles preprintsen_US
dc.identifierdoi:10.13016/m24vfu-goza
dc.identifier.citationPatrick Jenkins, Rishabh Sachdeva, Gaoussou Youssouf Kebe, Padraig Higgins, Kasra Darvish, Edward Raff, Don Engel, John Winder, Francisco Ferraro and Cynthia Matuszek, Presentation and Analysis of a Multimodal Dataset for Grounded LanguageLearning, https://arxiv.org/abs/2007.14987en_US
dc.identifier.urihttp://hdl.handle.net/11603/19644
dc.language.isoen_USen_US
dc.relation.isAvailableAtThe University of Maryland, Baltimore County (UMBC)
dc.relation.ispartofUMBC Computer Science and Electrical Engineering Department Collection
dc.relation.ispartofUMBC Faculty Collection
dc.relation.ispartofUMBC Student Collection
dc.relation.ispartofUMBC Office for the Vice President of Research
dc.rightsThis item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
dc.titlePresentation and Analysis of a Multimodal Dataset for Grounded Language Learningen_US
dc.typeTexten_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
2007.14987.pdf
Size:
11.45 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
2.56 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

UMBC Computer Science and Electrical Engineering Department
UMBC Faculty Collection
UMBC Office for the Vice President of Research & Creative Achievement (ORCA)
UMBC Student Collection