Deep Learning for Category-Free Grounded Language Acquisition

Author/Creator ORCID

Date

2019-06

Department

Program

Citation of Original Publication

Nisha Pillai , et.al, Deep Learning for Category-Free Grounded Language Acquisition, NAACL-SpLU-RoboNLP, Minneapolis, Minnesota, June 2019, http://iral.cs.umbc.edu/Pubs/PillaiNAACLws2019.pdf

Rights

This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.

Abstract

We propose a learning system in which language is grounded in visual percepts without pre-defined category constraints. We present a unified generative method to acquire a shared semantic/visual embedding that enables a more general language grounding acquisition system. We evaluate the efficacy of this learning by predicting the semantics of ground truth objects and comparing the performance with each of a predefined category classifier and a simple logistic regression classifier. Our preliminary results suggest that this generative approach exhibits promising results in language grounding without pre-specifying visual categories such as color and shape.