Using LLMs for Augmenting Hierarchical Agents with Common Sense Priors

dc.contributor.authorPrakash, Bharat
dc.contributor.authorOates, Tim
dc.contributor.authorMohsenin, Tinoosh
dc.date.accessioned2024-06-20T17:05:08Z
dc.date.available2024-06-20T17:05:08Z
dc.date.issued2024-05-12
dc.descriptionFLAIRS-37, May 19-21, Sandestin Beach, FL
dc.description.abstractSolving long-horizon, temporally-extended tasks using Reinforcement Learning (RL) is challenging, compounded by the common practice of learning without prior knowledge (or tabula rasa learning). Humans can generate and execute plans with temporally-extended actions and quickly learn to perform new tasks because we almost never solve problems from scratch. We want autonomous agents to have this same ability. Recently, LLMs have been shown to encode a tremendous amount of knowledge about the world and to perform impressive in-context learning and reasoning. However, using LLMs to solve real world problems is hard because they are not grounded in the current task. In this paper we exploit the planning capabilities of LLMs while using RL to provide learning from the environment, resulting in a hierarchical agent that uses LLMs to solve long-horizon tasks. Instead of completely relying on LLMs, they guide a high-level policy, making learning significantly more sample efficient. This approach is evaluated in simulation environments such as MiniGrid, SkillHack, and Crafter, and on a real robot arm in block manipulation tasks. We show that agents trained using our approach outperform other baselines methods and, once trained, don't need access to LLMs during deployment.
dc.description.sponsorshipThis project was sponsored by the U.S. Army Research Labo-ratory under Cooperative Agreement Number W911NF2120076.Copyright © 2024 by the authors.
dc.description.urihttps://journals.flvc.org/FLAIRS/article/view/135602
dc.format.extent7 pages
dc.genreconference papers and proceedings
dc.identifierdoi:10.13016/m2kla0-rtm9
dc.identifier.citationPrakash, Bharat, Tim Oates, and Tinoosh Mohsenin. “Using LLMs for Augmenting Hierarchical Agents with Common Sense Priors.” The International FLAIRS Conference Proceedings 37 (May 12, 2024). https://journals.flvc.org/FLAIRS/article/view/135602.
dc.identifier.urihttps://doi.org/10.32473/flairs.37
dc.identifier.urihttp://hdl.handle.net/11603/34649
dc.language.isoen_US
dc.publisherLibraryPress@UF
dc.relation.isAvailableAtThe University of Maryland, Baltimore County (UMBC)
dc.relation.ispartofUMBC Faculty Collection
dc.relation.ispartofUMBC Computer Science and Electrical Engineering Department
dc.rightsCC BY-NC 4.0 DEED Attribution-NonCommercial 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by-nc/4.0/
dc.titleUsing LLMs for Augmenting Hierarchical Agents with Common Sense Priors
dc.typeText

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
FLAIRS_37_72.pdf
Size:
3.66 MB
Format:
Adobe Portable Document Format