Does Reasoning Help LLM Agents Play Dungeons and Dragons? A Prompt Engineering Experiment

Simple item page
dc.contributor.authorDelafuente, Patricia J
dc.contributor.authorHonraopatil, Arya
dc.contributor.authorMartin, Lara J.
dc.date.accessioned2026-03-05T19:36:37Z
dc.date.issued2025-11-09
dc.descriptionWordplay: When Language Meets Games Workshop (EMNLP 2025), November 9th, 2025, Suzhou, China
dc.description.abstractThis paper explores the application of Large Language Models (LLMs) and reasoning to predict Dungeons & Dragons (DnD) player actions and format them as Avrae Discord bot commands. Using the FIREBALL dataset, we evaluated a reasoning model, DeepSeek-R1-Distill-LLaMA-8B, and an instruct model, LLaMA-3.1-8B-Instruct, for command generation. Our findings highlight the importance of providing specific instructions to models, that even single sentence changes in prompts can greatly affect the output of models, and that instruct models are sufficient for this task compared to reasoning models.
dc.description.urihttps://wordplay-workshop.github.io/pdfs/29.pdf
dc.format.extent11 pages
dc.genreconference papers and proceedings
dc.identifierdoi:10.13016/m26v97-axw8
dc.identifier.citationDelafuente, Patricia, Arya Honraopatil, and Lara J. Martin. “Does Reasoning Help LLM Agents Play Dungeons and Dragons? A Prompt Engineering Experiment.” Paper presented at Wordplay: When Language Meets Games Workshop, Suzhou, China. November 9, 2025. https://wordplay-workshop.github.io/pdfs/29.pdf.
dc.identifier.urihttp://hdl.handle.net/11603/42166
dc.language.isoen
dc.publisherWordplay Workshop
dc.relation.isAvailableAtThe University of Maryland, Baltimore County (UMBC)
dc.relation.ispartofUMBC Student Collection
dc.relation.ispartofUMBC Computer Science and Electrical Engineering Department
dc.relation.ispartofUMBC Faculty Collection
dc.relation.ispartofUMBC Data Science
dc.rightsThis item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
dc.subjectComputer Science - Computation and Language
dc.titleDoes Reasoning Help LLM Agents Play Dungeons and Dragons? A Prompt Engineering Experiment
dc.typeText
dcterms.creatorhttps://orcid.org/0009-0005-2291-8837
dcterms.creatorhttps://orcid.org/0009-0006-6891-3545
dcterms.creatorhttps://orcid.org/0000-0002-0623-599X

Files

Collections

UMBC Student Collection
UMBC Computer Science and Electrical Engineering Department
UMBC Data Science
UMBC Faculty Collection