FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information

Simple item page
dc.contributor.authorZhu, Andrew
dc.contributor.authorAggarwal, Karmanya
dc.contributor.authorFeng, Alexander
dc.contributor.authorMartin, Lara J.
dc.contributor.authorCallison-Burch, Chris
dc.date.accessioned2023-05-25T18:53:03Z
dc.date.available2023-05-25T18:53:03Z
dc.date.issued2023-05-02
dc.descriptionProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, July 2023, Toronto, Canada
dc.description.abstractDungeons & Dragons (D&D) is a tabletop roleplaying game with complex natural language interactions between players and hidden state information. Recent work has shown that large language models (LLMs) that have access to state information can generate higher quality game turns than LLMs that use dialog history alone. However, previous work used game state information that was heuristically created and was not a true gold standard game state. We present FIREBALL, a large dataset containing nearly 25,000 unique sessions from real D&D gameplay on Discord with true game state info. We recorded game play sessions of players who used the Avrae bot, which was developed to aid people in playing D&D online, capturing language, game commands and underlying game state information. We demonstrate that FIREBALL can improve natural language generation (NLG) by using Avrae state information, improving both automated metrics and human judgments of quality. Additionally, we show that LLMs can generate executable Avrae commands, particularly after finetuning.en_US
dc.description.urihttps://aclanthology.org/2023.acl-long.229/en_US
dc.format.extent23 pagesen_US
dc.genreconference papers and proceedingsen_US
dc.identifierdoi:10.13016/m20xxa-s8k3
dc.identifier.citationAndrew Zhu, Karmanya Aggarwal, Alexander Feng, Lara Martin, and Chris Callison-Burch. 2023. FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4171–4193, Toronto, Canada. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.acl-long.229
dc.identifier.urihttps://doi.org/10.18653/v1/2023.acl-long.229
dc.identifier.urihttp://hdl.handle.net/11603/28082
dc.language.isoen_USen_US
dc.publisherACL
dc.relation.isAvailableAtThe University of Maryland, Baltimore County (UMBC)
dc.relation.ispartofUMBC Computer Science and Electrical Engineering Department Collection
dc.relation.ispartofUMBC Faculty Collection
dc.rightsThis item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.en_US
dc.rightsAttribution 4.0 International (CC BY 4.0)*
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/*
dc.titleFIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Informationen_US
dc.typeTexten_US
dcterms.creatorhttps://orcid.org/0000-0002-0623-599Xen_US

Files

Original bundle
Now showing 1 - 2 of 2
Thumbnail Image
Name:
2023.acl-long.229 (1).pdf
Size:
557.66 KB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
2023.acl-long.229.mp4
Size:
82.15 MB
Format:
Video MP4
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
2.56 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

UMBC Computer Science and Electrical Engineering Department
UMBC Faculty Collection