Browsing by Author "Chambers, Nathanael"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item PASTA: A Dataset for Modeling Participant States in Narratives(2022-07-31) Ghosh, Sayontan; Koupaee, Mahnaz; Chen, Isabella; Ferraro, Francis; Chambers, Nathanael; Balasubramanian, NiranjanThe events in a narrative can be understood as a coherent whole via the underlying states of its participants. Often, these participant states are not explicitly mentioned in the narrative, left to be filled in via common-sense or inference. A model that understands narratives should be able to infer these implicit participant states and reason about the impact of changes to these states on the narrative. To facilitate this goal, we introduce a new crowdsourced Participants States dataset, PASTA. This dataset contains valid, inferable participant states; a counterfactual perturbation to the state; and the changes to the story that would be necessary if the counterfactual was true. We introduce three state-based reasoning tasks that test for the ability to infer when a state is entailed by a story, revise a story for a counterfactual state, and to explain the most likely state change given a revised story. Our benchmarking experiments show that while today's LLMs are able to reason about states to some degree, there is a large room for improvement, suggesting potential avenues for future research.Item SAGEViz: SchemA GEneration and Visualization(ACL Anthology, 2023-12) Devare, Sugam; Koupaee, Mahnaz; Gunapati, Gautham; Ghosh, Sayontan; Vallurupalli, Sai; Lal, Yash Kumar; Ferraro, Francis; Chambers, Nathanael; Durrett, Greg; Mooney, Raymond; Erk, Katrin; Balasubramanian, NiranjanSchema induction involves creating a graph representation depicting how events unfold in a scenario. We present SAGEViz, an intuitive and modular tool that utilizes human-AI collaboration to create and update complex schema graphs efficiently, where multiple annotators (humans and models) can work simultaneously on a schema graph from any domain. The tool consists of two components: (1) a curation component powered by plug-and-play event language models to create and expand event sequences while human annotators validate and enrich the sequences to build complex hierarchical schemas, and (2) an easy-to-use visualization component to visualize schemas at varying levels of hierarchy. Using supervised and few-shot approaches, our event language models can continually predict relevant events starting from a seed event. We conduct a user study and show that users need less effort in terms of interaction steps with SAGEViz to generate schemas of better quality. We also include a video demonstrating the system.