Decompositional Semantics: Examining Semantic Proto Roles For Domain Transfer

Abstract

Both the predicate and argument of a sentence can contain contextual information which can be used to understand the underlying semantic properties of a sentence. In semantic understanding, the area of decompositional semantics represents the meaning of particular aspects of language, such as an event described in a sentence, by inferring what properties or characteristics are likely to hold of a participant in that event. We examine the decompositional semantics technique known as Semantic Protoroles to explore how high performing language models transfer to new domains. We perform a targeted analysis of these models to determine the robustness of these models to changes. We find that some of the properties most accurately predicted on a standard NLP dataset of "broad" text transfers poorly to the new domain. In contrast, some of the least accurately predicted properties transferred the best. Our findings demonstrate that even with modern, high-performing large language models, domain transfer remains a challenge for semantic understanding, which must be addressed as communication-based technologies become more standard and wide-spread.