Spoken language generation in a multimedia system

Abstract

In this paper we address two important issues in generating spoken language within a multimedia system: the design of a speech generator to facilitate coordination between media, and extensions to the functionality of a written language generation system to produce natural speech output. We demonstrate how a speech generator can produce information that allows for temporal coordination between multiple media. We describe how our speech generator takes advan- tage of rich and accurate syntactic and semantic information during text planning and speech realization. This enables the system to accurately predict, generate, and utilize prosodic features to facilitate coordination of speech with graphical actions such as highlighting.