Browsing by Subject "multimodal"
Now showing 1 - 3 of 3
Results Per Page
Sort Options
Item The Effect of Perceptual Structure on Multimodal Speech Recognition Interfaces(1998-01-01) Grasso, Michael A.; Ebert, David; Finin, TimA framework of complementary behavior has been proposed which maintains that direct manipulation and speech interfaces have reciprocal strengths and weaknesses. This suggests that user interface performance and acceptance may increase by adopting a multimodal approach that combines speech and direct manipulation. This effort examined the hypothesis that the speed, accuracy, and acceptance of multimodal speech and direct manipulation interfaces will increase when the modalities match the perceptual structure of the input attributes. A software prototype which supported a typical biomedical data collection task was developed to test this hypothesis. A group of 20 clinical and veterinary pathologists evaluated the prototype in an experimental setting using repeated measures. The results of this experiment supported the hypothesis that the perceptual structure of an input task is an important consideration when designing a multimodal computer interface. Task completion time, the number of speech errors, and user acceptance improved when interface best matched the perceptual structure of the input attributes.Item The Integrality of Speech in Multimodal Interfaces(ACM, 1998-11-30) Grasso, Michael A.; Ebert, David S.; Finin, Timothy W.A framework of complementary behavior has been proposed which maintains that direct manipulation and speech interfaces have reciprocal strengths and weaknesses. This suggests that user interface performance and acceptance may increase by adopting a multimodal approach that combines speech and direct manipulation. This effort examined the hypothesis that the speed, accuracy, and acceptance of multimodal speech and direct manipulation interfaces will increase when the modalities match the perceptual structure of the input attributes. A software prototype that supported a typical biomedical data collection task was developed to test this hypothesis. A group of 20 clinical and veterinary pathologists evaluated the prototype in an experimental setting using repeated measures. The results of this experiment supported the hypothesis that the perceptual structure of an input task is an important consideration when designing a multimodal computer interface. Task completion time, the number of speech errors, and user acceptance improved when interface best matched the perceptual structure of the input attributes.Item Timing of Support in One-on-one Math Problem Solving Coaching: A Survival Analysis Approach with Multimodal Data(ACM, 2021-04) Chen, LujieIn this paper, we explore a kind of teaching-oriented temporal analytics on the timing of support in the context of one-on-one math problem-solving coaching. We build the analytical framework upon the human-human multimodal interaction data collected from the naturalist environments. We demonstrated the potential utility of leveraging survival analysis, a class of statistical methods to model time-to-event data, to gain insights into the timing decisions. We shed light on the heterogeneity of coaching decisions as to when to render support in connection to the problem-solving stages, coaching dyads, as well as the pre-intervention event characteristics. This work opens future avenues into a different type of human tutoring study supported by multimodal data, computational models, and statistical frameworks. This model framework may yield useful reflective teaching analytics to tutors, coaches, or teachers when further developed. We also envision that those analyses may ultimately inform the design of AI-supported autonomous agents that could learn the tutorial interaction logic from empirical data.