Affect, Support and Personal Factors: Multimodal Causal Models of One-on-one Coaching

Abstract

Human one-on-one coaching involves complex multimodal interactions. Successful coaching requires teachers to closely monitor students’ cognitive-affective states and provide support of optimal type, timing, and amount. However, most of the existing human tutoring studies focus primarily on verbal interactions and have yet to incorporate the rich aspects of multimodal cognitive-affective experiences. Meanwhile, the research community lacks principled methods to fully exploit the complex multimodal data to uncover the causal relationships between coaching supports and students’ cognitive-affective experiences and their stable individual factors. We explore an analytical framework that is explainable and amenable to incorporating domain knowledge. The proposed framework combines statistical approaches in Sparse Multiple Canonical Correlation, causal discovery and inference methods for observations. We demonstrate this framework using a multimodal one-on-one math problem-solving coaching dataset collected at naturalist home environments involving parents and young children. The insights derived from our analyses may inform the design of effective technology-inspired interventions that are personalized and adaptive.