Browsing by Subject "multimodality"
Now showing 1 - 4 of 4
Results Per Page
Sort Options
Item Digital Storytelling in ESL Instruction: Identity Negotiation through a Pedagogy of Multiliteracies(2011-01-01) Vinogradova, Polina; Crandall, JoAnn (Jodi); Bickel, Beverly; Language, Literacy & Culture; Literacy and CultureThis qualitative descriptive exploratory study investigated how a pedagogy of multiliteracies can be introduced to the ESL curriculum using digital stories and explored the ways ESL learners negotiate their identities through the multimodality of this narrative genre. The study was based on the premise that in language education it is crucial to account for multimodality of discourses and bring students' diverse lifeworlds and experiences into the classroom thus extending an understanding of literacy to multimodal communication and inviting ESL learners to explore their multilayered and dynamic identities. The study elicited data through focused participant observations, content analysis of students' essays, weekly journals, and final semi-structured interviews, and through discourse analysis of the students' drafts of verbal narratives for digital stories, digital storyboards, and final digital storytelling projects. The findings revealed the presence of situated practice; overt instruction; critical framing; and, in a more limited way, transformed practice. While situated practice evolved with the students narrating about their families, life-changing events, and important cultural practices, overt instruction included explicit and systematic instruction and scaffolding that fostered students' multimodal meaning making through story writing and production. Combined with situated practice it resulted in critical framing when students reflected on their progress as language learners and analyzed multimodal cultural representations in their digital stories. Some nascent examples of transformed practice were evident when students suggested how the projects had influenced their overall approaches to learning and understanding of meaning making. Students negotiated their identities through the process and product of digital stories. The study revealed that none of the participants articulated as central an ESL student identity indicating that this social role was not particularly significant at the time of the digital story project. Instead, the students were storytellers and producers, mothers and sons, daughters and sisters, granddaughters and world travelers, friends, women and men, and people with unique cultural backgrounds and social experiences. And since the students were in constant communication with each other, personal stories became experiences that moved other students and influenced their understanding of cultural diversity. This collaborative process created a multicultural classroom community of practice conducive to the recognition of diverse identities and social roles.Item Multi-modal data fusion using source separation: Application to medical imaging(IEEE, 2015-08-17) Adali, Tulay; Levin-Schwartz, Yuri; Calhoun, Vince D.The Joint ICA (jICA) and the Transposed IVA (tIVA) models are two effective solutions based on blind source separation that enable fusion of data from multiple modalities in a symmetric and fully multivariate manner. In [1], their properties and the major issues in their implementation are discussed in detail. In this accompanying paper, we consider the application of these two models to fusion of multi-modal medical imaging data—functional magnetic resonance imaging (fMRI), structural MRI (sMRI), and electroencephalography (EEG) data collected from a group of healthy controls and patients with schizophrenia performing an auditory oddball task. We show how both models can be used to identify a set of components that report on differences between the two groups, jointly, for all the modalities used in the study. We discuss the importance of algorithm and order selection as well as trade-offs involved in the selection of one model over another. We note that for the selected dataset, especially given the limited number of subjects available for the study, jICA provides a more desirable solution, however the use of an ICA algorithm that uses flexible density matching provides advantages over the most widely used algorithm, Infomax, for the problem.Item Multidataset Independent Subspace Analysis with Application to Multimodal Fusion(2019-11-11) Silva, Rogers F.; Plis, Sergey M.; Adali, Tulay; Pattichis, Marios S.; Calhoun, Vince D.In the last two decades, unsupervised latent variable models—blind source separation (BSS) especially—have enjoyed a strong reputation for the interpretable features they produce. Seldom do these models combine the rich diversity of information available in multiple datasets. Multidatasets, on the other hand, yield joint solutions otherwise unavailable in isolation, with a potential for pivotal insights into complex systems. To take advantage of the complex multidimensional subspace structures that capture underlying modes of shared and unique variability across and within datasets, we present a direct, principled approach to multidataset combination. We design a new method called multidataset independent subspace analysis (MISA) that leverages joint information from multiple heterogeneous datasets in a flexible and synergistic fashion. Methodological innovations exploiting the Kotz distribution for subspace modeling in conjunction with a novel combinatorial optimization for evasion of local minima enable MISA to produce a robust generalization of independent component analysis (ICA), independent vector analysis (IVA), and independent subspace analysis (ISA) in a single unified model. We highlight the utility of MISA for multimodal information fusion, including sample-poor regimes and low signal-to-noise ratio scenarios, promoting novel applications in both unimodal and multimodal brain imaging data.Item Multimodal Data Fusion Using Source Separation: Two Effective Models Based on ICA and IVA and Their Properties(IEEE, 2015-09) Adali, Tulay; Levin-Schwartz, Yuri; Calhoun, Vince D.Fusion of information from multiple sets of data in order to extract a set of features that are most useful and relevant for the given task is inherent to many problems we deal with today. Since, usually, very little is known about the actual interaction among the data sets, it is highly desirable to minimize the underlying assumptions. This has been the main reason for the growing importance of data-driven methods, and in particular of independent component analysis (ICA) as it provides useful decompositions with a simple generative model and using only the assumption of statistical independence. A recent extension of ICA, independent vector analysis (IVA), generalizes ICA to multiple data sets by exploiting the statistical dependence across the data sets, and hence, as we discuss in this paper, provides an attractive solution to fusion of data from multiple data sets along with ICA. In this paper, we focus on two multivariate solutions for multimodal data fusion that let multiple modalities fully interact for the estimation of underlying features that jointly report on all modalities. One solution is the joint ICA model that has found wide application in medical imaging, and the second one is the transposed IVA model introduced here as a generalization of an approach