Kernel-Based Lifelong Multitask Multiview Learning

Abstract

Lifelong learning capitalizes on the shared skill structure present in a stream of tasks that arrive over time to improve upon the performance of single-task learners. In contemporary lifelong learning applications, it is often the case that there are multiple sensing modalities or views associated with each task. A crucial aspect in lifelong multitask multiview learning is to capture not only the shared structure among the tasks but also across views effectively. In this work, a nonparametric kernel-based learning framework is adopted to model even nonlinear shared structures in the tasks and views in a flexible and robust way. An efficient lifelong learning formulation is derived by judicious approximation of the per-task learning objectives, based on which the shared skill libraries can be updated online in function space. Numerical tests verify the efficacy of the proposed approach.