ECE Distinguished Colloquium with Dr. Yi Ma, UC Berkley
Friday, January 28, 2022
3:30 p.m.-4:30 a.m.
SPRING 2022 SERIES: FRIDAYS AT 3:30 PM
"Closed-Loop Data Transcription via Minimaxing Rate Reduction"
Yi Ma, Ph.D.
Department of Electrical Engineering and Computer Sciences
The University of California, Berkeley
This work proposes a new computational framework for learning an explicit generative model for real-world datasets. More specifically,we propose to learn a closed-loop transcription between a multi-class multi-dimensional data distribution and a linear discriminativerepresentation (LDR) in the feature space that consists of multiple independent linear subspaces. We argue that the optimal encodingand decoding mappings sought can be formulated as the equilibrium point of a two-player minimax game between the encoder anddecoder. A natural utility function for this game is the so-called rate reduction, a simple information-theoretic measure for distancesbetween mixtures of subspace-like Gaussians in the feature space. Our formulation draws inspiration from closed-loop error feedbackfrom control systems and avoids expensive evaluating and minimizing approximated distances between arbitrary distributions in eitherthe data space or the feature space. To a large extent, this new formulation unifies the concepts and benefits of Auto-Encoding and GANand naturally extends them to the settings of learning both discriminative and generative representation for multi-class and multi-dimensional real-world data. Our extensive experiments on many benchmark imagery datasets demonstrate tremendous potential of thisnew closed-loop formulation: we notice that the so learned features of different classes are explicitly mapped onto approximatelyindependent principal subspaces in the feature space, and diverse visual attributes within each class are modeled by the independentprincipal components within each subspace. This work opens many deep mathematical problems regarding learning submanifolds inhigh-dimensional spaces as well as suggests potential computational mechanisms about how memory can be formed through a purelyinternal closed-loop process.
This is joint work with Xili Dai, Shengbang Tong, Mingyang Li, Ziyang Wu, Kwan Ho Ryan Chan, Pengyuan Zhai, Yaodong Yu, Michael Psenka,Xiaojun Yuan, Heung-Yeung Shum. A related paper can be found at:
Bio: Yi Ma is a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. His researchinterests include computer vision, high-dimensional data analysis, and intelligent systems. Yi received his Bachelor’s degrees in Automationand Applied Mathematics from Tsinghua University in 1995, two Masters degrees in EECS and Mathematics in 1997, and a Ph.D. degree inEECS from UC Berkeley in 2000. He has been on the faculty of UIUC ECE from 2000 to 2011, the principal researcher and manager of theVisual Computing group of Microsoft Research Asia from 2009 to 2014, and the Executive Dean of the School of Information Science andTechnology of ShanghaiTech University from 2014 to 2017. He then joined the faculty of UC Berkeley EECS in 2018. He has published about60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis. He received the NSF Career award in 2004 and the ONR Young Investigator award in 2005. He also received theDavid Marr prize in computer vision from ICCV 1999 and best paper awards from ECCV 2004 and ACCV 2009. He has served as the ProgramChair for ICCV 2013 and the General Chair for ICCV 2015. He is a Fellow of IEEE, ACM, and SIAM.