Clark School Home UMD

Booz Allen Hamilton Colloquium: Dr. Trac Tran, Professor, Johns Hopkins Whiting School of Engineering

Entropy Minimization for Sparse Recovery, Low-rank Approximation and Representative Selection


With an ever-increasing amount of data being produced each day in almost every aspect of our lives, the challenges we face nowadays are not just how to sense, store and communicate data efficiently, but also how we can quickly analyze, learn, and make intelligent decisions from the data we have collected. Fortunately, our signals of interest are often inherently sparse in certain bases or dictionaries where they can be approximately represented by only a few significant components carrying the most relevant information. In other words, the intrinsic signal information usually lies in a low-dimensional subspace and the semantic information is often encoded in the sparse representation. This talk revisits the earlier concept of entropy in information theory trying to capture sparsity in data. We show that minimizing entropy does promote sparse solutions and demonstrate its advantages over the conventional l1-minimization approach with noiseless and noisy signal recovery experiments. We also illustrate a straightforward extension of this entropy-minimization approach to the problems of low-rank matrix recovery and matrix completion. This direction naturally leads to an improved version of Robust Principal Component Analysis (RPCA) where entropy minimization is employed to compactly cluster both low-rank components as well as outliers. Next, we extend this concept to a related dictionary learning problem: how to select a few yet most informative representatives of the data? This question from an entropy angle leads to new insights on non-negative matrix factorization and its applications in data clustering, classification, and document/video summarization. Finally, if time permits, we demonstrate that components discussed in this talk can be connected to form a deep multi-layer sparse coding network.


Trac D. Tran received the B.S. and M.S. degrees from the Massachusetts Institute of Technology, Cambridge, in 1993 and 1994, respectively, and the Ph.D. degree from the University of Wisconsin, Madison, in 1998, all in Electrical Engineering. In July of 1998, Dr. Tran joined the Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, where he currently holds the rank of Professor. His research interests are in the field of digital signal processing, particularly in sparse representation, sparse recovery, sampling, multi-rate systems, filter banks, transforms, wavelets, and their applications in signal analysis, compression, processing, and communications. His pioneering research on integer-coefficient transforms and pre-/post-filtering operators has been adopted as critical components of Microsoft Windows Media Video 9 and JPEG XR – the latest international still-image compression standard ISO/IEC 29199-2. Dr. Tran received the NSF CAREER award in 2001, the William H. Huggins Excellence in Teaching Award from The Johns Hopkins University in 2007, and the Capers and Marion McDonald Award for Excellence in Mentoring and Advising in 2009. He is an IEEE Fellow for contributions in multi-rate and sparse signal processing.