Event
Xintong Han - PhD Dissertation Defense
Tuesday, June 12, 2018
2:00 p.m.
4172 AVW
Melanie Prange
301 405 3686
mprange@umd.edu
Name: Xintong Han
Committee:
Professor Larry Davis, Chair
Professor David Jacobs
Professor Joseph JaJa
Professor Min Wu
Professor Rama Chellappa, Dean’s Representative
Date/time: Tuesday, June 12th, 2018, 2:00pm-4:00pm
Location: AV Williams Conference Rooms/4172
Abstract:
Deep learning is the new electricity, which has dramatically reshaped people's everyday life. In this thesis, we focus on two emerging applications of deep learning - fashion and forensics.
The ubiquity of online fashion shopping demands effective search and recommendation services for customers. To this end, we propose an automatic spatially-aware concept discovery approach using weakly labeled image-text data from shopping websites. The resulting concepts enable effective product browsing and attribute-feedback product retrieval. For fashion recommendation tasks, we study two types of fashion recommendation: (i) suggesting an item that matches existing components in a set to form a stylish outfit, and (ii) generating an outfit with multimodal specifications. We propose to jointly learn a visual-semantic embedding and the compatibility relationships among fashion items in an end-to-end fashion. In addition to searching and recommendation, customers also would like to virtually try-on the returned fashion items. We present an image-based the Try-On Network (VITON) without using 3D information in any form, which seamlessly transfers a desired clothing item onto the corresponding region of a person using a coarse-to-fine strategy.
Interestingly, VITON can be modified to swap faces instead of swapping clothing items. Conditioned on the landmarks of a face, generative adversarial networks can synthesize a target identity on to the original face keeping the original facial expression. It is worth noting that these face swapping techniques can be easily used to manipulated people's faces, and will leads serious social and political consequences.
Researchers have developed powerful tools to detect these manipulations. In this thesis, we utilize convolutional neural networks to boost the detection accuracy of tampered faces in images by using a two-stream network to capture both low-level and high-level tampering artifacts. Besides faces, spliced people are also very common in image manipulation. We describe a system containing modules modeling different aspects of manipulation traces like inconsistent noise distributions, tampered edges, etc.
