Event

While systems based on deep neural networks have produced remarkable performance for many tasks such as face/object detection and recognition, they also require large amounts of labeled training data. However, there are many applications where collecting a relatively large labeled training dataset may not be feasible due to time and/or financial constraints, like medical images or images from unfamiliar domains. But training a deep network using a small dataset usually leads to serious over-fitting issues and poor generalization.

In this work, we explore the factors that affect the generalizability of deep networks trained on small datasets. The proposed experiments are set in the following context - given a network that was trained on a large training dataset in source domain, adapt it to generalize onto a target domain using a relatively small training dataset (that is typically hundred to ten thousand times smaller). We specifically recognize three different problems with decreasing amounts of available training data: (i) semi-supervised few shot domain adaptation, (ii) semi-supervised transfer learning and, (iii) supervised domain adaptation for the task of face verification. We use the IARPA JANUS Benchmark Multi-domain Face (IJB-MDF) dataset to create our training and test splits and propose two 1:N search protocols for evaluation. Visible and short-wave infrared (SWIR) domains are chosen as the source and target domains respectively.  The SWIR target domain is further divided into 4 sub-domains based on the wavelength at which the images were captured.

For the semi-supervised few shot domain adaptation problem, we employ the Minimax Entropy (MME) algorithm to train our networks. We found that smaller architectures learn to generalize better when trained on small datasets, while larger architectures tend to overfit. Even though pre-trained Resnet-101 (trained on UMDFaces and MS-Celeb-1M datasets) outperformed pre-trained Resnet-18 network, after training with MME on the IJB-MDF data, Resnet-18 improved significantly more and outperformed Resnet-101.  We also found that when we have little training data, using all of it is not necessarily the best strategy. A network trained with only 3 samples per class performed better than a network trained with all 7 samples per class. For the supervised domain adaptation problem, we use the fine-tuning and side-tuning algorithms to train the networks. While fine-tuning Resnet-101 led to overfitting, side-tuning improved the performance. We also explore how initialization of the side-tuning parameter affects the performance.