Event
Ph.D. Dissertation Defense: Yan Zhang
Thursday, July 31, 2025
10:00 a.m.
AVW 2460
Souad Nejjar
301 405 8135
snejjar@umd.edu
ANNOUNCEMENT: Ph.D. Dissertation Defense
Name: Yan Zhang
Committee:
Professor Shuvra Bhattacharyya (Chair/Advisor)
Professor Manoj Franklin
Professor Ang Li
Doctor Heesung Kwon
Professor Abhinav Shrivastava (Dean's representative)
Date/time: Thursday, July 31, 2025 at 10:00 AM
Location: AVW 2460
Title: Design Optimization for Resource-Constrained Deep Learning
Abstract:
Deep learning has achieved great success across numerous application areas in recent years, driven by major factors that include the continuous development of advanced models, and advances in hardware and software technologies for efficient execution of training and inference. Deep learning models are very computationally intensive and have large memory requirements, often involving millions of parameters or more, which need to be stored and operated on. At the same time, there is growing interest in migrating deep learning functionality toward the network edge, where operation has the potential to provide significant improvements in concerns that include latency, security, and privacy. Efficiently deploying deep learning models on edge-targeted platforms, which are highly constrained in terms of processing resources and memory, has therefore become a critical challenge. Additionally, the training process for complex deep learning models requires a substantial amount of data, and realistic, annotated data that is suitable for model training can be viewed as another type of constrained resource in many applications. In particular, labeled, real-world data is often difficult to obtain in sufficient quantities for producing highly accurate models.
This dissertation focuses on the optimization of deep learning systems that must be developed under stringent resource constraints. Two specific themes are pursued: (1) the deployment of models on edge computing platforms that have limited resources for processing and data storage, and (2) the development of methods to utilize synthetic data as a substitute for real data in deep learning system with the scarcity of available real data resources.
In the first part of this dissertation, we introduce a design space exploration framework based on dataflow techniques, integrated with a multi-objective particle swarm optimization strategy, to efficiently evaluate implementation trade-offs and derive Pareto-optimized configurations for deployment on lightweight edge computing platforms. The framework accommodates diverse model architectures and optimization strategies that are relevant to various types of edge computing platforms.
In the second part of this dissertation, we investigate the potential of synthetic data to significantly augment real data in aerial-view human detection applications when real training data is difficult to obtain. In our study, we focus on the effect of two key factors: the number of cross-domain real images and the characteristics of the test set being evaluated. We also introduce two new metrics that are designed to measure the representational ability of a training dataset, where the training dataset in question in general contains a mix of real and synthetic data.
The third part of this dissertation delves deeper into the effects of synthetic data by exploring the domain gap between synthetic and real data. We examine three key factors that impact the domain gap, including the features of the real data, the synthetic data selected for training, and the synthetic data pool from which the incorporated synthetic data is selected. Through a comprehensive set of experiments in both human detection and semantic segmentation scenarios, we draw insights that can benefit the overall study of synthetic data usage in a broader range of applications.
The fourth part of this dissertation examines the problem of resource-constrained speech emotion recognition (SER). This part specifically studies models for SER that employ a continuous representation of emotions. We analyze the effects of various fine-tuning strategies, explore models with different complexities, and investigate the impact of separate training for different emotion dimensions. The results quantify trade-offs between accuracy and computational cost, and offer insights into the application of SER under varying resource constraints.
