Event
Ph.D. Research Proposal Exam: Yan Zhang
Thursday, December 19, 2024
1:00 p.m.
(Remote) https://umd.zoom.us/j/93628796512
Maria Hoo
301 405 3681
mch@umd.edu
ANNOUNCEMENT: Ph.D. Research Proposal Exam
Name: Yan Zhang
Committee:
Professor Shuvra S. Bhattacharyya (Chair)
Professor Manoj Franklin
Professor Ang Li
Date/time: Thursday, December 19 2024 at 1PM-3PM
Location: (Remote)
Title: Design Optimization for Resource-Constrained Deep Learning
Abstract:
Deep learning has achieved great success across numerous application areas in recent years, driven by major factors that include the continuous development of advanced models, and advances in hardware and software technologies for efficient execution of training and inference. Deep learning models are very computationally intensive and have large memory requirements, often involving millions of parameters or more, which need to be stored and operated on. At the same time, there is growing interest in migrating deep learning functionality toward the network edge, where operation has the potential to provide significant improvements in concerns that include latency, security, and privacy. Efficiently deploying deep learning models on edge-targeted platforms, which are highly constrained in terms of processing resources and memory, has therefore become a critical challenge. Additionally, the training process for complex deep learning models requires a substantial amount of data, and realistic, annotated data that is suitable for model training can be viewed as another type of constrained resource in many applications. In particular, labeled, real-world data is often difficult to obtain in sufficient quantities for producing highly accurate models.
This research proposal focuses on the optimization of deep learning systems that must be developed under stringent resource constraints. Two specific themes that are pursued in the proposal are: (1) the deployment of models on edge computing platforms that have limited resources for processing and data storage, and (2) the development of methods to utilize synthetic data as a substitute for real data in deep learning system with scarcity in the available real data. In connection with the first theme, we introduce a design space exploration framework based on dataflow techniques, integrated with a multi-objective particle swarm optimization strategy, to efficiently evaluate implementation trade-offs and derive Pareto-optimized configurations for deployment on lightweight edge computing platforms. The framework accommodates diverse model architectures and optimization strategies that are relevant to various types of edge computing platforms. In connection with the second theme of the proposal, we investigate the potential of synthetic data to significantly augment real data in aerial-view human detection applications when real training data is difficult to obtain. In our study, we focus on the effect of two key factors: the number of cross-domain real images and the characteristics of the test set being evaluated. We also introduce two new metrics that are designed to measure the representational ability of a training dataset, where the training dataset in question in general contains a mix of real and synthetic data.
Our proposed future work includes extending the investigation of the effects of synthetic data when applied to training in more general scenarios. To gain a deeper understanding of the effects of synthetic data, we need to explore the domain gap between synthetic and real data, which is influenced by several factors, including the features of the real data, the synthetic data selected for training, and the synthetic data pool from which the incorporated synthetic data is selected. We plan to delve deeper into these factors through design, analysis, and interpretation of a comprehensive set of experiments. To draw broader conclusions that can benefit the overall study of synthetic data usage, we also intend to investigate a wider range of applications, not limited to aerial-view human detection scenarios.
Second, we propose to investigate the integrated application of graph neural networks and elastic network design methods to systematically optimize trade-offs between accuracy and real-time performance. We plan to carry out this investigation specifically in the context of mapping aerial-based object detection applications onto platforms with limited processing resources, continuing our line of contributions that are relevant to efficient and accurate object detection from aerial-view footage. Elastic neural networks incorporate early exit points to provide alternative trade-offs between accuracy and speed for real-time inference. Such flexibility enables dynamic adjustment of exit points and also introduces new possibilities for combining results from multiple predictions. Given that graph neural networks (GNNs) are frequently used to capture relationships between objects, we plan to integrate GNNs into the framework of object detection, and study the problem of multiobjective design optimization for GNN- and elasticity-integrated system design.
