Reg-TuneV2: Hardware-Aware and Multi-Objective Regression-Based Fine-Tuning Approach for DNNs on Embedded Platforms

Abstract

Fine-tuning Deep Neural Networks (DNNs) for deployment has traditionally relied on computationally intensive methods such as grid search and neural architecture search (NAS), which may not consider hardware-aware metrics. Moreover, it is essential to consider multiple objectives to develop a range of solutions for tinyML hardware deployment with real-time latency and low power constraints. To address these problems, we propose Reg-TuneV2, a systematic approach to fine-tune DNNs for hardware deployment by considering multiple objectives, including accuracy, power, and latency contours. In addition, this approach uses metric learning to achieve smaller and better-suited configurations for deployment, achieving 90.5% accuracy with only 340 KB of memory for keyword spotting on FPGA. When compared to baselines for keyword spotting and image classification on the Nvidia Jetson Nano 4 GB SDK, the proposed method achieves a 14.5× and 101.8× reduction in model size coupled with 2.5× and 5.9× better inference efficiency, respectively.