Reg-Tune: A Regression-focused Fine-tuning Technique for Energy-efficient Embedded Deployment

Abstract

Fine-tuning deep neural networks (DNNs) is essential for developing inference modules suitable for deployment on edge or FPGA (Field Programmable Gate Arrays) platforms. Traditionally, various parameters across DNN layers have been explored using grid search, neural architecture search (NAS), and other brute-force techniques. While these methods yield optimal network parameters, the search process can be time-consuming and may not account for the deployment constraints of the target platform. The dissertation addresses this issue by formulating hardware-aware regression polynomials for the energy-efficient deployment of DNN models through Reg-Tune. A general formulation is provided to profile different metrics across various device platforms for both single and multi-objective solutions. The objective is to ascertain hardware friendly configurations based on the experience span for a limited set of configurations derived from a combination of variables in terms of accuracy, power, and latency. Once the regression fits are established for the metrics of interest, single or multi-objective contours related to these metrics can be created to analytically and experimentally identify the near-optimal solution for deployment. The dissertation also demonstrates that introducing LIME (Local Model Agnostic Explanations)-based sample weights into the training framework or employing metric learning in conjunction with the Reg-Tune method can enhance the inference accuracy of deployed models without altering network parameters. Furthermore, the LIME-based weighting method is shown to be effective for handling unbalanced datasets and mitigating catastrophic forgetting in incremental learning. Additionally, different metrics are profiled across two different device platforms and for a variety of applications, thereby increasing the applicability of the method. For instance, deployments based on the fine-tuning method for physical activity recognition on FPGA demonstrate at least 5.7x better energy efficiency than recent implementations without compromising accuracy. On the other hand, when combined with weighted loss-based training, the Reg-Tune approach identifies a deployment configuration with approximately 8x better energy efficiency than recent keyword spotting implementations on FPGA. Finally, the integration of metric learning with Reg-Tune results in a 14.5x and 101.8x reduction in model size, alongside 2.5x and 5.9x improvements in inference efficiency for keyword spotting and image classification on the Nvidia Jetson Nano 4GB SDK, compared to baseline and recent implementations, respectively.