MetaE2RL: Toward Metareasoning for Energy-Efficient Multi-Goal Reinforcement Learning with Squeezed Edge YOLO

Abstract

Metareasoning shows promise in efficiently using the computational resources of tiny edge devices while performing highly computationally intensive Reinforcement Learning (RL) algorithms. This work proposes MetaE2RL: a hardware-aware framework that incorporates low-power pre-processing solutions and metareasoning to enable the deployment of multi-goal RL on tiny autonomous devices. For this aim, a meta-level is proposed to allocate resources efficiently in real-time by switching between models with different complexities. Moreover, Squeezed Edge YOLO is proposed for energy-efficient object detection in the pre-processing phase. For the experimental results, the proposed Squeezed Edge YOLO was deployed onboard a tiny drone named Crazyflie with GAP8 processor that includes 8 parallel RISC-V cluster cores. We compared latency and power consumption of Squeezed Edge YOLO and a lighter CNN-based model while deploying them separately onboard on GAP8. Experimental results show Squeezed Edge YOLO is 8x smaller than previous work and consumes 541 mW on GAP8 with inference latency of 130 ms.