RadSense: Enabling one hand and no hands interaction for sterile manipulation of medical images using Doppler radar

Abstract

In this paper, we show how surgeons can interact with medical images using finger and hand gestures in two situations: one hand-free and no hands-free interaction. We explain how interaction with only one hand or a couple of fingers is beneficial and can help surgeons have continuous interaction, without the need to release their tools and leave the operating table, saving valuable patient time. To this end, we present RadSense, an end-to-end and unobtrusive system that uses Doppler radar-sensing to recognize hand and finger gestures when either one or both hands are busy. Our system permits the following important capabilities: (1) touch-less input for sterile interaction with connected health applications, (2) hand and finger gesture recognition when either one or both hands are busy holding tools, extending multitasking capabilities for health professionals, and (3) mobile and networked, allowing for custom wearable and non-wearable configurations. We evaluated our system in a simulated operating room to manipulate preoperative images using four gestures: circle, double tap, swipe, and finger click. We collected data from five subjects and trained a K-Nearest-Neighbor multi-class classifier using 15-fold cross validation, achieving a 94.5% precision for gesture classification. We conclude that our system performs with high accuracy and is useful in cases where only one hand or a few fingers are free to interact when the hands are busy.