From Hearing to Feeling:Unlocking Through-Skin Acoustic Sensing on Smartphones

Abstract

Sound has connected people across distance for centuries, carrying voices, music, and emotion through the air. In the digital age, it connects people with their smartphones, bridging human communication and computational sensing. Today, smartphones use sound not only to transmit information but also to sense their surroundings. For example, voice assistants respond to spoken commands [10, 19, 21], while acoustic sensing techniques enable smartphones to detect hand gestures [1, 27], finger movements [13, 14], and even subtle physiological and behavioral signals such as respiration [3, 15], eye blinks [4, 16], and heartbeats [20, 26]. Together, these capabilities have transformed sound into a versatile sensing modality that allows smartphones to perceive and interpret the physical world.