Takashi Amsaka

We propose a novel gesture recognition method, “EarHover,” which enables mid-air gesture input for hearables. This method focuses on the phenomenon of sound leakage specific to hearables and utilizes speakers and external microphones, which are highly compatible with hearables, for gesture recognition. From 27 effective gestures, we selected 7 gestures based on signal discriminability and user acceptability (N=10). Subsequently, data for these 7 gestures and a neutral state, totaling 8 states, were collected using two prototype devices: in-ear and over-ear types. The recognition performance using deep learning was then evaluated (N=13). The experimental results showed F-scores of 78.7%/73.4% for in-ear/over-ear devices, respectively. Furthermore, by narrowing down to 5 gestures, the F-score improved to 86.2%/82.5% for the 6 states.

Publication

Shunta Suzuki, Takashi Amesaka, Hiroki Watanabe, Buntarou Shizuki, and Yuta Sugiura. 2024. EarHover: Mid-Air Gesture Recognition for Hearables Using Sound Leakage Signals. In The 37th Annual ACM Symposium on User Interface Software and Technology (UIST ’24), October 13–16, 2024, Pittsburgh, PA, USA. ACM, New York, NY, USA, 13 pages. [DOI]

鈴木俊汰，雨坂宇宙，渡邉拓貴，志築文太郎，杉浦裕太, EarHover：ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識, 情報処理学会インタラクション2024/IPSJ Interaction 2024，INT24011, pp. 88-97, 2024.[PDF]