We propose a novel gesture recognition method, “EarHover,” which enables mid-air gesture input for hearables. This method focuses on the phenomenon of sound leakage specific to hearables and utilizes speakers and external microphones, which are highly compatible with hearables, for gesture recognition. From 27 effective gestures, we selected 7 gestures based on signal discriminability and user acceptability (N=10). Subsequently, data for these 7 gestures and a neutral state, totaling 8 states, were collected using two prototype devices: in-ear and over-ear types. The recognition performance using deep learning was then evaluated (N=13). The experimental results showed F-scores of 78.7%/73.4% for in-ear/over-ear devices, respectively. Furthermore, by narrowing down to 5 gestures, the F-score improved to 86.2%/82.5% for the 6 states.
Publication
鈴木俊汰，雨坂宇宙，渡邉拓貴，志築文太郎，杉浦裕太, EarHover：ヒアラブルデバイスにおける音漏れ信号を用いた空中ジェスチャ認識, 情報処理学会インタラクション2024/IPSJ Interaction 2024，INT24011, pp. 88-97, 2024.[PDF]