Augmenting Around-Device Interaction by Geomagnetic Field Built-in Sensor Utilization

Ljubic, Sandi; Hržić, Franko; Salkanovic, Alen; Štajduhar, Ivan

In this paper, we investigate the possibilities for augmenting interaction around the mobile device, with the aim of enabling input techniques that do not rely on typical touch-based gestures. The presented research focuses on utilizing a built-in magnetic field sensor, whose readouts are intentionally affected by moving a strong permanent magnet around a smartphone device. Different approaches for supporting magnet-based Around-Device Interaction are applied, including magnetic field fingerprinting, curve-fitting modeling, and machine learning. We implemented the corresponding proof-of-concept applications that incorporate magnet-based interaction. Namely, text entry is achieved by discrete positioning of the magnet within a keyboard mockup, and free-move pointing is enabled by monitoring the magnet’s continuous movement in real-time. The related solutions successfully expand both the interaction language and the interaction space in front of the device without altering its hardware or involving sophisticated peripherals. A controlled experiment was conducted to evaluate the provided text entry method initially. The obtained results were promising (text entry speed of nine words per minute) and served as a motivation for implementing new interaction modalities. The use of neural networks has shown to be a better approach than curve fitting to support free-move pointing. We demonstrate how neural networks with a very small number of input parameters can be used to provide highly usable pointing with an acceptable level of error (mean absolute error of 3 mm for pointer position on the smartphone display).