Use the Force: Incorporating Touch Force Sensors into Mobile Music Interaction

Abstract

The musical possibilities of force sensors on touchscreen devices are explored, using Apple’s 3D Touch. Three functions are selected to be controlled by force: a) excitation, b) modification (aftertouch), and c) mode change. Excitation starts a note, modification alters a playing note, and mode change controls binary on/off sound parameters. Four instruments are designed using different combinations of force-sound mapping strategies. ForceKlick is a single button instrument that plays consecutive notes within one touch by altering touch force, by detecting force down-peaks. The iPhone 6s/7 Ocarina features force-sensitive fingerholes that heightens octaves upon high force. Force Trombone continuously controls gain by force. Force Synth is a trigger pad array featuring all functions in one button: start note by touch, control vibrato with force, and toggle octaves upon abrupt burst of force. A simple user test suggests that adding force features to well-known instruments are more friendly and usable.