SOUND SKETCHBOOK: SYNTHETIC SYNESTHESIA ON A MOBILE PLATFORM

Abstract

The authors present Sound Sketchbook, a mobile phone application featuring real-time sound synthesis based on simple yet evocative cross-modal data mappings. While originally designed as a tool for evaluation of audiovisual correspondences, the application is also appreciated as an enjoyable sound toy and has a strong potential as a multimedia education tool for children. The authors introduce the data mapping strategy of Sound Sketchbook with regard to synesthesia, describe new cross-modal interactions implemented on mobile devices, and discuss the effectiveness of the application based on user survey results.