For contextual evaluation of context-aware mobile phone applications, it is desired to observe user interaction with them and collect user feedback about them in situ. For this purpose, we designed and built an experience sampling system (ESS) for mobile phones. The resulting ESS is characterized by the following features: event-driven sampling of user context and feedback, real-time monitoring of user experience, and offline tools for data analysis. We conducted user experiments to confirm that the ESS was effective for evaluating mobile phone applications. We report in this paper the design considerations and the implementation detail of the ESS as well as the results of the user experiments with the ESS.
1st International Conference on Design, User Experience and Usability: Theory, Methods, Tools and Practice, DUXU 2011, Held as Part of 14th International Conference on Human-Computer Interaction, HCI International 2011, pp.648 - 657