Exprgram: A Video-based Language Learning Interface Powered by Learnersourced Video Annotations

Abstract

Foreign language learners are challenged to master pragmatic competence, the ability to use language in a contextually appropriate way. While a large number of language learning materials are accessible, they are often optimized for developing linguistic components (e.g., vocabulary and grammar). In this research, we turn to videos in foreign language as an underutilized source of real-life situations with rich contexts. To effi-ciently learn from diverse situations through videos, learners should be able to access relevant videos that share a context or an expression. We introduce Exprgram, a language learning interface that utilizes videos at scale to enable context- and expression-based browsing. To enable such browsing, contexts or semantically related expressions in videos should be annotated at scale. Exprgram combines crowdsourcing and machine learning to acquire the needed annotations. Specifi- cally, we introduce a learnersourcing workflow that harvests and organizes video annotations of highly contextual data and relevant expressions to improve our browsing system. Results of a pilot study show that Exprgram helps participants learn diverse expressions in a given context and generate reliable artifacts for future learners.