Crowdsourced Labeling for Worker-Task Specialization Model

Abstract

We consider crowdsourced labeling under a d-type worker-task specialization model, where each worker and task is associated with one particular type among a finite set of types and a worker provides a more reliable answer to tasks of the matched type than to tasks of unmatched types. We design an inference algorithm that recovers binary task labels (up to any given recovery accuracy) by using worker clustering, worker skill estimation and weighted majority voting. The designed inference algorithm does not require any information about worker/task types, and achieves any targeted recovery accuracy with the best known performance (minimum number of queries per task). 11This work was supported in part by National Research Foundation of Korea under Grant 2017R1E1A1A01076340; in part by the Ministry of Science and ICT, South Korea, under the ITRC support program under Grant IITP-2021-2018-0-01402; and in part by the Institute of Information and Communications Technology Planning Evaluation (IITP) grant funded by the Korea Government MSIT under Grant 2020-0-00626.