Automatic polyp detection in endoscopy (or colonoscopy) images is challenging because the types of polyp and their appearances are diverse, and the colors and textures of polyps are quite similar to those of normal tissues in many cases. It is thus often very difficult to distinguish polyps from normal tissues using conventional methodology. To effectively resolve these challenges, we propose a framework based on multi-classifier learning and a contour intensity difference (CID) measure. To detect polyps of diverse appearances, we first classify polyps into K types according to their shape via unsupervised learning. We then train K classifiers to detect the K types of polyp. This multi-classifier learning improves the polyp detection rate. However, false positives also increase because colon structures look similar to polyps. To reduce false positives while preserving the high detection rate, we propose a CID measure. Experimental results using public and our own datasets show that the proposed methods are promising for detecting polyps with diverse appearances.