Despite advances in predicting physical peptide-major histocompatibility complex I (pMHC I) binding, it remains challenging to identify functionally immunogenic neoepitopes, especially for MHC II. By using the results of > 36,000 immunogenicity assay, we developed a method to identify pMHC whose structural alignment facilitates T cell reaction. Our method predicted neoepitopes for MHC II and MHC I that were responsive to checkpoint blockade when applied to > 1,200 samples of various tumor types. To investigate selection by spontaneous immunity at the single epitope level, we analyzed the frequency spectrum of > 25 million mutations in > 9,000 treatment-naive tumors with > 100 immune phenotypes. MHC II immunogenicity specifically lowered variant frequencies in tumors under high immune pressure, particularly with high TCR clonality and MHC II expression. A similar trend was shown for MHC I neoepitopes, but only in particular tissue types. In summary, we report immune selection imposed by MHC II-restricted natural or therapeutic T cell reactivity.