Commercialization of organic solar cell (OSC) has faltered due to their low power conversion efficiency (PCE) compared to inorganic solar cell. Low electrical conductivity, low charge mobility, and short-range light absorption of most organic materials limit the PCE of OSCs. Carbon nanomaterials, especially carbon nanotubes (CNTs) and graphenes, are of great interest for use in OSC applications due to their high electrical conductivity, mobility, and unique optical properties for enhancing the performance of OSCs. In this review, recent progress toward the integration of carbon nanomaterials into OSCs is described. The role of carbon nanomaterials and strategies for their integration into various layers of OSCs, including the photoactive layer and charge transport layer, are discussed. Based on these, we also discuss the prospects of carbon nanomaterials for specific OSC layers to maximize the PCE.