An image processing method and apparatus using a line unit operation is disclosed. The image processing apparatus includes a receiver configured to receive 5 an image, at least one first line buffer configured to output the received image to be an image line of a line unit, a first convolution operator configured to generate a feature map by performing a convolution operation based on the output image line, and a feature map processor configured to store the generated feature map as at least one line unit and process the feature map stored as the line unit to be output in a two 10 dimensional (2D) form.