Since the introduction of a fast Fourier transform (FFT)-based convolution scheme between an impulse train and a ray spread function for the inverse synthetic aperture radar (ISAR) image formation using shooting and bouncing ray (SBR), remarkable progress has been made in reducing calculation time. However, because of an interpolation algorithm, which is essential to the FFT process, it necessarily causes the distortion of image. The present work is an attempt to generate an accurate image without loss of calculation time efficiency. Specifically, by removing dominant interpolation error contribution from the prior ray spread function, the distortion is prevented without the use of the time-consuming higher-order interpolation or high oversampling. The proposed scheme is simulated for several 3-D CAD models and demonstrated by comparing the image using the proposed and existing scheme under the same conditions.