Most video coding standards use the discrete cosine transform, known to be near optimal for original images, to transform prediction errors. Since the statistical characteristics of prediction errors are quite different from those of original images, a more suitable transform for prediction errors has to be devised. In this letter, we introduce a novel statistical model for inter-frame prediction error and propose an adaptive transform based on the model. In addition, in order to reduce the computation time, a fast and efficient algorithm is developed. Experiments on well-known image sequences confirm that our proposed transform can improve the performance of transform coding significantly.