The Skip mode of H.264/AVC is a coding mode reconstructed only by the prediction signal, and is the coding mode having the smallest bits compared to other modes. The Skip mode cannot deal with the occurrence of illumination change at the current block because it is reconstructed only by the prediction signal. Also, the Skip mode is the coding mode having the largest distortion compared to other modes. To solve the illumination change problem of the Skip mode and to reduce the distortion of the Skip mode, we propose a novel offset compensation method, which applies the estimated offset compensation to the Skip mode only. In the proposed method, flag bit is not used for the offset because the incorrectly estimated offsets are mostly filtered out through the rate distortion optimization based mode decision process. Since only a semantic change is required for the Skip mode in the proposed method, the proposed Skip with offset mode (SWO) can be easily replaced with the Skip mode of H.264/AVC. The proposed SWO method shows better coding performance than previous methods, including the weighted prediction (WP) method of H.264/AVC.