In this paper, an improved scaled-decoding (defined as residue-to-binary conversion of which outputs are scaled by a constant) technique for hardware implementations in the residue number system (RNS) is presented. The technique is based on the Chinese remainder theorem (CRT) and is equipped with a rounding error compensation circuit that can reduce the maximum scaling error below 0.5 least-significant-bit (LSB) for general four-moduli RNS, and down to about 0.53 LSB for general six-moduli RNS with a moderate amount of additional hardware.