This paper presents the microstructural topology optimization for the prescribed relaxation moduli of viscoelastic composites. The microstructures of the viscoelastic composites are assumed to be composed of periodic unit cells. A numerical homogenization method is systematically developed to obtain the effective relaxation modulus of the unit cell. The density-based topology optimization is adopted to find optimal microstructures of the visco-elastic composites. The design objective is to minimize the difference between the prescribed relaxation moduli and effective relaxation moduli at specified time instants. An adjoint sensitivity analysis is developed to compute the derivatives of the effective relaxation moduli with respect to the design variable. Several numerical examples demonstrate the effectiveness of the proposed approach. Various optimized microstructures of viscoelastic composites are presented and discussed based on the volume fraction of each constituent material, relaxation time, and stiffness.