The conventional image-based visual servoing leads to image singularities that might cause control instabilities. To avoid this problem, in this paper, the image moments are used as features for visual servoing, where the Jacobian matrix is full rank and upper triangular. Thus, it has the maximal decoupled structure and simplifies the controller. The general analytical form of the interaction matrix or the Jacobian matrix considering the camera parameters related to any image moments is derived in this paper. As a servoing controller, an optimal visual PD controller is presented to improve the performance of the visual servoing system instead of the P controller, which is the method extensively used in visual servoing. A genetic algorithm-based PD parameters tuning method is applied to obtain the optimal parameters. The method proposed is used to align the micropeg and hole, and the simulation results show that the object can reach its desired position faster and more smoothly.