We study additive higher Chow groups with several modulus conditions. Apart from exhibiting the validity of all known results for the additive Chow groups with these modulus conditions, we prove the moving lemma for them: for a smooth projective variety X and a finite collection W-o of its locally closed algebraic subsets, every additive higher Chow cycle is congruent to an admissible cycle intersecting properly all members of W-o times faces. This is the additive analogue of the moving lemma for the higher Chow groups studied by S. Bloch and M. Levine. As an application, we prove that any morphism from a quasiprojective variety to a smooth projective variety induces a pull-back map of additive higher Chow groups. More important applications of this moving lemma are derived in two separate papers by the authors.