Recently, hafnia-based ferroelectrics are currently being investigated as next-generation memory devices due to their excellent CMOS process compatibility and functionality. However, some of the ferroelectric devices commonly exhibit an imprint effect due to charged defects around the interfacial layer, which has negative impacts on the ferroelectric memory devices. However, it can be applied to various applications as long as the imprint field is carefully adjusted. In this work, we introduced a strategy to control the imprint field in bilayer capacitors by utilizing tantalum oxide (TaO) interfacial layers and various Zr contents in Hf0.83Zr0.17O2 (HZO) films. The TaO layer (1 nm) was inserted into the bilayer capacitors to alter the imprint field's (positive or negative) direction. Whereas to control the imprint fields, we adjusted the Zr doping content (17%-83%) in the ferroelectric HZO films (8 nm). As the Zr content increased, reduced imprint fields were observed in those bilayer capacitors. In addition, it was found that a high imprint field (+2.43 MV/cm) was observed in Hf-rich films (Hf0.83Zr0.17O2) due to the higher amount of oxygen vacancies. In addition, we examined those oxygen vacancies through x-ray photoelectron spectroscopy depth profile analysis by considering sub-oxide fractions in the tantalum, which further confirms the root cause of the imprint field variations in the bilayer capacitors. Our study will contribute to a deeper understanding of imprinted hafnia-based ferroelectrics and will provide an insight into devices that utilize the imprint effect.