In this research, the amorphous phase of $HfSiO_4$ is generated using the molecular dynamics simulation based on the first-principles calculations, and the atomic and electonic structures are investigated. The local bonding structures of the O atoms can be classified as the $SiO_2$-like, mixed, and $HfO_2$-like bonds by the number of Hf and Si atoms in the neighborhood. The bond lengths of Si-O and Hf-O are about 1.66 and 2.14 $\AA$, respectively and the coordination numbers are evaluated to about 4.02, 2.66, and 6.25 for Si, O, and Hf atoms, respectively. O-vacancy in a-$HfSiO_4$ can be also categorized by the number of Hf and Si atoms near the vacancy site : the Si-related, mixed, and Hf-related defects. For the Si- and Hf-related defects, the defect levels are about 0.56 and 2.59 eV, and the formation energies are about 4.89 and 6.12 eV, espectively. From the analysis of the defect levels of O-vacancies, the threshold voltage shift mechanism in p-channel metal-oxide-semiconductor (PMOS) devices can be explained, and the Hf-related defects mainly play the critical role in that mechanism.