Precise timing synchronization of an ultrafast electron diffraction facility is essential requirement for femtosecond resolution structure analysis. Recent studies of THz-based electron deflectors have enabled the timing drift measurement between ultrafast electrons and an optical pump beam with few femtosecond resolution [1]. In this work, we will introduce timing synchronization activities to suppress the drift of an electron beam. As timing drift of the electron beam originates from every sub-element, each timing drift contribution from RF transfer, RF-to-optical synchronization, and optical amplification is measured. Timing drift of RF transfer through coaxial cable, which exposed to temperature fluctuation, is actively stabilized from 2 ps to 50 fs by active feedback loop. Further additive drift from RF-to-optical synchronization is maintained below 100 fs. Also optical drift due to the regenerative amplifier, measured by optical correlator, is maintained below 20 fs over an hour. This work allows ultrafast electron diffraction system to operate with less drift correction procedure and increased user availability.