Video probes are increasingly used in recent industrial applications of coordinate measuring machines, since they are found more efficient than conventional ball-tip probes especially in inspecting small-sized features of complex shapes. However, no thorough investigation has yet been accomplished to comprehend the measuring capabilities of video probes. In this paper, analytical and experimental approaches are made to explore how the measuring uncertainty limit of video probes is determined by major design parameters related to imaging optics, coherence of illumination, and edge detection algorithms using a CCD array. Finally an exemplary optimal design is discussed to demonstrate that an ultraprecision measurement of 0.01 μm uncertainty can be practically achieved.