This paper describes a system that traces handwriting
on paper document under overlooking video camera. This
work is motivated to capture annotations on paper
documents written by ordinary pen as an input to computer.
As the trajectory of the pen tip is extracted from the video,
each part of the trajectory is classified as ‘pen-down’ or
‘pen-up’, according to whether the part makes a dark line.
Detecting written inks is not simple when handwriting is
made over printed documents. Because written inks may
fall on dark regions of the document and often overlap
previously written inks, simple background checking may
not work on dark regions. So, we interpolated the
decisions at the entering and the exiting of the dark region.
The system makes two-level decisions to achieve both
speed and accuracy. The classifier makes quick decisions
based on local information in order not to lose pen trace.
The local pen up-down decisions are corrected in the
global point of view when the whole information of the
writing process is available, such as when the hand is out
of the view. Experimental result shows that the system
detects handwritings accurately even on printed documents.