A system that recognizes light hand gestures is necessary to improve the usability of wearable devices. Thus, this paper proposes a real-time system that classifies the bio-acoustic signals of three light finger tap gestures done by the device-wearing hand using the index (index-tap), middle (middle-tap) and ring (ring-tap) which lightly tap the thumb of the same hand. The system detects body conducted acoustic vibrations that occur from the finger gestures of a user. It classifies the gestures using the frequency information of the signal and a machine learning technique. The accuracy of the system was evaluated with a real-time application for two simulated situations. The two situations were when a user uses the system for the first time and when a user uses the system continually. The system has an accuracy of about 80% for both the situations. The middle-tap gesture had the lowest accuracy. Finally, this paper discusses possible factors that decreases the accuracy of the system and further research directions.