A new adaptive thresholding algorithm concerning extraction of targets from the background in a given image sequence is proposed. The conventional histogram-based or fixed-value thresholdingsare deficient in detecting targets due to the poor contrast between targets and the background, or to the change of illumination. This research solves the problems mentioned above by learning the characteristics of the background from the given images and determines the proper thresholds based on this information. Experiments show that the proposed algorithm is superior to the optimal layering algorithm in target detection and tracking.