This paper introduces an advanced Lyapunov stability analysis for an attitude observer that has been developed on the special orthogonal group. In particular, when the attitude observer is constructed based on multiple direction measurements toward known reference points, a local exponential stability has been established by linearization, under the assumption that those reference points are fixed in the inertial frame. Several modifications have been proposed to deal with reference directions changing over time. Here, we present an alternative Lyapunov analysis to show that the attitude observer still exhibits exponential stability for time-varying reference directions, under the assumption that the observer gain is sufficiently large relative to the rate of change of the reference directions. These are illustrated by a numerical example, followed by experimental results with visual marker detection in an indoor space.