In this paper, we present a recursive modified Gram-Schmidt (RMGS) algorithm for least-squares (LS) linear phase filters to allow for the tracking of time-varying parameters. We examine both exponentially windowed and sliding window covariance cases, including the prewindowed case as a special case of the exponentially windowed one. We describe quantitatively the performance characteristics of the RMGS filters for the problem of linear phase system identification when the unknown system parameters vary with time.