Characterization of masses in computer-aided detection systems for digital breast tomosynthesis (DBT) is an important step to reduce false positive (FP) rates. To effectively differentiate masses from FPs in DBT, discriminative mass feature representation is required. In this paper, we propose a new latent feature representation boosted by depth directional long-term recurrent learning for characterizing malignant masses. The proposed network is designed to encode mass characteristics in two parts. First, 2D spatial image characteristics of DBT slices are encoded as a slice feature representation by convolutional neural network (CNN). Then, depth directional characteristics of masses among the slice feature representations are encoded by the proposed depth directional long-term recurrent learning. In addition, to further improve the class discriminability of latent feature representation, we have devised three objective functions aiming to (a) minimize classification error, (b) minimize intra-class variation within the same class, and (c) preserve feature representation consistency in a central slice. Experimental results have demonstrated that the proposed latent feature representation achieves a higher level of classification performance in terms of receiver operating characteristic (ROC) curves and the area under the ROC curve values compared to performance with feature representation learned by conventional CNN and hand-crafted features.