Healthcare outcomes such as blood pressure and heart rate are commonly tracked across time owing to technological advances in wearable devices. This advance then makes it possible to predict health risks and to practice personalized medicine. For this type of healthcare data, it is important to reflect huge variation among subjects where the subject becomes an experimental unit. The person-specific model becomes critical for accurate prediction, but it is not optimal due to the noisy nature of the data. It has been demonstrated that sharing information across subjects via a mixed effect model can improve the prediction of individual responses compared to a completely personalized model. However, sharing information across all patients can dilute signals when there are several different patterns present in the data. That is, subjects may form groups and each group behaves differently. To reflect this feature, we extend a deep mixed effect model via a mixture of deep mixed effect models. Our mixed effect model is based on Gaussian processes where the mean adopts the deep neural networks to capture flexible time trends. Our model finds a highly nonlinear trend shared among segments of patients while clustering patients with similar trends into groups. Our approach shows great performance in simulation studies as well as real data analysis, emphasizing the importance of modeling group-specific trends when making accurate predictions from healthcare time-series data.