As the popularization of augmented human, the influence of computing devices on human life is growing. We are facing lots of service in daily life, and it will be more in future. As a result, the border between real and virtual life is vanishing. To manage and maintain yourself in the flood of information, we need to know yourself in detail and quickly. For this reason, there were several research studies about the Quantified Self. However, the studies and applications of QS were held in separately by focusing each limited information, and it arises limitation on understanding in bird-eye view. To address these issues, we propose the unified concept of the Holistic Quantified Self (HQS) with its framework. HQS consists of the quantified indexes of user status which includes physical, emotional, social, and digital behavioral state. By collecting and quantifying the user’s multi-modal sensor data, HQS can be generated. Then, interpreting the pattern of HQS could be utilized to provide customized services to users. HQS framework describe an overall process to ensure the personalized service. Through these study, we expect to fulfill augmented human which enhances user abilities and contribute quality of social life through the development of the personal companion.