Autonomous driving has become a major goal of automobile manufacturers and an important driver for the vehicular technology. Hybrid electric vehicles (HEVs), which represent a trade-off between conventional internal combustion engine (ICE) vehicles and electric vehicles (EVs), have gained popularity due to their high fuel economy, low pollution, and excellent compatibility with the current fossil fuel dispensing and electric charging infrastructures. To facilitate autonomous driving, an autonomous HEV controller is needed for determining the power split between the powertrain components (including an ICE and an electric motor) while simultaneously managing the power consumption of auxiliary systems (e.g., air-conditioning and lighting systems) such that the overall electromobility is enhanced. Certain (partial) prior knowledge of the future driving profile is useful information for the automatic HEV control. In this paper, methods for predicting driving profile characteristics to enhance HEV power control are first presented. Based on the prediction results and the observed HEV system state (e.g. velocity, battery state-of-charge, propulsion power demand), we propose a reinforcement learning method to determine the power source split between the ICE and electric motor while also controlling the power consumptions of the air-conditioning and lighting systems in the automobile. Experimental results demonstrate significant improvement in the overall HEV system efficiency.