This paper proposes a novel one-day-ahead photovoltaic (PV) energy prediction scheme using on-site temperatures based on ensemble of convolutional neural networks (POST-enCNN). Even meteorological information from weather station can help predict PV energy, additional information is necessary to describe on-site weather condition. Therefore, the proposed scheme takes advantage of on-site weather data along with the meteorological information to predict PV energy generation accurately. The proposed prediction process is divided into two stages. In the first stage, the day-ahead on-site temperatures are predicted based on meteorological information and the past on-site temperatures data. Specifically, incremental trend of past on-site temperatures are analyzed and transformed into a numerical values. These values are applied to ensemble of CNNs to increase prediction accuracy of the day-ahead on-site temperatures. In the second stage, the predicted day-ahead on-site temperatures are combined with meteorological information. Finally, using the long short-term memory (LSTM) based neural network, the day-ahead PV energy generation is predicted using the combined weather information. Through the simulation results, we show that the proposed POST-enCNN achieve an 27% reduction of prediction error compared to a conventional method that predicts the day-ahead PV energy generation depending solely on meteorological information.