Recently High Performance Computing (HPC) users are interested in Cloud computing as an alternative to supercomputers and General Purpose Graphical Processing Unit (GPGPU) is now crucial part in HPC area. To provide GPGPU in Cloud computing for HPC, we suggest GPGPU HPC Cloud platform based on OpenStack. We combined OpenStack, KVM, and rCUDA to enable scalable use of GPUs among virtual machines. To use GPU resource more efficiently, we propose centralized, distributed GPU resource schedulers on Cloud computing, mainly considering GPU resource usage, Cloud network topology and traffic. Finally, we evaluated performance of our GPGPU HPC Cloud platform. Compared to Cloud platform with PCI pass-through, speedup ratio was 0.93 to 0.99 on compute intensive programs and 0.26 to 0.89 on data intensive programs.