WLAN fingerprinting has been extensively studied due to the widespread deployment of WLAN infrastructure. However, extensive calibration effort is required to collect fingerprints as are necessary for radio map construction, which is time-consuming and labor-intensive. Although many studies have been conducted to reduce the calibration efforts, it is inevitable to be applied in limited circumstances because it requires some location labels for initializing the learning models or requires continuous use of sensor data. In this research, we introduce a practical radio map construction method in a multi-story building utilizing only limited uses of sensors. The method determines the unlabeled data of each floor by clustering method utilizes the correlation of Wi-Fi and barometer data, and a radio map is constructed through a semi-supervised learning technique using candidates of location labels, PDR sensor data, and WLAN fingerprints. The extensive experiments carried out in three large multi-story buildings demonstrate that the method successfully builds accurate radio maps without any explicit effort to collect location reference.