We propose a framework based on the 5W1H model-based metadata for remote diagnosis in virtual reality (VR). For this purpose, we suggest unique metadata composed of Point of Interest (POI)-extended anchor (xAnchor)-content for a context-aware service in virtual and augmented reality. We define the attributes of the metadata based on the 5W1H context for information retrieval according to the context in a remote diagnosis. Second, we propose the ontology-based linker metadata that express the relations between AR scenes and that retrieve external information. Moreover, we suggest heritage building information metadata for information retrieval according to context. For evaluation, we created a geo-tagged content tool and a remote diagnosis VR application. We conducted focus-group interviews and heuristic evaluations for remote diagnosis in VR to verify the methodology of this study. As a result, we found that experts were most satisfied with the functions that provide the contextualized information. This study contributes to the geospatial metadata for a context-aware service in VR/AR as well as the remote diagnosis framework to overcome the time-consuming problem of the existing remote diagnosis.