Present input modality researches for the disable person and the elderly having difficulty in using the mobile interface have less focused on their physical and cognitive context. Therefore, the aim of this research is to develop the general architecture of designing multiinput modality convertor for universal accessibility. It would help determining optimal input modality sets and generating interaction grammars based on divers user's context and mobile circumstance.