MultiSoft: Soft Sensor Enabling Real-Time Multimodal Sensing with Contact Localization and Deformation Classification

Abstract

<jats:p>We introduce MultiSoft, a multilayer soft sensor capable of sensing real-time contact localization, classification of deformation types, and estimation of deformation magnitudes. We propose a multimodal sensing pipeline that carries out both inverse problem solving and machine learning tasks. Specifically, we employ an electrical impedance tomography (EIT) for contact localization and a support vector machine (SVM) for classifying deformations and regressing their magnitudes. We propose a deformation-aware system which enables maintaining a persistent single-point contact localization throughout the deformation. By updating a time-varying distribution of conductivity change caused by deformations, a single-point contact localization can be maintained and restored to support interaction using both contact localization and deformations.We devise a multilayer structure to fabricate a highly stretchable and flexible soft sensor with a short sensor settlement after excitations. Through a series of experiments and evaluations, we validate both raw sensor and multimodal sensing performance with the proposed method. We further demonstrate applicability and feasibility of MultiSoft with example applications.</jats:p>