Influence-directed policy generation using reinforcement learning for collaborations in system of systems

Abstract

A system of systems (SoS) tries to utilize constituent systems' (CSs) capabilities to achieve its goals. This activity is quite challenging because CSs are themselves systems that are autonomous and decide when and how to activate their capabilities on their own account. For example, in the case of mass casualty incidents, response systems such as fire fighters, emergency vehicles, rescuer agents, \textit{etc.} have to commit their capabilities and collaborate to achieve the response goals. For such collaborations, it is not only synchronizing CSs' actions and decisions, but requires influencing the CSs' decision making behavior in order to achieve the common goals. In this research, we present the design and development of an influence-directed policy generation approach that aims on generating collaboration policies. The collaboration policies are generated by leveraging decision pattern analysis and reinforcement learning techniques. The outcomes of a collaboration in a SoS can be significantly influenced by the CSs' autonomy and belongingness. The notion of autonomy describes CSs' managerial and operational independence to make choice about when and how their capabilities should be activated, while belongingness refers to the degree of relatedness (or overlap) between the SoS purpose and the CSs goals. As desirable as belongingness can be in achieving the SoS purpose, autonomy can be a limiting factor. We aim to utilize CSs' capabilities to perform the SoS specific tasks while maintaining the CSs' autonomy. This research contributes to resolving two major collaboration concerns in SoS: how to enable SoS designers to compose CSs' capabilities, and CSs developers to design their systems to make efficient collaboration in a SoS context. To a certain extent, the proposed approach addresses guided emergency problems which is rated as one of the highly valued research endeavors in the SoS engineering domain.