We present a dynamic programming-based heuristic algorithm for selecting the next task of in-process cars in automotive assembly plants. As car production demand diversifies, manufacturing processes require high flexibility. An alternative to a conventional production line, matrix-structured production system provides a flexible manufacturing environment. In this system, until the final assembly operation is completed, each autonomous mobile robot (AMR) is assigned to a single car body and transports among various workstations to accomplish its given operations. These operations require sequential procedure and can be performed at multiple workstations. Therefore, assigning AMRs to appropriate workstations and operations is the key decision making that determines the capacity of the plants. In this paper, we formulate the Markov decision process (MDP) model for the flexible job shop scheduling problem with a single AMR and get the optimal policy by applying dynamic programming. Moreover, we suggest a heuristic algorithm to deal with multiple AMRs. We, then, validate the proposed algorithm by comparing various algorithms using simulation. Finally, we analyze the effect of the scheduling algorithm on production capacity.