برنامه ریزی تولید کارگاهی پویا با استفاده از عوامل یادگیری تقویتی

Static and dynamic scheduling methods have attracted a lot of attention in recent years. Among these, dynamic scheduling techniques handle scheduling problems where the scheduler does not possess detailed information about the jobs, which may arrive at the shop at any time. In this paper, an intelligent agent based dynamic scheduling system is proposed. It consists of two independent components: the agent and the simulated environment. The agent selects the most appropriate priority rule according to the shop conditions in real time, while simulated environment performs scheduling activities using the rule selected by the agent. The agent is trained by an improved reinforcement learning algorithm through the learning stage and then it successively makes decisions to schedule the operations.

Scheduling, one of the key problems in manufacturing systems, has been a subject of interest for a long time. However, it is difficult to talk about a method that gives optimal solutions for every problem that emerges. The problem is to schedule a set of jobs subject to a set of constraints where each job consists of a set of operations. The aim is to get an appropriate schedule in terms of a certain criterion. Since previous studies have considered the set of jobs as having all required information at initial time, and hence most of the methods scheduled the jobs in a static manner. On the other hand, the relation between jobs and shop floor is not so static that the systems proposed in that manner are not suitable in real life. In fact, each job comes into shop over time and the required information is uncertain in most cases. Thus, a dynamic scheduling system is more suitable than a static one. Dynamic systems start with the jobs that come first, and assume that they come according to a stochastic rule over time. In order to build dynamic scheduling systems, several methods have been proposed so far. Some studies have focused on dynamic scheduling for flexible manufacturing systems. Yih and Thesen [40] considered the real-time scheduling system for an FMS as a semi-Markovian decision process to be optimized. Ishii and Talavage [15] generate short-term schedules for an FMS, while Arzi [1] suggests a two-step dynamic scheduling algorithm for such systems. Similarly, Matsuura et al. [22] proposed a switching technique for dynamic scheduling allowing consideration of machine break-downs and other emergent events. Most of the studies were also performed for generic systems. For example, Sun and Lin [34] viewed the scheduling system as an optimal control problem of discrete events and scheduled the jobs using a backward scheduling algorithm. On the other hand, there are some approaches developed based on artificial intelligence techniques such as neural networks, expert systems, fuzzy logic and genetic algorithms. Chang [10] developed a rule-based system that proposes incremental dispatching rules. Sim et al. [30] combined ES and NN for generating the most appropriate schedule in the current state. Both Shaw et al. [29] and Nakasuka and Yoshida [23] used a second generation ES model that acquires its knowledge automatically. In all of these approaches, the most appropriate dispatching rule is proposed. Genetic algorithms (GAs) are also used extensively for JSS. Bierwirth et al. [7] and Lin et al. [17] adapted GA to the Giffler and Thompson algorithm and constructed dynamic schedules. The literature review indicates that there has been little work on creating intelligent autonomous scheduling systems with a learning ability based on trial and error. In this study, an intelligent agent based scheduling system is proposed aiming at the generation of a more autonomous scheduler where the agent is trained by a new improved reinforcement learning algorithm, Q-III. In the following sections, first intelligent agents and then the Q-III learning algorithm are presented. Thereafter, details of the intelligent agent based scheduling system are discussed using the simulation results.

In this study, an intelligent agent based dynamic scheduling system is presented. The system is composed of the agent and the simulated environment (SE). The agent is able to perform dynamic scheduling based on the available information provided by the SE. It makes decision for selection of the most appropriate dispatching rule in real time. It was trained by Q-III learning algorithm. The results are encouraging and the performance of the agent will be improved by enriching the environment as well as the skills of the agent. This will be a good initiative to create fully automated intelligent manufacturing systems. The success of the agent in scheduling will also be a good example for other issues such as design, planning and control within the manufacturing systems to be taken into consideration.