SmartGantt - سیستم هوشمند برای تغییر زمان بندی مجدد زمان واقعی بر اساس یادگیری تقویت رابطه ای

With the current trend towards cognitive manufacturing systems to deal with unforeseen events and disturbances that constantly demand real-time repair decisions, learning/reasoning skills and interactive capabilities are important functionalities for rescheduling a shop-floor on the fly taking into account several objectives and goal states. In this work, the automatic generation and update through learning of rescheduling knowledge using simulated transitions of abstract schedule states is proposed. Deictic representations of schedules based on focal points are used to define a repair policy which generates a goal-directed sequence of repair operators to face unplanned events and operational disturbances. An industrial example where rescheduling is needed due to the arrival of a new/rush order, or whenever raw material delay/shortage or machine breakdown events occur are discussed using the SmartGantt prototype for interactive rescheduling in real-time. SmartGantt demonstrates that due date compliance of orders-in-progress, negotiating delivery conditions of new orders and ensuring distributed production control can be dramatically improved by means of relational reinforcement learning and a deictic representation of rescheduling tasks.

Increasing global competition, a shift from seller markets to buyer markets, mass customization, operational objectives that highlight customer satisfaction and ensuring a highly efficient production, give rise to complex dynamics and on-going disruptive events in industrial environments (Henning and Cerdá, 2000 and Zaeh et al., 2010). Moreover, stringent requirements with regard to reactivity, adaptability and traceability in production systems and supply chains are demanded for products, processes and clients all over the product lifecycle. In this context, established production planning and control systems must cope with unplanned events and intrinsic variability in manufacturing environments where difficult-to-predict circumstances occur as soon as plans are released to the shop-floor (Méndez et al., 2006 and Vieira et al., 2003). Equipment failures, quality tests demanding reprocessing operations, rush orders, delays in material inputs from previous operations and arrival of new orders give rise to uncertainty in real time schedule execution. In this way, for both human planners and shop floor operators (who interpret the plan) uncertainty in a real manufacturing system is a complex phenomenon that cannot be addressed exclusively through the inclusion of uncertain parameters into problem statement (Aytug, Lawley, McKay, Mohan, & Uzsoy, 2005). Several disturbances and events may produce different impacts depending on the context in which they occur, e.g. operators performance may vary during the week or events arising at night may have a greater impact due to the absence of specialized support personnel, as well as the uncertainty that affects materials availability may disrupt production processes in different ways, depending on product recipes. The vast majority of the scheduling research does not explicitly consider execution issues such as uncertainty, and implicitly assumes that the global schedule will be executed exactly as it emerges from the algorithm that generates it. The existing body of theory does not address different causes, the context in which uncertainty arises, or the various impacts that might result (McKay and Wiers, 2001, Pinedo, 2005 and Pinedo, 2008). Moreover, including additional constraints into global scheduling models significantly increases problem complexity and computational burden, of both the schedule generation and rescheduling tasks, which are (in general) NP-hard (Chieh-Sen, Yi-Chen, & Peng-Jen, 2012). Hence, schedules generated under deterministic assumptions are often suboptimal or even infeasible (Henning, 2009, Li and Ierapetritou, 2008, Vieira et al., 2003, Yagmahan and Yenisey, 2010 and Zaeh et al., 2010). As a result, reactive scheduling is heavily dependent on the capability of generating and representing knowledge about strategies for repair-based scheduling in real-time. Finally, producing satisfactory schedules rather than optimal ones in reasonable computational time, in an integrated manner with enterprise resource planning and manufacturing execution systems is mandatory for responsiveness ( Herroelen and Leus, 2004, Trentesaux, 2009 and Vieira et al., 2003). Reactive scheduling literature mainly aims to exploit peculiarities of the specific problem structure (Adhitya et al., 2007, Miyashita, 2000, Miyashita and Sycara, 1995, Zhang and Dietterich, 1995, Zhu et al., 2005 and Zweben et al., 1993). More recently, Li and Ierapetritou (2008) have incorporated uncertainty in the form of a multi-parametric programming approach for generating rescheduling knowledge for specific events. However, the tricky issue is that resorting to a feature-based representation of schedule state is very inefficient, and generalization to unseen schedule states is highly unreliable (Morales, 2004). Therefore, any learning performed and acquired knowledge are difficult to transfer to unseen scheduling domains, being the user-system interactivity severely affected due to the need of compiling the repair-based strategy for each disruptive event separately. Most of the existing works on rescheduling prioritize schedule efficiency using a mathematical programming approach, in which the repairing logic is not clear to the end-user. In contrast, humans can succeed in rescheduling thousands of tasks and resources by increasingly learning in an interactive way a repair strategy using a natural abstraction of a schedule: a number of objects (tasks and resources) with attributes and relations (precedence, synchronization, etc.) among them. Such conditions, as well as the requirements agility and productivity, together with poor predictability of a shop-floor dynamics, an increasing number of products, reconfigurable manufacturing lines and fluctuations in market conditions demand from production planning and control systems to incorporate higher levels of intelligence. Today’s standard, rigid and hierarchical control architectures in industrial environments have been unable to face with the above challenges, so it is essential to pursue a paradigm shift from off-line planning systems to on-line and closed-loop control systems (Zaeh & Ostgathe, 2009), which take advantage of the ability to act interactively with the user, allowing him to express his preferences in certain points of the decision making process to counteract the effects of unforeseen events, and set different schedule repair goals that prioritize various objectives as such stability, efficiency, or a mix between the two, having into account particular objectives related to customer satisfaction and process efficiency. A promising approach to sustainable improvements in flexibility and adaptability of production systems is the integration of artificial cognitive capabilities, involving perception, reasoning /learning and planning skills (Zaeh et al., 2010). Such ability enables the scheduling system to assess its operation range in an autonomic way, and acquire experience through intensive simulation while performing repair tasks. By integrating learning and planning, the system builds models about the production process, resource and operator capabilities, as well as context information, and at the same time discovers structural patterns and relations using general domain knowledge. Therefore, a scheduling system integrates continuous real-time information from shop-floor sensors/actuators with models that are permanently updated to adapt to a changing environment, and to optimize action selection. At the representation level, it is mandatory to scale up towards a richer language that allows the incorporation of the capabilities mentioned above (Morales, 2003 and Van Otterlo, 2009); in that sense, first-order relational representation it’s a natural choice because it enables the exploitation of the existence of domain objects and relations (or, properties) over these objects, and make room for quantification over objectives (goals), action effects and properties of schedule states ( Blockeel et al., 1999 and Džeroski et al., 2001). In this work, a novel real-time rescheduling prototype application called SmartGantt, which resorts to a relational (deictic) representation of (abstract) schedule states and repair operators with RRL is presented. To learn a near-optimal policy for rescheduling using simulations (Croonenborghs, 2009), an interactive repair-based strategy bearing in mind different goals and scenarios is proposed. To this aim, domain-specific knowledge for reactive scheduling is developed using two general-purpose algorithms already available: TILDE and TG ( De Raedt, 2008 and Džeroski et al., 2001).

A novel approach and a prototype application for – called SmartGantt – simulation-based learning of a relational policy dealing with automatic repair in real time of plans and schedules based on relational reinforcement learning has been proposed. The policy allows generation of a sequence of deictic (local) repair operators to achieve several rescheduling goals to handle abnormal and unplanned events such as inserting an arriving order with minimum tardiness based on a relational (deictic) representation of abstract schedule states, and repair operators. Representing schedule states using a relational (deictic) abstraction is not only efficient to profit from, but also potentially a very natural choice to mimic the human ability to deal with rescheduling problems, where relations between objects and focal points for defining repair strategies are typically used. These repair policies relies on abstract states, which are induced for generalizing and abstracting ground examples of schedules, allowing the use of a compact representation of the rescheduling problem. Abstract schedule states and repair actions facilitates and accelerates learning and knowledge transferring, which is independent of the type of event that has generated a disruption and can be used reactively in real-time. An additional advantage provided by the relational (deictic) representation of schedule (abstract) states and operators is that, relying in an appropriate and well designed set of background knowledge rules, it enables the automatic generation through inductive logic programming of heuristics that can be naturally understood by the end-user, and facilitates tasks like “what–if” analysis, interactive rescheduling and decision support. Scale-up reinforcement learning using relational modeling of rescheduling situations based on simulated transitions is a very appealing approach to compile a vast amount of knowledge about repair policies, where different types of abnormal events (order insertion, extruder failure, rush orders, reprocessing needed, etc.) can be generated separately and then compiled in the relational regression tree, regardless of the event used to generate the examples (triplets). This is another very appealing advantage of the proposed approach, since the repair policy can be used online to handle disruptive events that are even different from the ones used to generate the Q-function; once this function is known, reactive scheduling is straightforward. Finally, relational reinforcement learning favors denotational concept semantics in the communication between humans and SmartGantt. This capability of phrasing the rationale behind repair strategies so as to receive evaluative feedback verbally from human users is obviously highly useful in real-world applications of SmartGantt.