تجزیه و تحلیل حساسیت برای تصمیم گیری بر اساس شبیه سازی: برنامه برای طراحی خدمات اورژانس بیمارستان

An increasing concern of decision makers when dealing with system design is preparation for a wide range of potentially uncertain operating conditions. This paper provides a novel multiobjective approach for simulation-driven decision making that accounts for not only the conventional average system performance indices, but also (i) upper-tail, or extreme, values of these indices, and (ii) measures of their sensitivity to uncertainty in model parameters. The proposed approach is applied to a hospital emergency department service design case study wherein different design alternatives are compared using total time-in-system performance metric under multiple uncertain operating conditions.

One of the challenges encountered most by decision makers and system designers in government and industry is the capacity to face wide range of operating conditions. Recent malevolent man-made events, accidents, and natural disasters have increased this concern and highlighted the need to prepare for extreme event scenarios. Structural engineers and designers frequently use safety factors to ensure that a structure can withstand a range of operating conditions. Similar considerations should be made by designers of human-driven processes and systems. Our understanding of the effective operation of a system requires an understanding of the uncertainty affecting the system. Such uncertainty could originate from the system’s operating environment (i.e., uncertainty resulting from the stochastic nature of the system) or from the developed model to understand and interpret the behavior of the system (i.e., uncertainty resulting from model structure and parameters), among others. When a system model is simulated, subsequent decisions made based on the simulation results are only as good as one’s ability to understand and account for this uncertainty. Even our ability to represent uncertainty through probability distributions suffer when the parameters describing those distributions are inaccurate, either through their elicitation, their underlying field data, or their ability to describe a complex system [6], [10] and [32]. One approach to addressing uncertainty is through sensitivity analysis [24] and [38]. Understanding the sensitivity of decisions to underlying uncertainties in the system can improve the decision making process in a number of areas, including [33]: understanding the robustness of an optimal solution, identifying sensitive or important factors, investigating sub-optimal solutions, developing flexible recommendations based on certain scenarios, comparing simple and complex strategies, and assessing the “riskiness” of a strategy, among others. Sensitivity analysis is especially important in simulation-based decision making, as underlying uncertainty may skew simulation results. An example of such simulation-based decision making of interest in this paper is the simulation-driven comparison of design alternatives. In this context, several approaches have been reviewed by Bechhofer et al. [7], Goldsman and Nelson [20], and Kim and Nelson [23]. With respect to the comparison of alternatives in more extreme operating environments, Fishman [18] discusses methods for analyzing simulation-driven metrics such as mean occurrence for average situations and exceedence probabilities for extreme cases. This paper provides a novel multiobjective approach for simulation-driven decision making that considers, not only the conventional average time-in-system performance metric, but also: (i) upper-tail (extreme) values of time-in-system, and (ii) measures of sensitivity to uncertainty in model parameters. The proposed approach integrates the Partitioned Multiobjective Risk Method (PMRM) [4] and the Uncertainty Sensitivity Index Method (USIM) [21] and [29] with a discrete-event simulation model. For the purposes of our discussion, multiobjective optimization problems (MOPs) can be grouped into two types: (i) deterministic multiobjective optimization problems (DMOPs), where constraints and/or objective functions can be structured and described analytically, and (ii) simulation-based multiobjective optimization problems (SMOPs), known also as black-box simulation multiobjective optimization problem, where an analytical representation of the constraints and/or objective functions is not available, or very complex to obtain. This paper contribution is to SMOPs, where the comparison of several candidate designs is performed based on metrics obtained using discrete event simulation. DMOPs use deterministic formulations of objective functions to be optimized. Thus, different computational methods such as mathematical programming, weighted sum techniques (which converts the MOP into a single objective optimization problem by using a convex combination of objectives), evolutionary algorithms, or any combination of these methods can be used to solve the problem. Conversely, only little work has been devoted to SMOP. A method for approximating the Pareto frontier of an SMOP is proposed in [36]. The method iteratively approximates each objective function using a meta-modeling scheme and employs a weighted sum method to convert the SMOP into a set of single objective optimization problems. The major limitation of the proposed method is that it assumes deterministic simulation outputs. Caricato et al. [9] propose a SMOP formulation to solve a system configuration problem in a hybrid flow shop system. Both discrete-event simulation and mathematical programming tools are used to solve the problem. Thus, the stochastic nature of the problem is preserved at the difference with the approach proposed in [36]. Pareto-dominance concepts are used to eliminate inefficient solutions. As discussed in [9], most SMOP formulations found in literature are aimed toward the maximization or minimization of the expected value of the considered objective functions. The main referred techniques that have been used are statistical procedures, meta-heuristics, stochastic optimization, and other techniques which include ordinal and sample path optimization. When considering sensitivity in the optimization problem formulation, only few approaches have been previously discussed in [12], [27], [30] and [40], including a robust multiobjective simulation optimization approach which borrows from Taguchi’s philosophy of uncertainty to address uncertainty in the parameters of the optimization problem [14]. Arsham [3] discussed the application of conventional sensitivity estimation methods with respect to decision and system parameters variability when using discrete event simulation. Several algorithms for obtaining sensitivity information on discrete event system via simulation using finite difference, simultaneous perturbation, perturbation analysis, likelihood ratio, score function, and harmonic analysis methods where discussed. Kleijnen and Rubinstein [25] combine the score function method with classic experimental design for sensitivity analysis, and optimization of discrete-event systems. Contrary to most SMOP approaches (reviewed in the literature of Ref. [9]), the systematic methodology proposed here allows for tradeoffs among simulation-driven objectives and the sensitivity of those objectives to uncertainty, accounting for both, average and extreme operating conditions. This methodology is not aimed only toward optimization of the expected value but also considers sensitivity metrics based on the combination of risk theory methods with discrete event stochastic simulation, and not deterministic simulation as proposed in [36]. The development of this type of approach was previously recommended in [28], where non-expected value design analysis and risk assessment methods, were proved to be more appropriate means of informing healthcare decisions. Most SMOP applications refereed in literature belong to manufacturing and technical design areas, whereas only few applications in the field of healthcare design were available. In [34], an axiomatic framework is used to analyze the design of a triage system for healthcare emergency departments (EDs). The multiobjective problem was simplified by the introduction of a new design parameter to break the coupling between patient flow and treatment urgency criteria. An improved design is suggested based on a new triage index based on the estimated value of the patient’s time in ED. The design is tested using discrete event simulation model of an existing ED. In [1], a computer program is combined with optimization to determine the optimal number of staff required to maximize patient throughput and to reduce patient time in a hospital under budget restriction constraints, with the best solution being selected among feasible generated alternatives using a random walk based search procedure. In [39], a discrete event simulation model of a family practice healthcare clinic was developed, with generated input data fed into a fractional factorial design to determine input factors (number of physician, nurses, medical assistants, check-in rooms, examination rooms, and specialty rooms) which significantly affect overall clinic effectiveness. The approach proposed in this paper is applied to a hospital emergency department (ED) service design case study. The US Department of Homeland Security considers healthcare systems to be among its “critical infrastructure” [15], likewise with many other countries. The efficient operation of hospitals, particularly during emergency situations, is vital to the sustainment of public health and safety and has been the focus of several studies [19], [31] and [42]. In particular, a hospital ED is one such system whose service design should incorporate a potentially uncertain and wide range of operating conditions. The case study will measure the sensitivity of average and upper-tail times in the ED system with a stochastic patient arrival process. These measures are ultimately used to compare different ED system designs towards providing reliable performance under multiple operating conditions. The paper is developed as follows. Section 2 presents the proposed four-step simulation-driven decision making (SDDM) methodology for addressing uncertainty and extreme operating conditions while comparing candidate designs using discrete event simulation. Section 3 illustrates the application of the proposed approach to an ED system design case using simulation model, including results and discussion. Concluding remarks follow in Section 4.

A concern of decision making in choosing system design alternatives is that such decisions be robust to a wide range of uncertain operating conditions, establishing a balance between achieving current goals and flexibility in the future. This paper provides a systematic approach to perform candidate design sensitivity analysis based on discrete-event simulation modeling, under both average and extreme operating conditions. The purpose of this novel approach, the simulation-driven decision making (SDDM) methodology, is to perform a multiobjective comparison of several candidate layout and operational designs, where the metrics quantifying the efficacy of each design are found through discrete-event simulation. The SDDM, whose framework is depicted graphically in Fig. 1, integrates the Partitioned Multiobjective Risk Method (PMRM) [4] with the Uncertainty Sensitivity Index Method (USIM) [21] and [29], extending the work of Barker and Haimes [5] specifically for decision making driven by simulation models. The SDDM is applied to risk analysis in health care facility service design. Health care delivery, especially in emergency conditions, is intrinsically uncertain and yet is expected to operate under a number of conditions. Patient time in the emergency department system was the simulation metric used here, though others could be explored. While illustrated with emergency department design, such a methodology can be applied to other critical systems and infrastructures where the consideration of average and extreme operating conditions are important, e.g., transportation network design, evacuation planning, vehicle routing. Even more generally, facility layout and operational design are discussed, though this approach could be used in a number of applications of simulation-driven comparisons of candidate options. Several novel contributions are integrated within the SDDM, including the consideration of simulation-driven average and upper-tail values for more holistic decision making, the application of the five-point derivative method in the calculation of the sensitivity index, and the product approach for comparing multiple objectives. Future work includes the integration of a weighting approach for eliciting the importance of multiple objectives (e.g., the Analytic Hierarchy Process [37]). Such weighting may be necessary as several other objectives exist outside of the average and upper-tail system time and sensitivity metrics for the emergency department design case study, including investment costs for each candidate design, space availability, and doctor and staff availability, among others. Such metrics, including the average, upper-tail, and sensitivity metrics should be integrated into Step 1, the development of candidate designs, a priori. Further, an extension of the SDDM for dynamic decision making scenarios is a consideration for future work.