مهندسی اعتبار و امنیت سیستم تحلیل حساسیت و قدرت حل مسئله چند معیاری با کاربرد فن‌های قانون محقق انگیزه مضاعف معیارهای تصمیم‌گیری قانون فراگیران

In many situations, a decision-maker is interested in assessing a set of alternatives characterized simultaneously by multiple criteria (attributes), and defining a ranking able to synthesize the global characteristics of each alternative, for example, from the best to the worst. This is the case of the assessment of several projects through attributes such as cost, profitability, among others. The behavior of each object, for every criterion, is quantified via numerical or categorical “performance values”. Several multiple criteria decision techniques could be used to this aim. However the base rank could be influenced by uncertain factors associated to specific criteria (e.g., the “ratio Benefit/Cost of a project” could be affected by variations in the interest rate) or by decision-maker preferences. In this situation, the decision-maker could be interested knowing what sets of factors are responsible of specific ranking conditions. This paper describes the input space of a set of factors responsible of a given model behavior specification, based on the use of rule learners able to provide a description through a set of “If-Then” rules derived from model samples. These techniques also allow determining the most important factors. An example related to a real decision problem illustrates the proposed approach.

In many situations, a decision-maker is interested in assessing a set of m objects or alternatives ai characterized simultaneously by n criteria or attributes, and defining a ranking able to synthesize the global characteristics of each object, e.g., from the best to the worst. This is the case, for example, in the assessment of several engineering projects through attributes such as cost, availability, environmental impact, among others. The behavior of each object, for every criterion, is quantified via performance values PVij (for each alternative i=1, .., m and for each criterion j=1, … ,n) which can either be numerical or categorical. The idea of ranking alternatives is based on one of the four discrete decision-making problems defined as “Problematique γ” in [1], that is, ranking the alternatives from the best to the worst ones. Several multicriteria decision techniques (MC) or ranking techniques could be used to this aim [2]. Ranking techniques to generate the desired rank are classified as parametric and non-parametric. The first group, like ELECTRE [1], PROMETHEE [3], TOPSIS [4] to name a few, requires information about decision-maker preferences (e.g., criterion weights), while non-parametric techniques (partial order ranking [5], Hasse diagram technique [6] and Copeland Scores [7]) do not use such information. In general, the ranking assessment is performed as follows: 1. Define m alternatives and n criteria. 2. Define the multi-indicator matrix Q, based on each PVij (for each alternative i=1, .., m and for each criterion j=1, .., n). 3. Select a ranking technique. 4. Produce a rank of objects according to the selected technique. However, no matter which MC technique is selected, the ranking derived using crisp PV (defined as the Base Rank (BR)), could be influenced by uncertain factors associated to specific criteria (for example, the criterion “Cost/Benefit ratio of a project” could be affected by variations in the interest rate) or by decision-maker preferences (e.g., criterion weights). If these uncertain factors are modeled as a probability distribution function then the rank of each alternative could be considered as a random variable. Several authors [8], [9], [10] and [11] have analyzed this problem: how the uncertainty in the PV (the input) is propagated or affects the object ranks (the output)? Recently, Rocco and Tarantola [12] presented two approaches that extend previous works in two directions: 1. Ranking assessment: based on Monte Carlo simulation, the approach allows answering several questions regarding ranking robustness. For example, under uncertainty: What is the probability that the base rank position is maintained? Which is the rank position with the highest probability? What are the possible rank positions and their corresponding probability? 2. Sensitivity analysis: this approach, based on global sensitivity analysis techniques [13], allows evaluating the importance of uncertain factors. This type of analysis, from input to output, can provide to the decision-maker a sharper picture of the effects of the uncertainty in the final ranking that MC techniques provide. Therefore, the decision-maker can have a better perspective of how stable his/her final decision is and often needs to know which factors determine specific output behavior (output specifications). For example, what are the values associated to each criterion that make a particular project be ranked as the best project? Procedures to cope with such problems are termed as Factor Mapping setting, “in which specific points/portions of the model output realizations, or even the entire domain, are mapped backwards onto the space of the input factors” [14]. Note that the solution space could be a non-convex and/or sparse space [15], [16] and [17]. Several approaches have been proposed in the literature to produce such mapping like Monte Carlo Filtering [13] and [14], Regional Sensitivity Analysis [18], Generalized Likelihood Uncertainty Estimation [19] and Tree-Structured Density Estimation [20]. Other approaches, based on optimization instead of mapping from the output into the input space, have been suggested in [15] or recently in [16] and [17]. These approaches are able to extract the maximum volume hyperbox of the solution space, where factor variations are assigned independently. The solution space is represented through intervals [x1_inf, x1_sup], [x2_inf, x2_sup], …, and [xl_inf, xl_sup], where xj is the jth factor and l is the number of uncertain factors considered. Fig. 1 illustrates the approach in the case of two factors x1 and x2. The area delimited by dashed lines defines the feasible zone. The rectangle (solid lines) represents the box with maximum area. Full-size image (9 K) Fig. 1. Maximum area box (solid lines) in the feasible zone defined by dashed lines. Figure options The approach proposed in [15] requires an analytical model f(x1, x2, .., xl), while in [16] and [17] the model is considered as a black-box. In both approaches the hyperbox could be centered at a predefined feasible point or freely centered across the feasible zone. The widths of the final intervals that define the solution space could be considered as a sensitivity index. This paper proposes an approach based on the use of machine learning classification techniques [21] able to provide a description of the solution space, based on a set of “If (premise) then (consequence)” rules derived from model samples (i.e., could be used for analytical or black-box models) where (premise) is a condition (or the logical product of several conditions) related to a specific factor or variable whereas (consequence) gives a class assignment. For example, for a given project B, the structure of the hypothetical rule If (Cost/Benefit_Project_B>8 AND Employment_Project B>120) then (Rank_project_B=1) explains when project B is ranked as the first project. Each rule extracted represents a specific hyperbox of the solution space. This allows to model non-convex solution spaces. Additionally, some rule generation techniques are able to extract the most important factor, can detect non important factors or can provide a numerical sensitivity index. The rest of the paper is organized as follows: Section 2 describes the problem to be analyzed and proposes a solution based on machine learning classification techniques. Section 3 presents an overview of rule generations concepts and examples of solutions through several algorithms. Section 4 describes a case study. Finally, Section 5 shows the conclusions and future work.

In this paper, Machine Learning techniques are used to assess the robustness of a ranking in multicriteria decision problems. A set of “If-Then” rules derived from model samples, allows describing the condition of the input variables (performance of alternatives under a defined set of criteria and/or decision-makers preferences), given a model behavior specification, represented by a global constraint. The approach presented is considered as an additional tool for decision-makers interested in characterizing a decision problem. Indeed, the information provided by previous works on the effect of uncertain inputs on the final ranking of alternatives or the knowledge of the important inputs that affect ranking variations, could be now complemented by an approach (based on rule generation techniques) able to describe the feasible space of a set of inputs responsible of a given model behavior constraint. Additionally, such techniques produce the subset of variables actually correlated and its importance under the specified constraints. The case study presented (a real portfolio of 20 projects evaluated through 3 criteria) shows that the approach is able to approximately synthesize the ranking model responses, under a specific constraint, through a group of simple rules that relates the variables and their possible variation ranges. In addition, it is also possible to determine the set of important variables that affect the selected behavior. Although the analysis is based on the use of the PROMETHEE methods, any other multicriteria technique could be used. Different rule generation techniques could be selected. In this paper several procedures were tested, using their default settings, to only illustrate their use and capabilities. However no comparison among techniques was performed. The quality of the extracted classifier (mainly assessed through sensitivity, specificity and accuracy indexes) also depends on the cardinality of the set of samples used. For the area suggested in this paper (i.e., multicriteria project evaluations), this is not a limitation since the set of samples are randomly generated using a model that is not computationally intensive. However, the set of samples must be carefully selected to avoid unbalanced classes and therefore derive naïve classifiers. This situation could happen when the constraint imposed produces a feasible region with a small hyper-volume. In any case, the performance indexes along with the quality of the rules extracted must be taken into consideration before making the final decision.