مقایسه رقابتی از طرح های بهینه آزمایش برای تجزیه و تحلیل حساسیت بر اساس نمونه گیری

A widely used strategy to explore the sensitivity of the model to its inputs is based on a finite set of simulations. These are usually performed for a chosen set of points in a parameter space. An estimate of the sensitivity can be then obtained by computing correlations between the model inputs and outputs. The accuracy of the sensitivity prediction depends on a quality of the points distribution in the parameter space, so-called the design of experiments. The aim of the presented paper is to review and compare available criteria determining an optimal design of experiments for sampling-based sensitivity analysis.

Sensitivity analysis (SA) is an important tool for investigating properties of complex systems. It represents an essential part of inverse analysis procedures [1], response surface modelling [2] or uncertainty analysis [3]. To be more specific, SA provides some information about the contributions of individual system parameters/model inputs to the system response/model outputs. A number of approaches to SA has been developed, see e.g. [4] for an extensive review. The presented contribution is focused on widely used sampling-based approaches [2], particularly aimed at an evaluation of Spearman’s rank correlation coefficient (SRCC), which is able to reveal a nonlinear monotonic relationship between the inputs and the corresponding outputs. When computing the SA in a case of some real system using expensive experimental measurements or some computationally exhaustive numerical model, the number of samples to be performed within some reasonable time is rather limited. Randomly chosen sets of input parameters do not ensure appropriate estimation of related sensitivities. Therefore the sets must be chosen carefully. In this contribution we would like to present a review and comparison of several criteria, which can govern the stratified generation of input sets – the so called design of experiments (DoE). Generation of optimal DoEs is a very broad topic and all pertinent aspects cannot be discussed within this paper. Hence, we focus especially on DoEs in discrete domains. The presented methods can be of course applied also to discretized continuous domain. Nevertheless, other possibilities for generation DoEs in continuous domains are, however, beyond the scope of this paper. The following section reviews the criteria for optimisation of DoE, which are available in literature. Section 3 includes some comments on widely used methods for stratified generation of DoE and Section 4 presents the discussion on difficulties arising from optimisation of particular criteria. Section 5 is devoted to the comparison of mutual qualities of particular optimal DoEs and Section 6 compares their quality in terms of projective properties which are important in a screening phase of model analysis. Sequential improvement of the existing DoE is discussed in Section 7. Finally, Sections 8 and 9 present the assessment of the optimal designs quality for usage in sampling-based SA for theoretical analytical functions and structural models, respectively. Concluding remarks are summarised in Section 10.

This paper reviews eight criteria used for optimising a design of experiments and presents their comparison in terms of ease of their optimisation, their mutual qualities and their suitability for usage in sampling-based SA on 15 analytical and two structural models. The overall results can be summarised in several following conclusions: • CN criterion was poorly evaluated in terms of all presented aspects. It may make the optimisation process more complicated by its tendency to the higher number of local extremes. The resulting designs have very poor space-filling properties and the subsequent sensitivity predictions for analytical as well as for structural models contain large errors. • The correlation-based criteria (PMCC, SRCC and KRCC) may also pose some difficulties during the optimisation process: SRCC and KRCC due to their discrete nature and PMCC due to its stronger tendency to the multi-modality. All these criteria provide the designs with very bad space-filling property and even LH restriction does not improve it significantly. While the free designs achieved also very bad results in SA, the predictions of LH designs can be evaluated as very good, but they suffer from higher variances. • AE and EMM criteria do not exhibit any explicit difficulties regarding the optimisation process. The advantage of these criteria is that their interpretation is very simple and computation very fast. They provide designs with similar properties: good space-filling and a moderate level of orthogonality. The free designs achieved good results in sensitivity predictions but also with a high variance among the results. The results of EMM criterion are in overall worse then those of AE criterion. Their qualities are generally deteriorated by applying the LH restriction. • Finally, the best results in sensitivity predictions were obtained using the LH designs optimised w.r.t. the ML2 criterion. The D-optimal designs were slightly worse and comparable to AE free designs or LH designs optimised w.r.t. the correlation-based criteria. ML2 and Dopt criteria provide designs with moderate space-filling properties, but they have a good level of orthogonality. The LH restriction slightly improves the results of both. The ML2 designs achieved less varying results in the SA for analytical functions, while the D-optimal designs provided more balanced results in the SA for the ten-bar truss structure. The ML2 designs also significantly overcame all the other designs in terms of the projective properties. An important shortcoming of the D-optimal criterion concerns its formulation and optimisation. Besides the fact that the criterion results in an excessive number of local extremes, the principal drawback lies in the necessity of its Bayesian modification for elimination of duplicates or closely neighbouring points. Therefore, the ML2 criterion, which is not so common in DoE optimisation, can be considered as a winner in the presented competitive comparisons.