This paper introduces two new modern methods of global sensitivity analysis for computer models: Fourier Amplitude and Sobol, as well as a modern factor screening method: the Morris method. The methods are applied to the sensitivity analysis of the apartment fire module of the CESARE-Risk building fire computer model with eight input factors and door and window open. Two output variables are considered: the maximum temperature reached and the time of onset of untenable conditions. Response surfaces previously derived for the model [Jianguo Qu, Response surface modelling of Monte Carlo fire data, Ph.D. Thesis, Victoria University, Melbourne, Australia, 2003, http://eprints.vu.edu.au/archive/00000260/01/Qu,_Jianguo.pdf] are used to speed up the computations. For the maximum temperature all three methods agree that the most sensitive factors are the window height and width factors, followed by the fuel area factor. The largest interaction was between the length of room and the fuel area factor. For the time of untenable conditions the Fourier Amplitude and Sobol methods agreed that one factor, the flame spread rate, had overwhelming significance. The only significant Sobol interaction was between the width of room and the flame spread rate.
A sensitivity analysis (SA) is a study of how changes in a computer model parameter affect the results generated by the model. Model predictions may be sensitive to uncertainties in input data, to the level of rigour employed in modelling the relevant physics and chemistry, and to the accuracy of numerical treatment. SA is defined by Saltelli [1] as the study of how the variation in the output of a model (numerical or otherwise) can be apportioned, (qualitatively or quantitatively) to different sources of variation. These sources are referred to as inputs or factors and can be input variables, model parameters, model structure, assumptions and specifications.
The main purpose of conducting a SA is to determine the following:
1.
the important input variables in the models (those which effect changes in important output variables),
2.
the required accuracy for input variables (particularly sensitive input variables may need to be known to a greater precision than insensitive ones),
3.
the sensitivity of output variables to variations in input data (which inputs affect most the output variable of interest).
These results can be used by the modeller to determine:
1.
if the model resembles the system or process under study;
2.
the inputs that require additional research to strengthen the knowledge base;
3.
the model parameters (or parts of the model itself) that are insignificant, and that can be eliminated from the final model (e.g. by fixing them to a nominal value);
4.
if there is some region in the space of input parameters for which the model variation is maximum;
5.
the optimal region within the space of input parameters for use in a subsequent calibration study;
6.
if and which factors interact with each other.
Conducting a SA of a complex fire model is a difficult task. Many models require extensive input data and generate predictions for numerous output variables over a period of simulated time. Several classical methods of SA have been applied to fire models, but most have had limited utility. These range from explicit evaluation of the equations used in simple models such as ASET [2] to pointwise analysis of complex models from numerous computer runs of the model [3].
Many different methods of SA have been tried out over the years in many fields. But it is only recently that thanks to the enormous expansion of the computing power of personal computers during the 1990s, new and more powerful methods of SA became feasible. Major contributions have in particular been made to the field by the Institute for Systems, Informatics and Safety at the Joint Research Centre of the European Commission in Ispra, Italy, under the leadership of Andrea Saltelli. The emphasis was on what is known as methods based on decomposing the variance of the output, namely the Sobol and Fourier Amplitude methods. And a consensus has developed that these methods are superior to earlier methods. For example Ascough II et al. [4] write in 2005:
“Integrated natural resource models (e.g., APSIM) are typically large and complex, thus, it can be difficult to prioritize parameters that are most promising with respect to system management goals. It is important to evaluate how a model responds to changes in its inputs as part of the process of model development, verification, and evaluation. There are several techniques for SA used by practitioners and analysts in numerous fields. In this paper, we concentrate on qualitatively evaluating four SA methods:
(1)
Fourier Amplitude Sensitivity Test (FAST),
(2)
Response Surface Method (RSM),
(3)
Mutual Information Index (MII), and
(4)
the methods of Sobol’. For SA of natural resource models, the FAST and Sobol’ methods are particularly attractive. These methods are capable of computing the so-called “Total Sensitivity Indices” (TSI), which measure parameter main effects and all of the interactions (of any order) involving that parameter.”
They have three important advantages:
(1)
they are model independent in the sense that the level of additivity or linearity of the model does not influence the accuracy of the method;
(2)
they have the capacity to account for factor to factor interaction as well as the ability to group factors into sets and to treat each set as one factor;
(3)
they are computationally efficient.
Saltelli edited a comprehensive textbook on SA [5]. This was followed by the release of a software package called SIMLAB for carrying out various modern methods of SA, as well as the publication of a book titled “Sensitivity Analysis in Practice” [6].
In this paper, after a short discussion of the various settings for SA, an overview of early work will be given. A description of a modern factor screening method due to Morris will be followed by a description of the variance decomposition models: Fourier Amplitude and Sobol. The Morris, Fourier Amplitude and Sobol methods will then be applied to the SA of the Centre for Environmental Safety and Risk Engineering (CESARE)-Risk fire computer model [7] with eight input parameters and two output variables: the maximum temperature and the time of onset of untenable conditions in the compartment of fire origin.
