مدل چند معیاری برای ممیزی یک برنامه نگهداری پیشگویانه

Auditing tools can play a key role in the continuous improvement of maintenance policies, in particular to enhance predictive maintenance (PM). This paper proposes a multi-criteria model for auditing a Predictive Maintenance Programme (PMP) developed and implemented in the General Hospital of Ciudad Real (GHCR) in Spain. The model has a two-level structure, with top level auditing areas specified by second level auditing criteria on which the performance of the PMP should be appraised. This structure resulted from the analysis and discussion of an internal questionnaire to the management, technical and consulting staff of GHCR. This also guided the association of a performance scale with each criterion, describing several reference levels of accomplishment. Using the MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) approach, a hierarchical additive value model was constructed, with criteria weights and value scales derived from staff judgments of comparison of different reference levels and profiles of performance. This model enables managers to measure the performance of the PMP and its added value for the hospital, not only against each audit criterion individually, but also on each area and in overall terms. Integrated in a management “tableau de bord”, the model outputs permit the identification of PMP deficiencies requiring urgent intervention and corrective measures for its continuous improvement.

Preventive maintenance consists of the systematic inspection, detection, correction, and prevention of incipient failures, before they become actual or major failures; and, predictive maintenance (PM) is a special type of preventive maintenance in which maintenance is performed continuously or at intervals according to the requirements to diagnose and monitor a condition or system (Luthra, 2010). PM can generate high gains in terms of increased availability, safety, quality, productivity of equipment and facilities and decreased maintenance and insurance costs (Christer et al., 1997, Villar et al., 2000, Lupinucci et al., 2000, Weyerhaeuser, 2000, Swanson, 2001, Wang, 2002, Mobley, 2002 and Carnero, 2005). The use of PM is particularly crucial where security is critical (e.g. in nuclear power plants) and where machines must operate with an availability, safety and quality close to 100% (e.g. in hospitals). Accordingly, the definition and implementation of a Predictive Maintenance Program (PMP) has been recognized as a strategic management decision (Tsang, 2002). PMPs use the most cost-effective tools (for example, vibration monitoring, lubricant analysis or thermography) to obtain information on the actual operating condition of critical facilities, and based on this, all maintenance activities are scheduled on an as-needed basis (Mobley, 2002). PMPs have been increasingly adopted (Carnero, 2005); according to Mobley (2002), the average plant invests 15.8% of its annual maintenance budget in PMPs. Nevertheless, evidence points out that a high percentage of PMPs have limited use or are eliminated after brief periods of time because they fail to provide promised and measurable benefits (Mobley, 1997). This is certainly related to practical difficulties in evaluating maintenance performance in general (Waevenberg and Pintelon, 2002), in comprehensively defining a PMP which involves multiple users, equipment and facilities and in making the organizational changes for a full utilization of predictive tools (Mobley, 2002). However, above all, it is because there is a lack of proper tools to monitor and control factors that lead to the appearance and development of deficiencies during the life cycle of a PMP (Carnero, 2004). The multi-criteria model for auditing a PMP proposed in this paper is a contribution to fill this gap. Previous applications of multi-criteria decision analysis (MCDA) in other auditing contexts can be found in Akoka and Comyn-Wattiau, 1996, Comyn-Wattiau and Akoka, 1996, Bayraktar, 1998 and Pasiouras et al., 2007, but we did not find in the literature applications of MCDA to a PMP audit. The proposed multi-criteria audit model was developed and implemented in the General Hospital of Ciudad Real (GHCR) in Spain, within the scope of a R&D project with the University of Castilla-La Mancha. The project involved management, technical and consulting staff of GHCR and was motivated by the recognition of the importance of PM to pursue the hospital mission. In fact, the GHCR was the first hospital in Spain to set up a PMP. The project included the study of the potentialities of applying the MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) approach to PM (Carnero, 2009), which led to an initial prototype of the model (Carnero, 2006). MACBETH (Bana e Costa and Vansnick, 1994, Bana e Costa and Vansnick, 1999 and Bana e Costa et al., 2011) enables a quantitative PMP audit model to be built based on staff qualitative judgements about differences in attractiveness (or added value to the hospital) between reference levels of performance of the PMP in multiple audit dimensions. A brief technical overview of MACBETH is included in Section 3.1.1. Before that, Section 2 presents background information on the scope and purpose of the project. The model was developed as described in Section 3 and implemented in the GHCR in 2008. Results from the first year of application of the model to audit the hospital PMP are presented in Section 4. Finally, Section 5 discusses issues and lessons learned from the process of developing and implementing the model.

PM managers deemed the structuring phase in the development of the multi-criteria model at GHCR as particularly valuable, as it clarified what was to be achieved with the PMP, which were the PMP boundaries (i.e. what was to be included in the program) and which were the performance “targets”. Involving GHCR managers, technicians and consultants in structuring contributed to higher acceptance of the auditing model (as suggested by Scarf (1997)). Nonetheless, the model building process had to overcome four main difficulties. First, when collecting information to fill the questionnaire, the maintenance terminology used was not always easily understood by the technicians. Some modifications to the questionnaire were thus carried out. Second, the process was time consuming given the large number of criteria and amount of information required about PM activities and also because the construction of the model followed an iterative and dynamic approach, with successive adjustments being done until the model was found to be requisite (as defined by Phillips (1984)). Nevertheless, although time consuming at the development phase, this was recognized as a very useful learning process, and moreover, the cost of auditing the PMP with the new model has been shown to be low. It should be noted that the completeness of the model can only be ensured with its use along time, and that the model should be kept open to the inclusion of new criteria (for example, revealed by auditing incidents). However, in order to enable some stability in the comparison of successive audits, frequent revisions of the PMP audit model should be avoided. Third, the two-level areas and criteria structure of the model was adopted because PMP auditing involves inspecting multiple procedures and resources in distinct auditing areas, translated in an exhaustive and non-redundant set of no more than 10 auditing criteria within each area (as can be observed in Table 2). Four, the definition of the “base” and of the “target” levels for each criterion were key both to improve the intelligibility of the criteria and their independence. If the managers invoked other criteria when asked for the reference levels in a given criterion and their differences in attractiveness, then interdependences between criteria existed and therefore the structure of the model was not yet requisite and new interactions were needed. For instance, in some cases reference levels were adjusted accordingly, in particular to better set the “base” level so that it guaranteed that regular working conditions were accomplished. In other cases, interdependences arose between the quantity and quality components of performance, which required some restructuring to merge them in a single criterion (see example in Table 3) to keep valid the adoption of the simple additive aggregation model (1). A methodological alternative would be to explore more complex additive models to include additive components taking dependencies between criteria explicitly into account, as it is the case of the model developed by Clivillé et al. (2007) using the Choquet integral and MACBETH and of models discussed by Grabisch and Labreuche (2010). Another important methodological issue concerns the assessment of value judgments from the evaluators, in order to weight the criteria in the framework of the construction of the additive value model. In general, these judgments can be of different nature, depending on the specific weighting method used, e.g. judgments of indifference are asked in the trade-off technique and quantitative ratings in swing weighting (von Winterfeldt and Edwards, 1986, Keeney and Raiffa, 1993 and Edwards and Barron, 1994), and qualitative judgments in MACBETH. These methods use reference levels of performance, fixed in the multiple criteria, in their questioning procedures, which may be subject to anchoring effects. This phenomenon has been behaviorally observed in the common practice of weighting with reference to the most and least preferred levels of performance on the criteria (Borcherding et al., 1991 and Bottomley and Doyle, 2001). To avoid asking for unrealistic comparisons, it is advantageous to use “central rather than extreme” references (Belton and Stewart, 2002, p. 122), as done in the GHCR weighting process (see Section 3.2.2). This “clearly impacts the quality of the dialogue” (Bouyssou and Pirlot, 2005, p. 112) and has been empirically validated (Bana e Costa and Vansnick, 1997, Bana e Costa and Chagas, 2004 and Bana e Costa et al., 2008). However, the weights assessed with MACBETH could not be considered as requisite before validating their substantive meaning with the PM managers, by analyzing the trade-offs between performances implied by weights ratios, as detailed in Bana e Costa (2001). An alternative comprehensive validation would be to ask for holistic judgments between the separation profiles of the categories of accomplishment (see Section 3.2.3), following a questioning mode compatible with MACBETH, as in the GRIP method (Figueira et al., 2009), and checking the consistency with the weights assessed. This validation approach could be extended, to test for interactions between criteria using the Choquet integral, by following the developments of Angilella et al. (2010). GHCR managers regard the multi-criteria model as very helpful in monitoring the profiles of quantitative or qualitative performance of the PMP for all auditing areas and criteria, for tracing partial and overall value over time (in a sort of “tableau de bord”) and for estimating the added value of PMP improving interventions. The use of the model to communicate and discuss audit outputs and fine-tune PM policies, in a structured and procedural sequence, eased PMP-related decisions and prompted quick changes in auditing procedures. Furthermore, the PMP auditing contributed to a higher availability and reliability of equipments and thus to improved quality in healthcare delivery. Namely, the implementation of the audit process led to the detection of 102 alarms in machines that revealed malfunctions and an effective and timely reporting of problems was critical for avoiding increased waiting times and re-scheduling of surgeries. An important amelioration of the PMP multi-criteria auditing at GHCR would consist of the conception of a Decision Support System (DSS) facilitating the transmission and management of the model input and output data in a truly interactive “tableau de bord” format. The DSS would be implemented online, to permit a continuous PMP auditing, instead of at a discrete moment (each six months nowadays). It would also be desired to include in the DSS a resource allocation model, developed for instance with PROBE (Portfolio Robustness Evaluation, Lourenço et al., 2011). It would then be possible to prioritize the potential interventions taking into consideration not only their added values to the hospital estimated by the multi-criteria model, but also the associated costs. Despite PM being mainly applied to industrial companies, this study has shown that a PMP can be useful in a service company, and especially to a hospital, being one of the first studies to analyze this type of technological maintenance policy in that environment. We also believe in the potential of multi-criteria audit models to monitor and control other technology policies.