بهینه سازی شبکه های پس انتشار از طریق یک الگوریتم هیوریستیک کالیبره شده با آرایه متعامد

Improving the performance of neural networks is of considerable importance. Although previous studies have investigated how to design the optimal neural network, the heuristic algorithms developed to support the optimization process contain flaws. These heuristic algorithms do not perform efficiently and they require prior expert knowledge. This study commences by employing an orthogonal array using the Taguchi method to calibrate the factor levels of a heuristic algorithm and to estimate the percent contribution from various individual factors. Subsequently, the calibrated heuristic algorithm is used to optimize a back-propagation network (BPN). Changing the level of each individual factor systematically and then analyzing the main and interactive effects of the design factors by using the analysis of variance (ANOVA) leads to the optimal heuristic algorithm factor levels with regard to experimental cost. The proposed optimization procedure is demonstrated on the classification problems using the University of California’s Department of Information and Computer Science (ICS) server. The results indicate that the quality of the solution from the proposed approach is superior to that from a non-calibrated conventional design.

The back-propagation network (BPN) is the most popular form of artificial neural network (ANN), and has been successfully applied to solve many problems (Lee et al., 2005, Liu et al., 2001, Mendyk and Jachowicz, 2005, Quah and Srinivasan, 1999 and Salchenberger et al., 1992). Notwithstanding its popularity, traditional BPNs using the Gradient Steepest Descent Learning Algorithm (GSDLA) may be inconsistent and unpredictable for certain applications (Sexton, Dorsey, & Johnson, 1998a). This kind of BPN suffers two major shortcomings: (1) the GSDLA sometimes converges slowly to the optimal solution, and (2) the GSDLA may yield a poor solution since it can become trapped at a locally optimized solution (Yasumasa et al., 1994). Hence, improving the application performance of BPNs remains an important research issue. Search techniques such as Genetic Algorithms (GA), Tabu Search (TS), and Simulated Annealing (SA), represent potential means of improving the performance of BPNs. It has been shown that these techniques are particularly effective in optimizing the BPN performance (Sexton et al., 1998b, Sexton et al., 1999 and Yasumasa et al., 1994). Sexton used GA, TS, and SA for optimizing BPN, and has validated the priority of the BPN together with a heuristic algorithm. Moreover, in 1999, Sexton compared the relative performances of SA and GA in optimizing an ANN for a specific BPN topology (after a lot of trial-and-error). Optimizing the BPN parameters and topology is crucial to enhancing performance, since both aspects significantly influence the quality of the solution. Many heuristic algorithms require the factor levels to be optimized in order to improve performance. Clearly, different factor levels of a heuristic algorithm will influence the BPN performance (Kirkpatrick et al., 1983 and Sexton et al., 1999). Hence, it is necessary to calibrate the influential heuristic algorithm’s factors before the algorithm is applied to the BPN optimization task. It has been shown that the Taguchi method (Taguchi, 1986) is an effective tool for factor calibration (Forouraghi, 2000, Khaw et al., 1995 and Ko et al., 1999), and Taguchi’s orthogonal array can be applied to establish the optimal factor levels of a heuristic algorithm. Khoei, Masters, and Gethin (2002) presented an experimental investigation of the aluminum recycling process, in which the Taguchi method was used to determine the optimal configuration of process parameters such that the process performance and quality were both enhanced. Casab, Orsolya, Anna, Eya, and Lstyan (1999) also applied the Taguchi method in the field of ELISA (Enzyme Linked Immunosorbent Assay) optimization, and successfully reduced the interaction effects of the optimized variables such that it was possible to identify the optimum conditions, even when significant interactions existed between the assay variables. Nowadays, the Taguchi method is the method most commonly chosen when analyzing interaction effects and calculating the percent contribution of separate factors in order to screen and rank them (Roy, 1990). Furthermore, it is also an appropriate approach for solving problems characterized by continuous, discrete, and qualitative design variables (Lin & Tseng, 2000). Consequently, in this paper, we employ the signal to noise ratio (S/N), the analysis of variance (ANOVA), and the analysis of means (ANOM) from the Taguchi method ( Nelson and Dudewicz, 2002 and Roy, 1990) to investigate the main effects and interactions of a heuristic algorithm’s factor levels. We anticipate that the optimal combination of the adopted heuristic algorithm factors and the appropriate levels of each of these factors will yield a superior result for the BPN’s performance. The present study performs a three-step investigation into the BPN optimization procedure; namely (1) an orthogonal array of the Taguchi method is employed to identify the factor levels for a heuristic algorithm, (2) the calibrated heuristic algorithm is used to design the BPN’s parameters and topology in order to enhance its performance, and (3) the ANOVA of the Taguchi method is used to estimate individual percent contributions in order to rank and screen these controllable factors of the adopted heuristic algorithm. Fig. 1 provides an outline of the current research, which can be broadly summarized as follows: Step 1. Identify the BPN’s decision parameters and topology: The decision parameters and topology of a BPN include the initial weight values factor, learning rate, momentum factor, number of hidden layers, number of neurons in the first layer, and number of neurons in the second layer. After that, the calibrated heuristic algorithm is used to specify appropriate network values for the BPN. Step 2. Calibrate the heuristic algorithm factor levels: A suitable orthogonal array is employed to examine the levels of the heuristic algorithm factors in order to ensure the robustness of the experimental design. Step 3. Rank and screen factors of the heuristic algorithm: To estimate the percent contributed by individual factors, information about the relative merits of individual factors is obtained using the analysis of variance of the experimental results. Step 4. Pool factors of the heuristic algorithm: In order to reduce experimental cost and difficulty, it may be necessary to pool one or more of the controllable factors of the heuristic algorithm during the ranking and screening processes. Step 5. Optimize the BPN parameters and topology: The calibrated heuristic algorithm is used to optimize the BPN in order to enhance the BPN’s application performance. Step 6. Compare the performance of different factors of the heuristic algorithm: The solution quality of the BPN is compared when optimized by a heuristic algorithm with various combinations of calibrated factor levels.

This paper presents a three-stage optimization procedure for BPN networks. Initially, the factor levels of the heuristic algorithm were calibrated using the Taguchi orthogonal array approach. Subsequently, the heuristic algorithm controllable factors were re-evaluated by analyzing their main and interaction effects according to ANOVA. Finally, the BPN was optimized using a calibrated heuristic algorithm. Following this optimization procedure, the BPN was applied to an authentic data classification problem. The proposed BPN optimization procedure can facilitate the successful and efficient application of a BPN when applied to classification problems. Studying the ANOVA table for a given analysis helps to determine which factors need to be controlled and which do not. This determination is important to the decision maker. To adopt the approach of this study, decision makers may determine the levels of a heuristic algorithm’s controllable factors in advance in order to enhance the performance of a BPN model together with a heuristic algorithm. The examples illustrated show that a calibrated heuristic algorithm has greater effectiveness than a non-calibrated heuristic algorithm in the BPN’s data classification problem. The heuristic algorithm used in this work explores many possible settings following the calibration of an orthogonal indication before selecting the starting parameters of the heuristic algorithm, thus enhancing the possibility of finding better solutions.