پیش بینی کوتاه مدت سرعت باد بر اساس الگوریتم رگرسیون بردار پشتیبانی تکاملی

Hyper-parameters estimation in regression Support Vector Machines (SVMr) is one of the main problems in the application of this type of algorithms to learning problems. This is a hot topic in which very recent approaches have shown very good results in different applications in fields such as bio-medicine, manufacturing, control, etc. Different evolutionary approaches have been tested to be hybridized with SVMr, though the most used are evolutionary approaches for continuous problems, such as evolutionary strategies or particle swarm optimization algorithms. In this paper we discuss the application of two different evolutionary computation techniques to tackle the hyper-parameters estimation problem in SVMrs. Specifically we test an Evolutionary Programming algorithm (EP) and a Particle Swarm Optimization approach (PSO). We focus the paper on the discussion of the application of the complete evolutionary-SVMr algorithm to a real problem of wind speed prediction in wind turbines of a Spanish wind farm.

The Support Vector Machine (SVM) (Vapnik, 1998) is a powerful and robust methodology in statistical machine learning, that has been successfully applied to regression problems (SVMr) (Akay, 2009, Cherkassky and Ma, 2004, He et al., 2008, Lázaro et al., 2005 and Wu et al., 2008), including problems of wind speed prediction (Mohandes et al., 2004 and Salcedo-Sanz et al., 2009). The SVMr is considered a good methodology because it allows the use of the kernel theory to increase the quality of regression models and also because, in most cases, it can be solved as a convex optimization problem. Several fast algorithms there exist to carry out the SVMr training, such as the sequential minimal optimization algorithm (Smola & Schölkopf, 1998). In spite of this, the SVMr performance heavily depends on the choice of several hyper-parameters, necessary to define the optimization problem and the final SVMr model. Unfortunately, there is not an exact method to obtain the optimal set of SVMr hyper-parameters, so that a search algorithm must be applied to obtain the best possible set of hyper-parameters which guarantees the maximum possible quality of the final model. In general, the search algorithms used to obtain SVMr hyper-parameters can be divided in three groups. The first group of algorithms for SVMr hyper-parameters search is based on grid searches ( Akay, 2009 and Mohandes et al., 2004), where the search space of parameters is divided into groups of possible parameters to be tested, usually in an uniform fashion. The second group of search algorithms is formed by local search type approaches, such as pattern search proposed in Momma and Bennett (2002). Finally, the third group of algorithms applied to obtain SVMr hyper-parameters is based on metaheuristics, or global optimization algorithms, such as evolutionary computation ( Hou and Li, 2009, Wang et al., 2005 and Wu et al., 2009). This latter set of methodologies include several approaches, such as genetic algorithms, evolutionary algorithms, particle swarm or differential evolution, etc., which are applied to implement a robust research on the hyper-parameters search space. This work is focused on this type of methodologies, because they have shown very good performance in previous applications. For example Friedrichs and Igel (2005) was one of the first approaches where the evolutionary optimization of SVM parameters was proposed. More recently, similar approaches have also been successfully applied to different problems, as in Hou and Li (2009), where an evolutionary strategy algorithm has been applied to obtain the SVM parameters in a problem of short-term fault prediction, or in Cheng and Wu (2009), where an evolutionary SVM approach has been proposed to a problem of construction management. Novel evolutionary computation algorithms have also been tested, for example a quantum-inspired evolutionary approach has been proposed in Luo et al. (2008), and a novel hybrid genetic algorithm for both selecting the optimal kernel function and optimizing its corresponding hyper-parameters has been proposed in Wu et al. (2009). A hybrid approach involving a genetic algorithm for selecting SVM hyper-parameters has been recently applied to a problem of mitochondrial toxicity prediction ( Zhang et al., 2009). In Wu, Tzeng, Goo, and Fang (2007) a real-coded genetic algorithm has been successfully applied to a problem of bankruptcy prediction. Also, different approaches based on Particle Swarm Optimization (PSO) algorithms can be found in the literature for the SVM hyper-parameters tuning. In Lin, Ying, Chen, and Lee (2008) a PSO has been successfully applied to the optimization of the SVM hyper-parameters together with a feature selection problem. A similar approach also including feature selection has been presented in Huang and Dun (2008). Novel PSO approaches have been proposed and tested in specific applications within the frame of SVM hyper-parameters tuning. For example, in Wang et al. (2009), an immune PSO approach is proposed to optimize the SVM parameters in an interesting problem of forest fire prediction, and in Wu (2010) and PSO with Gaussian mutation hybridized with a SVM is applied to a problem of power load forecast. It is easy to see that the evolutionary-based optimization of SVM hyper-parameters is really a hot topic, not only because the large amount of works applying these techniques, but also because the majority of works are very recent. In this paper we deal with evolutionary-based approaches for SVMr hyper-parameters tuning in a real problem of wind speed prediction. We compare the performance of an Evolutionary Programming algorithm (Yao, Liu, & Lin, 1999) and a PSO approach (Eberhart & Shi, 2001) for SVMr parameter tuning, and how the resulting algorithm can be integrated in a complete system for wind speed prediction in wind farms. The specific model for SVMr parameters estimation described in the paper starts from a coarse grid search to reduce the SVMr hyper-parameters search space size. Then, in a second step, the evolutionary-based algorithms are applied to refine the search, improving the performance of the SVMr. Note that the regression SVMr algorithm has been previously applied to a problem of wind speed in Mohandes et al. (2004). In that paper, the authors successfully applied the SVMr standard algorithm with grid search to obtain the best set of kernel parameters. In this paper we show that the evolutionary estimation of hyper-parameters performs better than the grid search. The complete Evolutionary SVMr algorithm has been tested integrated into the forecasting model presented in Salcedo-Sanz et al. (2009), in a wind farm located at the south of Spain. The rest of this paper is structured as follows: next section describes the main characteristics of the regression SVMr algorithm used in this paper. Section 3 presents the SVMr hyper-parameters model considered in this paper, consisting of a first coarse grid search, that will be refined using the evolutionary computation algorithms. Section 4 presents the basic prediction system where the proposed evolutionary SVMr approach will be integrated, and shows the good performance of this model in a problem of wind speed forecast in several turbines of a Spanish wind farm. Section 5 closes the paper with some final considerations and remarks.

In this paper we have presented a model of hyper-parameters search in Support Vector Machines for regression (SVMr) based on evolutionary computation, and how to incorporate this approach to a system of wind speed prediction. Specifically, two different evolutionary algorithms, Evolutionary Programming and Particle Swarm Optimization have been considered, and their performance in the problem of SVMr parameters tuning, has been tested in the frame of the wind speed prediction problem tackled. The complete system for prediction, including the evolutionary SVMr algorithm has shown very good performance in the wind speed forecast at an Spanish wind farm, outperforming a previous model using a multi-layer perceptron as final regression algorithm.