مدل سازی پاسخ به رگرسیون بردار پشتیبانی

Response modeling has become a key factor to direct marketing. In general, there are two stages in response modeling. The first stage is to identify respondents from a customer database while the second stage is to estimate purchase amounts of the respondents. This paper focuses on the second stage where a regression, not a classification, problem is solved. Recently, several non-linear models based on machine learning such as support vector machines (SVM) have been applied to response modeling. However, there is a major difficulty. A typical training dataset for response modeling is so large that modeling takes very long, or, even worse, modeling may be impossible. Therefore, sampling methods have been usually employed in practice. However a sampled dataset usually leads to lower accuracy. In this paper, we employed an ε-tube based sampling for support vector regression (SVR) which leads to better accuracy than the random sampling method.

A response model, given a mailing campaign, predicts whether each customer will respond or how much each customer will spend based on the database of customers’ demographic information and/or purchase history. Marketers will send mails or catalogs to customers who are predicted to respond or to spend large amounts of money. A well-targeted mail increases profit while a mistargeted or unwanted mail not only increases marketing cost but also may worsen a firm’s relationship with its customers (Gönül et al., 2000 and Potharst et al., 2000). Various methods have been used for response modeling such as statistical techniques (Bentz and Merunka, 2000, Haughton and Oulabi, 1997, Ling and Li, 1998 and Suh et al., 1999), machine learning techniques, (Cheung et al., 2003, Chiu, 2002, Shin and Cho, 2006, Viaene et al., 2001, Wang et al., 2005 and Yu and Cho, 2006) and neural networks (NN) (Bentz and Merunka, 2000, Potharst et al., 2000 and Zahavi and Levin, 1997). In general, there are two stages in response modeling. The first stage is to identify respondents from a customer database while the second stage is to estimate purchase amounts of the respondents. The response modeling task in the first stage has been usually formulated as a binary classification problem. The customers are divided into two classes, respondents and non-respondents. A classifier is constructed to predict whether a given customer will respond or not. However, as pointed out in KDD98 Cup (1998) for the KDD-CUP-98 task, there is an inverse correlation between the likelihood to buy and the dollar amount to spend (Wang et al., 2005). This is because the more dollar amount is involved, the more cautious a customer becomes in making a purchase decision. Hence, one may need a regression model, in a second stage, to estimate the purchase amount of responding customers. Support vector machine (SVM) has been recently spot-lighted with great generalization performances by employing the structural risk minimization (SRM) principle (Vapnik, 1995). Support vector regression (SVR), a regression version of SVM, was developed to estimate regression functions (Drucker, Burges, Kaufman, Smola, & Vapnik, 1997). Like SVM, SVR is capable of solving non-linear problems using kernel functions and has been successful in various domains (Drucker et al., 1997 and Müller et al., 1997). However, there is a difficulty to train SVR on real-world dataset. As the number of training patterns increases, SVR training takes much longer with a time complexity of O(n3) where n denotes the number of training patterns. So far, many algorithms such as Chunking, SMO, SVMlight and SOR have been proposed to reduce the training time. However, their training time complexity is still strongly related to the number of training patterns ( Platt, 1999). We take another direction called pattern selection which focuses on reducing the number of training patterns. Neighborhood property based pattern selection (NPPS) proposed by Shin and Cho (2003) is a powerful pattern selection method for SVM, but it is not for regression, but for classification. Recently, a pattern selection method based on the ε-tube (SVR-ε) was proposed which is specifically designed for SVR ( Kim & Cho, 2006). Thus, we employ SVR as a response model to predict an amount of money spent by each respondent. One can improve the performance of a response model by identifying profitable respondents instead of just respondents among all customers. Hence, after applying a classification model that predicts likelihoods to buy, one needs a regression model that predicts amounts to buy. Since classification is not our main concern, we assume that a perfect classifier exists which can identify all respondents without false positive (FP) errors. So, an SVR model is constructed on a subset of customers which consists only of respondents. The procedure of constructing the proposed response model is depicted in Fig. 1. In order to reduce the training time of SVR, we employ the pattern selection method. The DMEF4 dataset from the Direct Marketing Educational Foundation (DMEF) is used in our experiments. A small dataset is used to measure improvement in efficiency. However, for a very large dataset, some kind of sampling is inevitable, anyway. The remaining of this paper is organized as follows. The concept of SVR is briefly introduced and the main idea of the pattern selection method is presented in Section 2. In Section 3, the DMEF dataset and our experimental settings are described in detail. Section 4 presents the experimental results. Section 5 concludes this paper with remarks on limitations and future research directions.

In response modeling, while identifying respondents is considered important, estimating purchase amount of the respondents is as important. We applied SVR for response modeling to find more profitable customers. In our experiments, the SVR models could select profitable customers among respondents. The regression model would be useful for situations where the mailing cost is relatively high. Also, SVR training takes very long when the size of training data becomes large. To reduce the training time complexity of SVR, the ε-tube based pattern selection method was employed. SVR-ε was more efficient than SVR-All and more accurate than SVR-Random. In practice, however, it is always the case that a large dataset is available. To run SVR, some kind of sampling is required. The proposed ε-tube based sampling is then highly recommendable. There are some limitations in this research. First, it is unrealistic and/or impractical to assume a perfect classifier for respondent identification. Further research should be conducted that uses a “real” classification model. Second, SVR-ε was efficient, but it was inferior to SVR-All. An improvement of its performance would be desirable for its practical utility. Finally, parameter selection for pattern selection was carried out only in an empirical manner. A systematic guideline would be helpful.