Multiple linear regression with special properties of its coefficients parameterized by exponent, logit, and multinomial functions is considered. To obtain always positive coefficients the exponential parameterization is applied. To get coefficients in an assigned range, the logistic parameterization is used. Such coefficients permit us to evaluate the impact of individual predictors in the model. The coefficients obtained by the multinomial–logit parameterization equal the shares of the predictors, which is useful for interpretation of their influence. The considered regression models are constructed by nonlinear optimization techniques, have stable solutions and good quality of fit, have simple structure of the linear aggregates, demonstrate high predictive ability, and suggest a convenient way to identify the main predictors.
Multiple linear regression is one of the main tools of statistical modeling widely used for estimation of a dependent variable by its predictors. Regressions are very effective for prediction, but are not always useful for the analysis and interpretation of the individual predictors’ input due to multicollinearity effects. Multicollinearity distortion of the regression coefficients is well known and described in numerous works, for instance [1], [2], [3] and [4]. Beginning from a one-parameter ridge-regression approach [5], [6] and [7], various other techniques have been developed for overcoming the effects of multicollinearity on the coefficients of regression, see, for instance, [8], [9], [10] and [11]. Among the latest innovations in regression and principal component analyses, a lot of attention has been paid to the regularization methods based on the quadratic L2-metric, lasso L1-metric, and other Lp-metrics and their combinations such as elastic net or sparse analysis [12], [13], [14], [15], [16] and [17].
The current paper considers another approach to constructing a sparse linear combination of the predictors in the regression model using the coefficient parameterization in a special form of exponential, logistic, and multinomial–logit functions. This approach is motivated by necessity to obtain a multiple regression, for instance, with positive coefficients if the pair correlations are positive as well. In many practical problems, particularly, in marketing and advertising research, all the predictors by their meaning should have a definite positive impact on the dependent variable, and it can be easily proven by their pair correlations. However, the coefficients in multiple regression being proportional to the partial correlations could often receive signs opposite to their pair relation signs. Of course, this can be attributed to multicollinearity effects, but it hardly helps in interpretation of the model and in estimation of the individual predictors’ contribution.
In these situations, the exponent parameterization of a linear model’s coefficients always produces positive coefficients, or coefficients with the signs of their pair correlations. Logistic parameterization can be used to attain all the coefficients in any assigned range of values, for instance from zero to one. Multinomial-logit parameterization yields coefficients with their total equal to one, so such coefficients directly present the shares of the predictors’ impact on the response variable.
Estimation of the parameterized coefficients can be performed by an optimization objective reduced to a Newton–Raphson procedure for nonlinear equations [18], [19] and [20]. Regressions with special properties of the coefficients can be easier to interpret than ordinary regression models. Such regressions generate stable coefficients of a simple structure in the linear aggregate, demonstrate good prediction ability, and suggest a convenient way to identify the main predictors.
A similar parameterization technique has recently been applied in principal component analysis (PCA) and in singular value decomposition (SVD) to produce loadings with only positive elements, or elements totaling one hundred percent. In contrast to regular PCA and SVD, non-negative loadings have a clear meaning of variable contribution to data approximation and explicitly show which variables with which shares are composed at each step of approximation [21]. Application of the nonlinear parameterization for obtaining only non-negative weights has been considered for sample balance problems in [22].
The paper is arranged as follows. Section 2 presents regressions with several parameterization functions of the coefficients, and describes algorithms for their estimation. Section 3 discusses numerical results, and Section 4 summarizes.