Briefly describe the following three stepwise procedures in terms of how they build a regression model: forward selection, backward elimination, and stepwise selection. Which method do you prefer and why?

Answer:

To find the subset of variables to use in the find regression model it is natural to consider fitting models with various combination of the candidate regression.

1) forward selection

2) backword elimination

3) stepwise selection

• 1) forward selection :-

This procedure begins with assumption that there are no regressors in the model other than the intercept. An effort is made to find an optimal subset by inserting regressors into the model one at a time. The first regressor selected for entry into the equation is the one that has the largest simple correlation with the response variable y. Suppose that this regression is x3. This is also the regressor that will produce the largest value of the F-statistics for testing significant of regression. This regressor is entered if the F statistic exceeds a preselected F value say F_IN or F-to-enter or value of alpha.

The second regressor chosen for entry is the one that now has the largest correlation with y after adjusting for the effect of the first regressor entered (x3) on y such correlation are called as partial correlation. They are the simple correlations between the residuals from the regression.

Y^=B0^+B3^x3 and the residuals from the regressions of the other candidate regressors on x3,say xj^=a^0j+a^3j ; j=1,2,4.....k

Suppose that at step 2 the regressor with the highest partial correlation with y is x2 . This implies that the largest partial F-statistic is,

F= SS_R (x2/x3)/ MS_Res (x3,x2)

If this F value exceeds F-to-enter then x2 is added to the model.

In general, at each step the regressor having the highest partial correlation with y (or equivalent the largest partial F statistic given the other regressors already in the model) is added to the model if it's partial F - statistic exceeds the preselected entry level F-to-enter.

This procedure terminates either when the partial F-statistic at a particular step partial F-statistic at a particular step does not exceed F -to- enter.

This procedure terminates either when the partial F-statistic at a particular step does not exceed F-to-enter or when the last candidate regressor is added to the model.

• 2) Backword elimination :-

Forward selection begins with no regressors in the model and attempts to insert variables until a suitable model is obtained backward elimination attempts to find a good model by working in the opposite direction. that is being with a model that includes all k regressors. Then the partial F-statistic is computed for each regressor as if it were the last variable to enter the model. The smallest of these partial F statistics is compared with a preselected value Fout or F-to-remove. for example, and if the smallest partial F-value is less than F-to-remove that regressor is removed with the model. Now a regression model with k-1 regressors is fit, the partial F-statistics for this new model calculated, and the procedure repeated. The backwardelimination procedure terminates when the smallest partial F value is not less than the preselected cutoff value F -to- remove backword elimination is often a very good variable selected procedure.

• 3) stepwise regression :-

it is a modification of forward selection method in which at each step all regression entered into the model previously are regression via their partial F-statistics.

I prefer stepwise regression ,where variables are intered sequentially into the model but there is also a criteria to remove variable.

becase, with stepwise you combine forword and backwored in a sort of alteration.