Links : Home Index (Subjects) Contact StatsToDo

Explanations and References The references on this page includes the text book for the Javascript algorithm, and a web based teaching article on multiple regression. Other than these, it is assumed that users coming to this page will have other access to information and advice on the subject. The explanations and discussions that follows are therefore intended to help users follow the procedures and interpret the results, and not meant to be teaching or authoratative in nature. Example used on this page Mage Mht Gest Sex Bwt Col 1 Col 2 Col 3 Col 4 Col 5 24 170 37 1 3048 29 161 36 0 2813 29 167 41 1 3622 21 165 36 1 2706 35 168 35 0 2581 27 161 39 0 3442 26 163 40 1 3453 34 167 37 0 3172 25 165 35 1 2386 28 170 39 0 3555 32 167 37 1 3029 31 169 37 0 3185 26 161 36 1 2670 21 165 38 0 3314 21 166 41 1 3596 24 164 38 0 3312 34 169 38 0 3414 25 161 41 0 3667 26 167 40 0 3643 27 162 33 1 1398 27 160 38 1 3135 21 167 39 1 3366 An example is provided to help users to follow the information and results in this page. It should be noted that the data presented here are artificially generated to demonstrate the procedures, and not real. Also that the sample size is deliberately small so they are easier to visualize, that in real life an adequate sample size will be very much larger. See subject index to find estimating sample size for multiple regression. The example is to develop a model to predict birth weight of babies, using multiple regression. The independent predictive variables are maternal age in years (Mage), maternal height in cms (Mht), gestation age in weeks since beginning of pregnancy (Gest), sex of the baby (Sex, 0 for boys and 1 for girls). The dependent variable is birth weight in grams (Bwt). For this exercise we will use 22 cases. The data is presented in the table to the right. Please note that, the Javascript program on this page allows any number of variables (columns of data), but designate the last column to the right as dependent variable. Again please note that many other variables are related to birth weight, and included in widely published models. Only 4 are included here to demonstrate the procedures and results Correlation analysis Correlation Coefficients Cols 1 2 3 4 5 1 1 0.2622 -0.2511 -0.3803 -0.0975 2 0.2622 1 0.0708 -0.1312 0.235 3 -0.2511 0.0708 1 -0.1089 0.9237 4 -0.3803 -0.1312 -0.1089 1 -0.3222 5 -0.0975 0.235 0.9237 -0.3222 1 Partial Correlation Coefficients Cols 1 2 3 4 5 1 -1 0.2185 -0.1468 -0.3183 0.0418 2 0.2185 -1 -0.3771 0.2392 0.4395 3 -0.1468 -0.3771 -1 0.4605 0.9493 4 -0.3183 0.2392 0.4605 -1 -0.5476 5 0.0418 0.4395 0.9493 -0.5476 -1 The program firstly calculates the correlation and partial correlation coefficients between all variables, and present them in tables as shown above. As well as serving any needs users may have, the purpose here is a check on correlations between the independent variables, as excessively close correlation between these may indicate that they may have same precursors, and risk introducing a bias. The partial correlation coefficient reflects correlation between 2 variables after correcting for correlations with other variables in the data set. A large difference between partial and non-partial coefficients therefore reflect the possibility of excessive overlapping of measurements. Multiple Regression Col Reg (b) SEb t Sig (p,α) 1 1.70 9.86 0.17 0.8651 2 23.65 11.72 2.02 0.0598 3 223.19 17.92 12.45 <0.0001 4 -209.15 77.51 -2.70 0.0152 Constant(a) = -9165.48 Multiple Regression Coefficient R=0.961 F = 51.36, df=4,17, p<0.0001 Multiple regression analysis creates an equation which predicts birth weight (Bwt) from maternal age (Mage), maternal height (Mht), gestational age (Gest) and sex of the baby (Sex). The results are shown in the table to the right. Equation: y(col 5) = -9164.45(col 1) + 1.70(col 2) + 23.65(col 3) - 223.19(col 4) Col 1, Mage, increases Bwt by 1.7g per year, and is not statistically significant (p=0.97) Col 2, Mht, increases Bwt by 22.7g per cm, statistically significant at the p<0.1 level Col 3, Gest, increases Bwt by 223.2g per week, statistically highly significant at p<0.0001 Col 4, Sex, decreases Bwt by 209.2g if its a girl, statistically significant at the p<0.05 level The equation, as a whole, has a multiple correlation coefficient of R=0.96, statistically highly significant with p<0.0001 Multiple Standardized Regression Multiple standardized regression is the same as multiple regression, except that all measurements are standardized to the Standard Deviation unit z, where z = (value - mean) / SD. The coefficients produced are therefore of the same scale, making the structure of relationships between variables easier to visualize. Each partial standardized regression coefficient (β) represents the change in the dependent variabe (y) in number of SDs, for each 1 SD change of the independent variable. The difference between the βs also reflect their relative influence on the dependent variable. Col PSReg (β) SEβ t Sig (p,α) 1 0.01 0.08 0.17 0.8651 2 0.14 0.07 2.02 0.0598 3 0.90 0.07 12.45 <0.0001 4 -0.20 0.07 -2.70 0.0152 Constant(a) = 0 Multiple Regression Coefficient R=0.961 F = 51.36, df=4,17, p<0.0001 The results are as shown in the table to the right Equation: y(z5) = 0.01(z1) + 0.14(z2) + 0.98(z3) - 0.20(z4) An incresse of 1 SD in maternal age affects 0.01 SD increase in birth weight, a trivial effect An incresse of 1 SD in maternal height affects 0.14 SD increase in birth weight, 14 times the effect of maternal age An incresse of 1 SD in gestational age affects 0.98 SD increase in birth weight, almost in tandem, 98 times the effect of maternal age and 7 time that of maternal height An incresse of 1 SD in Sex affects 0.2 SD decrease in birth weight, 20 times the effect of maternal age, 1.4 times that of maternal height, and 0.2 times (20%) that of gestation References Steel RGD, Torrie JH, Dickey DA (1997) Principles and procedures of statistics. A biomedical approach. 3rd Ed. McGraw-Hill Inc New York NY 10020 ISBN 0-07-061028-2 p. 322-351 https://wiki.gis.com/wiki/index.php/Multiple_Regression a detailed explanation of multiple regression available on line Javascript Program Data Entry The data is a matrix of measurements   - Each row from a case   - Each column a variable   - The last column to the right is the dependent variable y in multiple regression R Codes The following is a single program, divided into parts so it is easier to follow Part 1. Data entry # Data entry to dataframe myDat = (" Mage Mht Gest Sex Bwt 24 170 37 1 3048 29 161 36 0 2813 29 167 41 1 3622 21 165 36 1 2706 35 168 35 0 2581 27 161 39 0 3442 26 163 40 1 3453 34 167 37 0 3172 25 165 35 1 2386 28 170 39 0 3555 32 167 37 1 3029 31 169 37 0 3185 26 161 36 1 2670 21 165 38 0 3314 21 166 41 1 3596 24 164 38 0 3312 34 169 38 0 3414 25 161 41 0 3667 26 167 40 0 3643 27 162 33 1 1398 27 160 38 1 3135 21 167 39 1 3366 ") df <- read.table(textConnection(myDat),header=TRUE) #summary(df) # optional display of input data Part 2: Means and Standard Deviations # mean and SD meanMage = mean(df$Mage) sdMage = sd(df$Mage) meanMht = mean(df$Mht) sdMht = sd(df$Mht) meanGest = mean(df$Gest) sdGest = sd(df$Gest) meanSex = mean(df$Sex) sdSex = sd(df$Sex) meanBwt = mean(df$Bwt) sdBwt = sd(df$Bwt) # show means and Sds c(meanMage,sdMage) c(meanMht,sdMht) c(meanGest,sdGest) c(meanSex,sdSex) c(meanBwt,sdBwt) The means and SD results are as follows > # show means and Sds > c(meanMage,sdMage) [1] 26.954545 4.281491 > c(meanMht,sdMht) [1] 165.227273 3.191235 > c(meanGest,sdGest) [1] 37.772727 2.136571 > c(meanSex,sdSex) [1] 0.5000000 0.5117663 > c(meanBwt,sdBwt) [1] 3113.955 532.697 Part 3: Multiple regression RegRes<-lm(Bwt~Mage+Mht+Gest+Sex,data=df) # Multiple regression summary(RegRes) # show multiple regression results The results are as follos > summary(RegRes) # show multiple regression results Call: lm(formula = Bwt ~ Mage + Mht + Gest + Sex, data = df) Residuals: Min 1Q Median 3Q Max -469.89 -86.47 46.49 84.17 198.44 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -9165.476 1945.069 -4.712 0.000201 *** Mage 1.701 9.864 0.172 0.865124 Mht 23.649 11.724 2.017 0.059759 . Gest 223.194 17.920 12.455 5.68e-10 *** Sex -209.150 77.511 -2.698 0.015228 * --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: 163.7 on 17 degrees of freedom Multiple R-squared: 0.9236, Adjusted R-squared: 0.9056 F-statistic: 51.36 on 4 and 17 DF, p-value: 2.849e-09 Part 4: Repeat Multiple regression using Standardized values Standardized values z = (value-mean) / SD Part 4a: Create standardized z values # standardization # create z variables df$ZMage <- (df$Mage - meanMage) / sdMage df$ZMht <- (df$Mht - meanMht) / sdMht df$ZGest <- (df$Gest - meanGest) / sdGest df$ZSex <- (df$Sex - meanSex) / sdSex df$ZBwt <- (df$Bwt - meanBwt) / sdBwt Part 4b: Standardized multiple regression using z values RegZRes<-lm(ZBwt~ZMage+ZMht+ZGest+ZSex,data=df) # Multiple regression summary(RegZRes) # show multiple regression results The results are as follows. For all variables mean = 0 and SD = 1 > summary(RegZRes) # show multiple regression results Call: lm(formula = ZBwt ~ ZMage + ZMht + ZGest + ZSex, data = df) Residuals: Min 1Q Median 3Q Max -0.88209 -0.16233 0.08728 0.15800 0.37252 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -1.267e-16 6.551e-02 0.000 1.0000 ZMage 1.367e-02 7.928e-02 0.172 0.8651 ZMht 1.417e-01 7.024e-02 2.017 0.0598 . ZGest 8.952e-01 7.188e-02 12.455 5.68e-10 *** ZSex -2.009e-01 7.447e-02 -2.698 0.0152 * --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: 0.3073 on 17 degrees of freedom Multiple R-squared: 0.9236, Adjusted R-squared: 0.9056 F-statistic: 51.36 on 4 and 17 DF, p-value: 2.849e-09 To make the coefficients easier to read,it is trans;ated as follows Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0 0.0655 0.0000 1.0000 ZMage 0.0137 0.0793 0.1720 0.8651 ZMht 0.1417 0.0702 2.0170 0.0598 . ZGest 0.8952 0.0719 12.4550 <0.0001 *** ZSex -0.2009 0.0745 -2.6980 0.0152 *