This page provides explanations, clarifications, and supports to Linear Discriminant Analysis, as a web page based php program in the Discriminant Analysis Program Page , and as a R program in a latter panel on this page.

Discriminant Analysis is clearly and succinctly described in Wikipedia. and general theories and descriptions of the model will not be repeated here.

However, Discriminant Analysis was initially developed over a century ago, and options on how the algorithm is used have since developed. The algorithm in StatsToDo chose different options to some of the statistial packages in the following manner.

• Discriminant Analysis was created as a variant of multiple regression, based on the least squares analysis of variance. The coefficients were calculated using the covariance matrix derived from the values of the independent variables. Experience in using the method showed that the results can be distorted by differences in the scales used to represent the independent variables. For example, using weight in pounds and height in inches will produce different results to when the same data are represented as cms and kgms.

  By firstly converting all measurements to its standard deviates z, where z=(value-mean) / Standard Deviation, all measurements can be converted to the same scalar with mean of 0 and Standard Deviation of 1, and the results obtained would be consistent, regardless of the units of measurement in the original data. Some but not all statistical packages offer this option, but the programs described here and in Discriminant Analysis Program Page uses the z values by default, so the results are sometimes different to those produced by the commonly used statistical packages

• The Linear Discriminant functions are estimated using Principal Component Analysis, and the maximum number of fuctions produced is one less than the number of outcome groups. As these functions are based on Principal Component extractions, they have a heirarchy of precision, and the earlier extracted functions contains more discriminant capability than the later ones, particularly if there are correlations between the independent variables. Experience shows that, in some cases, the use of all Discriminant functions produced actually reduces the precision of the algorithm, as the latter functions contain mostly statistical noise.

  However, most statistical packages continue to present the use of all Discriminant functions, which is useful to understanding the structure of the model produced, but less useful when the functions are used to predict outcome in future and different sets of data. The program in the Discriminant Analysis Program Page provides 2 buttons, allowing the user to choose whether all functions, or only those found to be statistically significant (and therefore containing less random noise) are used

The contents of the other panels on this page are as follows

• Example explains how the program works, and how the results can be interprested, using the algorithm and default example data from the Discriminant Analysis Program Page
• R presents the basic program in the R language. This allows users familiar or willing to learn R to use the codes as a template to begin deveolop his/her own program independent to the www and StatsToDo
• References is a short list of references for Discriminant Analysis

The panel describes and explains the procedures used and results presented by the Discriminant Analysis Program Page , using the default example data to demonstrate.

tannin	color	acidity	sugar	Wine
1.2	45	3.16	72.7	SR
1.3	67	3.38	102.4	SR
1.1	48	3.61	33.7	SR
1.6	36	3.51	58.2	SR
1.5	47	3.20	44.2	DR
1.5	74	3.21	91.8	DR
1.7	47	3.39	53.1	DR
1.6	56	3.36	88.5	DR
1.1	27	3.30	36.3	SW
1.0	53	3.55	74.7	SW
0.9	37	3.23	94.2	SW
1.2	23	3.07	53.8	SW
1.4	44	3.34	20.7	DW
1.3	34	3.24	9.5	DW
1.1	37	3.24	17.8	DW
1.4	55	3.35	35.9	DW

Default Example Data

The default example data are artificially created to demonstrate the algorithm, and does not reflect reality. It purports to be a study to discriminate the type of wine (SR for sweet red, DR for dry red, SW for sweet white, and DW for dry white). It uses 4 measurements to do so, col 1 is measurement of tannin, col 2 color, col 3 acidity, and col 4 sugar. Sixteen (16) wines are used to create the model, 4 from each type. The data table is as shown to the right, and entered (without the headers) in the data input text area in the program

Options

The program provides two buttons for 2 options in how the results are presented

• Calculate Discriminant Functions from Data (Use All Functions) presents all of the discriminant functions produced (number of outcome groups -1). This is the standard output in most algorithms for Discriminant Analysis. With the default example where there are 4 outcome groups, all 3 functions are used
• Calculate Discriminant Functions from Data (Use Only Significant Functions) uses only those discriminant functions tested as statistically significant by the Chi Squares Test (p<0.05). With the default example with 4 outcome groups, all 3 functions are initially produced. However, as the third function is not statistically significant, only the first 2 functions are used to produce the results for presentation

Part 1 of Program. Create the Model

Part 1, to create the Disciminant model, is the same for both options, regardless of which buttion is chosen.

Grp	1	2	3	4
Names	DR	DW	SR	SW
Number of cases nc=16 Number of variables nv=4 Number of groups ng=4

Step 1. The program places the outcome group names in alphabetical order, and label them as groups 1, 2, and so on. The reason for doing so is to reduce the space required when presenting results, in case there are large number of outcome groups with long names. The table for naming groups is as shown to the right, followed by the basic parameters of case numbers variable numbers, and outcome group numbers

var	Means	SD
1	1.3063	0.2351
2	45.625	13.564
3	3.3213	0.1461
4	55.4688	29.3126

Step 2. This calculates the means and Standard Deviations (SD) for all the independent variables. The means and Standard Deviations estimated using the reference data are assumed to be population parameters, and can be used validely with future data that use this model. The table of means and Standard Deviations are shown in the table to the right. Please note that, with the default example data, variable 1=tannin 2=color, 3=acidity, and 4=sugar

	ChiSq	df	p
Func1	92.0895	12	<0.0001
Func2	19.3497	6	0.0036
Func3	1.8596	2	0.3946

Step 3. This calculates the standard deviates (z) for all independent variables, where z = (value-mean)/Standard Deviation. z values are not displayed, but are used for all subsequent calculations

	Func1	Func2	Func3#
z1	-1.371	0.7463	0.1365
z2	-1.7209	-0.4157	-0.2198
z3	0.6709	0.2961	-0.8886
z4	1.5797	1.4083	0.2128

Step 4. This calculates the coefficients for each combination of function and standardized variable values z. It then calculates whether its contribution to discrimination is statistically significant, using the Chi Square Test. The table of significance is shown to the left, and the coefficients to the right. The # character is used to identify those functions that are not statistically significant. Please note that, in the coefficient table, the columns are functions, and the rows are for the z values of each independent variable. The function coefficients are used to estimate function scores.

	Func1	Func2	Func3#
C1	-2.276	1.141	0.279
C2	-1.558	-1.6395	-0.0286
C3	0.6466	0.6087	-0.5464
C4	3.1874	-0.1101	0.2961

Step 5. The Linear Discriminant fu.nction coefficients are then used to estimate the centroid value (equivalent to the mean) for each combination of functio and outcome, and the results are as shown to the left. The centrod values will be used to estimate the distance between individual scores and the center, an intermediary step towards estimating the probabilities of belonging to each of the outcome groups. Please note; that the table of centroids have columns for functions, and rows for centroid values for each outcome group.

This concludes Part 1 of the program, the development of the Discriminant model, and the tables produced are the parameters of the model. Part 2 of program consists of the description of the model produced.

Part 2 of Program. Description of the Model Created in Part 1

Depending on the button clicked in Discriminant Analysis Program Page , the choice of either using all Discriminant functions created (with default example data this is 3), or only those that are statistically significant (with default example data this is 2 as function 3 is not statistically significant). The following explanations describes both, but users should be aware that only one of the results will be shown by the program, depending on which button is clicked.

	Function Scores			Maximum Likelihood of Belonging to Group
Row	Sf1	Sf2	Sf3	MLp1	MLp2	MLp3	MLp4
1	0.89	0.18	1.05	0.01	0.02	0.81	0.17
2	0.12	1.70	-0.37	0.08	0.00	0.92	0.00
3	1.05	-1.19	-2.07	0.00	0.06	0.89	0.05
4	0.52	1.74	-0.80	0.02	0.00	0.98	0.01
5	-2.47	-0.21	0.75	0.66	0.33	0.00	0.00
6	-3.28	1.27	0.59	0.99	0.00	0.00	0.00
7	-2.28	1.23	-0.23	0.97	0.01	0.01	0.00
8	-1.07	2.28	0.01	0.83	0.00	0.17	0.00
9	2.44	-1.05	0.17	0.00	0.00	0.08	0.92
10	2.94	0.19	-1.55	0.00	0.00	0.19	0.81
11	5.13	0.65	0.74	0.00	0.00	0.00	1.00
12	2.25	-0.23	1.82	0.00	0.00	0.06	0.94
13	-2.13	-1.29	-0.29	0.05	0.94	0.00	0.00
14	-1.34	-2.04	0.35	0.00	0.99	0.00	0.00
15	-0.11	-2.36	0.24	0.00	0.97	0.03	0.00
16	-2.66	-0.87	-0.41	0.20	0.79	0.00	0.00

	Function Scores		Maximum Likelihood of Belonging to Group
Row	Sf1	Sf2	MLp1	MLp2	MLp3	MLp4
1	0.89	0.18	0.00	0.01	0.92	0.07
2	0.12	1.70	0.09	0.00	0.90	0.00
3	1.05	-1.19	0.00	0.11	0.68	0.21
4	0.52	1.74	0.03	0.00	0.96	0.01
5	-2.47	-0.21	0.62	0.38	0.01	0.00
6	-3.28	1.27	0.99	0.01	0.00	0.00
7	-2.28	1.23	0.98	0.01	0.01	0.00
8	-1.07	2.28	0.82	0.00	0.18	0.00
9	2.44	-1.05	0.00	0.00	0.10	0.90
10	2.94	0.19	0.00	0.00	0.07	0.93
11	5.13	0.65	0.00	0.00	0.00	1.00
12	2.25	-0.23	0.00	0.00	0.23	0.77
13	-2.13	-1.29	0.06	0.93	0.00	0.00
14	-1.34	-2.04	0.00	0.99	0.00	0.00
15	-0.11	-2.36	0.00	0.97	0.03	0.00
16	-2.66	-0.87	0.23	0.77	0.00	0.00

Step 6 This combines the z values of the independent variables and the Discriminant function coefficients to estimate the function scores of each function for each case (row of data). The function scores are then collated in the multidimensional Euclidean space, and their distance to each centrod computed. The distances are then converted to probabilities of belonging to each group. These probabilities are named Maximum Likelihood, as the assumptions that the a priori probability of belonging to each group is the same regardless of the sample sizes in the reference data.

Two tables are shown here. The one on the left uses all 3 functions, and on the right the 2 significant functions.

Please Note: the program displays results with 4 decimal point precision, but these tables presents only 2 decimal points precision to conserve space and to make the tables easier to read.

	Alloc(1)	Alloc(2)	Alloc(3)	Alloc(4)	Total
Desig(1)	4	0	0	0	4
Desig(2)	0	4	0	0	4
Desig(3)	0	0	4	0	4
Desig(4)	0	0	0	4	4
Total	4	4	4	4	16

Step 7 The group with the highest probability is the group that case is allocated to. This is compared with the original group designation in the data. A table of counts is thus produced, where the numbers in the diagonal represent the number of cases correctly allocated by the model, while of off diagonal those erroneously allocated. With Only 1 table is shown here as, with the default reference data, the allocations perfectly reflects the designation with both options. The table is shown to the right.

Step 8 This produces 2 dimensional plots of the scores between Linear Discriminant function scores. Each line in the plot joins the function scores with the centroid value of the designated group. With the default example data and all functions used, 3 plots are produced. If only significant functions are included, then only 1 plot, between the first and second functions, will be presented. The plots are shown as follows.

Step 8 This reproduces the model parameters in the form of Javascript codes, which can be copied and pasted onto html pages, to be used remote from the www and when interacted with new data. A template html file DiscriminantTemplate.html is provided to incorporate and use these parameters. As details on how to import the parameters and edit the html files are presented in the template file itself, they are not elaborated here.

Summary, Commentary, and Technical Issues

Discriminant Analysis was introduced by Fisher more than a century ago, as a variant of the multiple regression model, but with multinomial groups as dependent variable and normally distributed measurements as independent variable. It enjoys continued usage, but with time many modifications and additions are made, producing options that users may wish to choose, and producing results that are similar but numerically different. This confusion is further aggravated by some statistical packages keeping to the original algorithm, some including useful additions and and modifications, and some providing menus from which users can choose different options. The following comments attempts to clarify some of these issues

• The original algorithm used actual measurements of independent variables and their covariate matrix for Principal Component extraction. The problem is that the results can be distorted if the scalars of different variables differ widely. For example, using height in inches and weight in pounds will produce different results from the same data using cms and Kgs. Increasingly therefore the original measurements are normalized to Standard Deviation units z, where z=(value-mean)/SD, so that all measurements are converted to have a mean of 0 and Standard Deviation of 1 before they are used.

  Statistical packages vary in how this problem is managed. The algorithm descibed on this page uses z values by default, but provides the calculated means and Standard Deviations as model parameters to be used in DiscriminantTemplate.html, so that z transformations can be made with future data.

• The number of Linear Discriminant functions created by the original algorithm is one less than the number of outcome groups (nf=ng-1). The calculation of probabilities and plotting of these functions requires comparing each function with every others, the total number of comparisons is therefore Factorial (number of functions). With increasingly large number of outcome groups, the display of results can become unweildy and confusing.

  However, it can be clearly demonstrated that, because the functions are essentially transforms of Principal Components, the functions extracted earlier have greater discriminating power than the latter ones, as the latter ones contain mostly statistical noise. To ignore the more trivial functions therefore will create only minor distortions to the numerical results, and in most cases not altering how the cases are allocated to the groups.

  The effects of not using non-significant functions are clearly demonstrated in the tables of Part 2, as shiown previously. The difference between 2 and 3 functions are the third decimal points of the probability calculations, and there are no difference in how the cases are allocated using the default example data. It is not the case whether it is more correct to use all or only significant functions. Rather that the inclusion of non-significant functions does not materially affect the interpretation of the results.

  In order not to invite unnecessary controversy or criticism, the full number of functions should be formally presented, especially if the analysis is offered as a proof of some hypothesis. For discussion and summary purposes however, the exclusion of non-significant functions greatly reduce the complexity and confusion in the results without adversely affecting interpretation.

• There are two major reasons for using Discriminant analysis. Firstly, to analyse the structure in the data itself, and interpret the results as scientific realities represented by the data. Secondly, to use the data to create a model, and use that model to make decisions on future and different set of similar measurements. Both approaches require the allocation of outcome groups, based on the probabilities that are calculated from the Discriminant Functions. How the probabilities are calculated, however, differ according to the purpose of the analysis.

  If the purpose is to analysis and interpret the data (and the data set is representative of the population), then the calculation of probability should include the a priori probability of each outcome (Bayesean Probability), represented by the sample size in each outcome groups. If the purpose is to develop a model to interpret future data, then the sample sizes of different groups can be arbitrarily determined by the model creator, and not relevant to interpretation. Probabilities can therefore be calculated assuming the same a priori probability for all groups (Maximum Likelihood).

  To demonstrate this difference, we can imagine an analysis relating a set of symptoms to discriminate headache caused by tension or by brain tumor. To study the relationship between symptoms and diagnosis, a data set representative of the clinical scenario (say from medical records) is used, which contains many cases of tension and few cases of brain tumor. The calculation of probabilities should therefore take the difference in prevalence into consideration (Bayesean). However, if the purpose is to develop a tool to make diagnosis for future data, the developer may select similar number of cases with each diagnosis to create the model, and the probabilities merely describes how well the model performs. As the relative sample sizes do not represent prevalence but arbitrarily determined, the Maximum Likelihood is a better description of the model, and Bayesean probabilities should be calculated only when the model is used in the community as a diagnostic tool, using the expected prevalence in the community as a priori probabilities.

  Statistical packages differ in the inclusion of a priori probabilities. Many includes a priori probabilities as the default, estimated from the sample sizes of the outcome groups in the data. Some use Maximum Likelihood as default. Some require users to insert the priori probabilities. All these approaches are correct in the right context, and the user is left with the option to choose.

  The program in the Discriminant Analysis Program Page , and described here, produces the Maximum Likelihood as probabilities in the outcome groups, to describe the relationship between the variables in the model, but not necessrily the data. The parameters resulted from the program can be exported as Javascript to DiscriminantTemplate.html, which allows the inclusion of a priori probabilities to calculate Bayesean probabilities.

The Linear Discriminant analysis using R was carried out to check the accuracy of the web based php program in the Discriminant Analysis Program Page . Only the minimum amount of coding is used. User can search R for the numerous version of calculation and graphic support for Discriminant analysis.

The R code

myDat = ("
v1    v2   v3     v4     Grp
1.2   45   3.16   72.7   SR
1.3   67   3.38  102.4   SR
1.1   48   3.61   33.7   SR
1.6   36   3.51   58.2   SR
1.5   47   3.20   44.2   DR
1.5   74   3.21   91.8   DR
1.7   47   3.39   53.1   DR
1.6   56   3.36   88.5   DR
1.1   27   3.30   36.3   SW
1.0   53   3.55   74.7   SW
0.9   37   3.23   94.2   SW
1.2   23   3.07   53.8   SW
1.4   44   3.34   20.7   DW
1.3   34   3.24    9.5   DW
1.1   37   3.24   17.8   DW
1.4   55   3.35   35.9   DW
") 
myDataFrame <- read.table(textConnection(myDat),header=TRUE)      

myDataFrame$z1<-(myDataFrame$v1-mean(myDataFrame$v1)) / sd(myDataFrame$v1)
myDataFrame$z2<-(myDataFrame$v2-mean(myDataFrame$v2)) / sd(myDataFrame$v2)
myDataFrame$z3<-(myDataFrame$v3-mean(myDataFrame$v3)) / sd(myDataFrame$v3)
myDataFrame$z4<-(myDataFrame$v4-mean(myDataFrame$v4)) / sd(myDataFrame$v4)

myDataFrame

#install.packages("MASS")   # if not already installed
library(MASS)
fit <- lda(Grp ~ z1 + z2 + z3 + z4, data=myDataFrame)  
fit
predict(fit,newdata=myDataFrame,prior=c(1,1,1,1)/4)$x          #calculate faction scores
predict(fit,newdata=myDataFrame,prior=c(1,1,1,1)/4)$posterior  #calculate probabilities

The Code and Results Explained

Please note that R codes are in maroon, and results in blue

Step 1. Data entry

myDat = ("
v1    v2   v3     v4     Grp
1.2   45   3.16   72.7   SR
1.3   67   3.38  102.4   SR
1.1   48   3.61   33.7   SR
1.6   36   3.51   58.2   SR
1.5   47   3.20   44.2   DR
1.5   74   3.21   91.8   DR
1.7   47   3.39   53.1   DR
1.6   56   3.36   88.5   DR
1.1   27   3.30   36.3   SW
1.0   53   3.55   74.7   SW
0.9   37   3.23   94.2   SW
1.2   23   3.07   53.8   SW
1.4   44   3.34   20.7   DW
1.3   34   3.24    9.5   DW
1.1   37   3.24   17.8   DW
1.4   55   3.35   35.9   DW
") 
myDataFrame <- read.table(textConnection(myDat),header=TRUE)      

Please note that the headers are included in R as they are required to call the algorithm

Step 2. Calculate the Standard Deviate z=(v-mean/SD)

myDataFrame$z1<-(myDataFrame$v1-mean(myDataFrame$v1)) / sd(myDataFrame$v1)
myDataFrame$z2<-(myDataFrame$v2-mean(myDataFrame$v2)) / sd(myDataFrame$v2)
myDataFrame$z3<-(myDataFrame$v3-mean(myDataFrame$v3)) / sd(myDataFrame$v3)
myDataFrame$z4<-(myDataFrame$v4-mean(myDataFrame$v4)) / sd(myDataFrame$v4)

Step 3. Display the data object, including the calculated z values

summary(myDataFrame)

The results are

    v1 v2   v3    v4 Grp          z1         z2         z3          z4
1  1.2 45 3.16  72.7  SR -0.45185501 -0.0460777 -1.1033570  0.58784387
2  1.3 67 3.38 102.4  SR -0.02657971  1.5758573  0.4019983  1.60105898
3  1.1 48 3.61  33.7  SR -0.87713031  0.1750953  1.9757788 -0.74264062
4  1.6 36 3.51  58.2  SR  1.24924619 -0.7095966  1.2915264  0.09317656
5  1.5 47 3.20  44.2  DR  0.82397089  0.1013709 -0.8296560 -0.38443326
6  1.5 74 3.21  91.8  DR  0.82397089  2.0919275 -0.7612308  1.23944012
7  1.7 47 3.39  53.1  DR  1.67452149  0.1013709  0.4704235 -0.08080988
8  1.6 56 3.36  88.5  DR  1.24924619  0.7648898  0.2651478  1.12686066
9  1.1 27 3.30  36.3  SW -0.87713031 -1.3731154 -0.1454036 -0.65394166
10 1.0 53 3.55  74.7  SW -1.30240561  0.5437168  1.5652273  0.65607384
11 0.9 37 3.23  94.2  SW -1.72768091 -0.6358722 -0.6243803  1.32131609
12 1.2 23 3.07  53.8  SW -0.45185501 -1.6680127 -1.7191841 -0.05692938
13 1.4 44 3.34  20.7  DW  0.39869559 -0.1198020  0.1282973 -1.18613545
14 1.3 34 3.24   9.5  DW -0.02657971 -0.8570452 -0.5559551 -1.56822331
15 1.1 37 3.24  17.8  DW -0.87713031 -0.6358722 -0.5559551 -1.28506892
16 1.4 55 3.35  35.9  DW  0.39869559  0.6911655  0.1967226 -0.66758765

Please note: z1, z2, z3, and z4 are the z values for v1, v2, v3, and v4

Step 4. Perform Linear Discriminant analysis and display results

#install.packages("MASS")   # if not already installed
library(MASS)
fit <- lda(Grp ~ z1 + z2 + z3 + z4, data=myDataFrame)  
fit

Please note: the calculations are based on the z values and not the original measurements

Prior probabilities of groups:
  DR   DW   SR   SW 
0.25 0.25 0.25 0.25 

Group means:
            z1         z2         z3         z4
DR  1.14292737  0.7648898 -0.2138289  0.4752644
DW -0.02657971 -0.2303885 -0.1967226 -1.1767538
SR -0.02657971  0.2488196  0.6414866  0.3848597
SW -1.08976796 -0.7833209 -0.2309352  0.3166297

Coefficients of linear discriminants:
          LD1        LD2        LD3
z1  1.3710438 -0.7462890  0.1364612
z2  1.7208986  0.4156835 -0.2198221
z3 -0.6709095 -0.2961205 -0.8885895
z4 -1.5797134 -1.4082862  0.2128178

Proportion of trace:
   LD1    LD2    LD3 
0.7899 0.1899 0.0202 

The prior probabilities are calculated from the sample sizes of the groups

The LD1, 2, and 3 are the 3 Linear Discriminant functions

The proportion of trace represents the proportion of discriminating power of each function, and can be used to test for statistical significance

Step 5. Calculate function scores

predict(fit,newdata=myDataFrame,prior=c(1,1,1,1)/4)$x          #calculate function scores

please note that the prior term is actually unnecesary, as it is not used when function scores are calculated

When the data object is called as newdata, a separate sets of data can be used, providing the the appropriately labelled independent variables are present (in this example z1, z2, z3, and z4)

          LD1        LD2          LD3
1  -0.8871802 -0.1830651  1.054003243
2  -0.1234701 -1.6988951 -0.366512945
3  -1.0536723  1.1881588 -2.071887461
4  -0.5220622 -1.7409330 -0.801348480
5   2.4680678  0.2142880  0.745565901
6   3.2824521 -1.2654109  0.592784805
7   2.2823362 -1.2330374 -0.228987474
8   1.0710619 -2.2798046  0.006542448
9  -2.4349834  1.0478052  0.172180524
10 -2.9365081 -0.1894505 -1.548469220
11 -5.1313958 -0.6508717  0.740034619
12 -2.2466446  0.2331076  1.820538675
13  2.1281402  1.2850848 -0.285692745
14  1.3390091  2.0367135  0.345040379
15  0.1061805  2.3646456  0.240614788
16  2.6586689  0.8716648 -0.414407058

Step 6. Calculate the posterior (Bayesean) Probability of belonging to each group

predict(fit,newdata=myDataFrame,prior=c(1,1,1,1)/4)$posterior  #calculate posteriori (Bayesean) probabilities

Please note that the prior term for apriori probability is used here. If this is left out, the program assumes the prior probabilities are the same as that in the reference data, depending on the sample sizes of the groups there.

             DR           DW           SR           SW
1  1.027719e-02 1.738244e-02 0.8056435151 1.666969e-01
2  7.584478e-02 1.695596e-03 0.9196816594 2.777961e-03
3  2.543569e-04 5.741198e-02 0.8883330540 5.400061e-02
4  1.793459e-02 5.428730e-04 0.9760628920 5.459645e-03
5  6.615467e-01 3.338972e-01 0.0045559379 1.919581e-07
6  9.948814e-01 4.791425e-03 0.0003272035 5.284128e-10
7  9.744939e-01 1.355788e-02 0.0119480765 1.254452e-07
8  8.326310e-01 1.399772e-03 0.1659475333 2.173213e-05
9  2.634318e-06 5.441063e-04 0.0758854570 9.235678e-01
10 7.201969e-07 1.160473e-05 0.1924352689 8.075524e-01
11 9.464596e-12 1.011071e-10 0.0001827173 9.998173e-01
12 2.029302e-05 2.288982e-04 0.0560473803 9.437034e-01
13 5.420201e-02 9.416221e-01 0.0041755258 3.779071e-07
14 4.865341e-03 9.917932e-01 0.0033348804 6.545613e-06
15 7.663001e-04 9.729023e-01 0.0250258317 1.305537e-03
16 2.029808e-01 7.940588e-01 0.0029603250 4.639190e-08

when translated to normal numerical format with 2 decimal point precisions

	DR	DW	SR	SW
1	0.01	0.02	0.81	0.17
2	0.08	0.00	0.92	0.00
3	0.00	0.06	0.89	0.05
4	0.02	0.00	0.98	0.01
5	0.66	0.33	0.00	0.00
6	0.99	0.00	0.00	0.00
7	0.97	0.01	0.01	0.00
8	0.83	0.00	0.17	0.00
9	0.00	0.00	0.08	0.92
10	0.00	0.00	0.19	0.81
11	0.00	0.00	0.00	1.00
12	0.00	0.00	0.06	0.94
13	0.05	0.94	0.00	0.00
14	0.00	0.99	0.00	0.00
15	0.00	0.97	0.03	0.00
16	0.20	0.79	0.00	0.00

The function coefficients, scores, and probabilities are the same as that produced by the program in Discriminant Analysis Program Page , other than minor discrepancies caused by different rounding errors.

Genral references on Discriminant analysis

Wikipedia on Discriminant Analysis.

George D and Mallery P (1999) SPSS for Windows Step by Step. A Simple Guide and Reference. Allyn and Bacon, Sydney. ISBN 0-205-28395-0 Chapter 26. The Discriminant Procedure p.313-328.

Resources used to developed the web bases php program

These are very old books, that still present actual formulae and algortithms for all step in the calculations. Most newer references do not provide detailed algorithms, but advise users to access available packages such as SAS, SPSS, R, and Python

Overall JE and Klett CJ (1972) Applied Multivariate Analysis. McGraw Hill Series in Psychology. McGraw Hill Book Company New York. Library of Congress No. 73-14716407-047935-6

• Chapter 2 p.24-56 : Matrix math. Particularly the Square Root method of matrix inversion. Also calculations for between and within group Sum Product and Covariance matrices
• Chapter 10 p.280-306 : Multiple Discriminant Analysis, particularly the algorithm.
• Chapter 13 p. 345-371 Normal Probability Density Model for classification.
• Chapter 14 p. 373-383 Use of Canonical Correlates for Classification. Chapters 13 and 14 provided the algorithms for calculating Maximum Likelihood and the Bayesian Probabilities

Press WH, Flannery VP, Teukolsky SA, Vetterling WT (1989). Numerical Recipes in Pascal. Cambridge University Press IBSN 0-521-37516-9 p.395-396 and p.402-404. Jacobi method for finding Eigen values and Eigen vectors

Norusis MJ (1979) SPSS Statistical Algorithms Release 8. SPSS Inc Chicago Chapterr 23 : Discriminant p. 69-83. Formulae for algorithm provided by SPSS.

Useful references and advice on Discriminant analysis using R

https://www.geeksforgeeks.org/linear-discriminant-analysis-in-r-programming/

https://www.statmethods.net/advstats/discriminant.html

Venables, W. N. and Ripley, B. D. (2002) Modern Applied Statistics with S. Fourth edition. Springer.