Content Disclaimer Copyright @2020. All Rights Reserved. |
Links : Home Index (Subjects) Contact StatsToDo |
Explanations and References
This page provides 3 statistical tests for paired differences in scalar measurements. Theey are the Paired t Test, the Wilcoson, Paired Signed Rank Test, and the Permutation Test for paired differences.
Javascript Program
Paired t Test
We purports to conduct a study to test whether the order of birth resulted in differences in the birth weight of twins. We obtained the birth weights (grams) from 20 sets of twins, which are shown in the first two columns in the table to the right, and the differences (Twin1 - Twin 2), as shown in the third columns. The calculations are as follows
Mean Difference = -101 Standard Deviation of Difference = 456 Standard Error of Mean = 105 t Test : t = -0.9659 df = 19 p(α) = 0.17 (1 tail) and 0.35 (2 tail) 95% Confidence Interval of Paired Difference (1 tail) = >-282 or <80 95% Confidence Interval of Paired Difference (2 tail) = -320 to 118 Wilcoxon Paired Signed Rank Test
We purports to study whether a new analgesic is effective in relieving headaches. We ask the subjects to describe their headache as none (0), some (1), moderate (2), and severe (3), a 4 point scale (0 to 3), before and after administering the analgesics, and use the paired differences to evaluate the analgesics. We collected data from 22 subjects. The headache scales are as shown in the first 2 columns, and the paired difference (Before - After) in column 3 of the table to the left.
There were 8 subjects whose headache scores did not change (0), and these are not included in the table of counts. On the negative side, there were:
As the numbers included in the count is 14, less than 16, Segal's Table H is used to determine statistical significance, and this is p<0.05. We can therefore conclude that headaches decreased significantly after receiving the analgesic. Permutation Test for paired differencesThe general principle underlying the permutation of paired differences is that, in a randomly allocated study, the data obtained could have been in either of the paired measurements. In ther words, each paired difference can either be + or - of that difference. The test therefore consists of calculating every possible permutation of the paired differences, and examine the results. If the results from the original data is near the extremes (e.g. less than 5 percentile or more than 95 percentile in a one tail model), then a decision can be made that it is unlikely to be null and therefore statistically significant. The advantages of using the Permutation test are :
The disadvantages of using the test are related to the computation resources (time and memory) required. The number of permutation is 2^{n}, were n is the number of pairs. Computation time therefore increases exponentially with increasing sample size, and large dataset may either crash the program when available RAM is exhausted, or the computation becomes unacceptably too long. In theory, the Permutation Test can cope with any number of pairs. However, α<0.05 cannot be computed with less than 6 pairs unless the differences are uniformly in one direction, and computation will take an unacceptably long time with 22 pairs or more. The Permutation Test is therefore ideal for handling small sets of interval data with uncertain distributions. With larger sample size, Wilcoxon PSRT for non-parametric data or Paired t test for parametric data should be preferred. The mathematical argument of the Permutation Test is as follows
The following example uses the same data as that presented in the Paired t Test.
Comparison of the 3 methodsUsing the same twin data by the 3 tests provides the following results
ReferencesPaired t Test :Armitage P. Statistical Methods in Medical Research (1971). Blackwell Scientific Publications. Oxford. P.189-207 Wilcoxon Paired Signed Rank Test :
Permutation Test : Siegel S and Castellan Jr. NJ (2000) Nonparametric Statistics for the Behavioral Sciences. Second Edition. McGraw Hill, Sydney. ISBN0-07-100326-6 p. 95-101.
Program 1: Paired t test# Paired t test # data entry dat = (" V1 V2 3163 3124 3245 2807 3391 3014 2547 2727 3042 3254 3200 3826 3115 2596 3294 2952 3019 3279 3222 3325 2831 2984 3043 2765 2646 3109 3327 2757 3182 3781 2984 3061 2878 3658 2770 3665 3092 2739 2735 3324 ") df <- read.table(textConnection(dat),header=TRUE) # conversion to data frame df$Diff <- df$V1 - df$V2 #df # optional display of input data and paired differences # Clculation: paired t n = nrow(df) # sample size degFm = n - 1 # degrees of freedom mean = mean(df$Diff) # mean of paired difference sd = sd(df$Diff) # Standard Deviation of paired differences se = sd / sqrt(degFm) # Standard Error of mean t = mean / se # paired t p = 1 - pt(abs(t),degFm) # probability of Type I Error α # 95% confidence interval t = qt(1 - 0.05, df=degFm) # t for 95% CI 1 tail ll1 = mean - t * se # lower limit 1 tail ul1 = mean + t * se # upper limit 1 tail t = qt(1 - 0.025, df=degFm) # t for 95% CI 2 tail ll2 = mean - t * se # lower limit 2 tail ul2 = mean + t * se # upper limit 2 tail # result output c(n, mean, sd) # sample size, mean difference and SD difference c(degFm, se, t, p, p*2) # deg freedom, Standard Error t, p(α) 1 and 2 tail c(ll1, ul1) # 95% CI 1 tail c(ll2, ul2) # 95% CI 2 tailThe results are as follows > # result output > c(n, mean, sd) # sample size, mean difference and SD difference [1] 20.0000 -101.0500 455.9936 > c(degFm, se, t, p, p*2) # deg freedom, Standard Error t, alpha (1 and 2 tail) [1] 19.0000000 104.6121044 2.0930241 0.1731023 0.3462046 > c(ll1, ul1) # 95% CI 1 tail [1] -281.93822 79.83822 > c(ll2, ul2) # 95% CI 2 tail [1] -320.0057 117.9057 Program 2 Wilcoxon Paired Signed Rank Test# Data input dat = (" V1 V2 3 1 2 1 2 0 3 0 1 1 1 0 2 2 1 2 1 1 3 1 1 2 1 1 1 1 3 0 3 1 1 2 1 1 1 3 1 1 3 1 3 2 1 1 ") df <- read.table(textConnection(dat),header=TRUE) # conversion to data frame df$Diff <- df$V1 - df$V2 # paired difference df # optional display of input data and paired difference # Make Wilcoxon table with non-equal cases x <- vector() # rank of non-zero differences in abs values y <- vector() # neg or pos ranks for(i in 1:nrow(df)) { v = df$Diff[i] # v = diff if(v !=0) # non-zero v { x <- append(x, abs(v)) # abs diff # abs(diff) y <- append(y, v / abs(v)) # sign + or - 1 # -1 or +1 } } n = length(x) # number of non-zero differences x <- rank(x) # abs diff ranked #x mx <- xtabs(~ x + y) # matrix +/- ranks # calculate Wicoson W ranks <- rownames(mx) # ranks as value array #ranks # optional display during debug tp = 0; # T+ sum of positive ranks tm = 0; # T- sum of negative ranks for(i in 1 : nrow(mx)) { r = as.numeric(ranks[i]) # rank of ith row tm = tm + mx[i,1] * r # T+ sum of positive ranks tp = tp + mx[i,2] * r # T- sum of negative ranks } t = tp if(tm>t) t = tm # use the larger of the 2 sums for calculation # significance testing if(n>15) # large sample size use z and its probability { z = (t - n * (n + 1) / 4) / sqrt(n * (n + 1) * (2 * n + 1) / 24) p = 1-pnorm(abs(z)) res = sprintf("z=%.4f p=%.4f", z, p) } else # small sample size use coefficients { Sig <- c(c(0,0,0),c(0,0,0),c(0,0,0),c(0,0,0),c(0,0,0),c(15,0,0), # 1-6 c(19,0,0),c(24,28,0),c(31,45,0),c(37,42,0),c(45,50,55), # 7-11 c(53,59,65),c(61,69,76),c(70,79,87),c(80,90,99),c(90,100,111)) # Table H. Siegal p.332-334 mxSig <- matrix(data = Sig, nrow=3,ncol=16) # converted to matrix for reference nn = n + 1 if(n<5) { res = "p = n.s." } else if(mxSig[3,nn]>0 && t>=mxSig[3,nn]) { res = "p = <0.001"; } else if(mxSig[2,nn]>0 && t>=mxSig[2,nn]) { res = "p = <0.01"; } else if(mxSig[1,nn]>0 && t>=mxSig[1,nn]) { res = "p = <0.05"; } else { res = "p = n.s."; } } # result output mx # matrix of +/- ranks c(nrow(df), n, tm, tp) # sample size, pairs different, Sum Rank + and - res # statistical significanceThe results are as follows
> # result output > mx # matrix of +/- ranks y x -1 1 3.5 3 3 9.5 1 5 13.5 0 2 > c(nrow(df), n, tm, tp) # sample size, pairs different, Sum Rank + and - [1] 22 14 20 85 > res # statistical significance [1] "p = <0.05" > Program 3: Permutation of paired differencesSection 1: declare global variabes# Global variables rows <- 0 # number of rows (pairs) sumDiff <- 0 # sum of paired difference from input data arV <- vector() # array of abs(paired diff) from all pairs nLess <- 0 # number of pairs with values less than sumDiff nSame <- 0 # number of pairs with values same as sumDiff nMore <- 0 # number of pairs with values more than sumDiffSection 2: subroutine for recursive calculation of paired differences All variables except col, sum, sumPlus and sumMinus are globally assigned and read Recurse <- function(col,sum) # col = pair number, sum = sum of paired diff { sumPlus = sum + arV[col] # sum with +ve [aired diff sumMinus = sum - arV[col] # sum with -ve [aired diff if(col==(rows)) # reaching last row { if(abs(sumPlus-sumDiff)<1e-10) # sum == sumDiff (allowing minor differences by binary math) { assign("nSame", nSame+1, envir = .GlobalEnv) # add to nSame } else { if(sumPlus<sumDiff) { assign("nLess", nLess+1, envir = .GlobalEnv) # add to nLess } else { assign("nMore", nMore+1, envir = .GlobalEnv) # add to nMore } } if(abs(sumMinus-sumDiff)<1e-10) { assign("nSame", nSame+1, envir = .GlobalEnv) # add to nSame } else { if(sumMinus<sumDiff) { assign("nLess", nLess+1, envir = .GlobalEnv) # add to nLess } else { assign("nMore", nMore+1, envir = .GlobalEnv) } } return() } arCount <- Recurse(col+1,sumPlus) # recursive call if not end row arCount <- Recurse(col+1,sumMinus) # recursive call if not end row return () }Main program # input data dat = (" V1 V2 3163 3124 3245 2807 3391 3014 2547 2727 3042 3254 3200 3826 3115 2596 3294 2952 3019 3279 3222 3325 2831 2984 3043 2765 2646 3109 3327 2757 3182 3781 2984 3061 2878 3658 2770 3665 3092 2739 2735 3324 ") df <- read.table(textConnection(dat),header=TRUE) # conversion to data frame rows <- nrow(df) # number of pairs df$Diff <- df$V1 - df$V2 # paired differences #df # optional check t debug sumDiff <- sum(df$Diff) # sumDiff, sum of paired diff from input data arV <- abs(df$Diff) # vector of abs(paired differences for use in permutation col = 1 # start permutation with first row sum = 0 # start permutation with sum of diff = 0 arCount <- Recurse(col, sum) # permutate calculations subroutine # Assemble results nTot = nLess + nSame + nMore # total number of possible permutations pLess = nLess / nTot # proportion with smaller effect size pSame = nSame / nTot # proportion with the same effect size pMore = nMore / nTot # proportion with larger effect size # probability of obtaining reference effect size or more extreme p = min(pLess,pMore) + pSame # significance (probability, α) # result output c(rows, sumDiff) # number of pairs and sum of paired diff from input data c(nTot, nLess, nSame, nMore) # result numbers c(pLess, pSame, pMore) # result proportions p # probability of sum(x) or more extreme (p, α)The results are as follows > # result output > c(rows, sumDiff) # number of pairs and sum of paired diff from input data [1] 20 -2021 > c(nTot, nLess, nSame, nMore) # result numbers [1] 1048576 173010 262 875304 > c(pLess, pSame, pMore) # result proportions [1] 0.1649951935 0.0002498627 0.8347549438 > p # probability of sum(x) or more extreme (p, α) [1] 0.1652451 |