[R] Bootstrap P-Value

[AbouEl-Makarim Aboueissa]

*Dear All:*

*I am trying to compute the p-value of the bootstrap test; please see
below.*

*In example 1 the p-value agrees with the confidence interval.*
*BUT, in example 2  the p-value DOES NOT agree with the confidence
interval. In Example 2, the p-value should be zero or close to zero.*

*I am not sure what went wrong, or not sure if I missed something.*

*any help would be appreciated.*


*with many thanks*
*abou*



#####  Two - Sample Bootstrap

#####  Source:
http://www.ievbras.ru/ecostat/Kiril/R/Biblio_N/R_Eng/Chernick2011.pdf

#####  Example 1:
#####  ----------



set.seed(1)

n1 <- 29
n1
x1 <- rnorm(n1, 1.143, 0.164) #some random normal variates: mean1 = 1.143
x1

n2 <- 33
n2
x2 <- rnorm(n2, 1.175, 0.169) #2nd random sample: mean2 = 1.175
x2

obs.diff.theta <- mean(x1) - mean(x2)
obs.diff.theta

theta <- as.vector(NULL) #### vector to hold difference estimates

iterations <- 1000

for (i in 1:1000) {                        #bootstrap resamples
 xx1 <- sample(x1, n1, replace = TRUE)
 xx2 <- sample(x2, n2, replace = TRUE)
 theta[i] <- mean(xx1) - mean(xx2)
 }



##### Confidence Interval:
##### --------------------


quantile(theta, probs = c(.025,0.975)) #Efron percentile CI on difference
in means

##### 2.5% 97.5%
##### - 0.1248539 0.0137601


##### P-Value
##### -------

p.value <- (sum (abs(theta) >= obs.diff.theta) + 1)/ (iterations+1)

#####  p.value <- (sum (theta >= obs.diff.theta) + 1)/ (iterations+1)

p.value



#### R OUTPUT

#### > quantile(theta, probs = c(.025,0.975))
####        2.5%       97.5%
#### -0.12647744  0.02099391

#### > p.value <- (sum (abs(theta) >= obs.diff.theta) + 1)/ (iterations+1)
#### > p.value
#### [1] 1

#####  Example 2:
#####  ----------


set.seed(5)

n1 <- 29
### n1
x1 <- rnorm(n1, 10.5, 0.15) ######   sample 1 with mean1 = 10.5
### x1

n2 <- 33
### n2
x2 <- rnorm(n2, 1.5, 0.155) #####  Sample 2 with mean2 = 1.5
### x2

obs.diff.theta <- mean(x1) - mean(x2)
obs.diff.theta

theta <- as.vector(NULL) #### vector to hold difference estimates

iterations <- 1000

#####   bootstrap resamples

for (i in 1:1000) {
 xx1 <- sample(x1, n1, replace = TRUE)
 xx2 <- sample(x2, n2, replace = TRUE)
 theta[i] <- mean(xx1) - mean(xx2)
 }



##### Confidence Interval:
##### --------------------


######  CI on difference in means

quantile(theta, probs = c(.025,0.975))



##### P-Value
##### -------

p.value <- (sum (abs(theta) >= obs.diff.theta) + 1)/ (iterations+1)

##### p.value <- (sum (theta >= obs.diff.theta) + 1)/ (iterations+1)

p.value

##### R OUTPUT

####   > ######  CI on difference in means
####   >
####   > quantile(theta, probs = c(.025,0.975))
####       2.5%    97.5%
####   8.908398 9.060601

####   > ##### P-Value
####   > p.value <- (sum (abs(theta) >= obs.diff.theta) + 1)/ (iterations+1)

####   > p.value
####   [1] 0.4835165

______________________


*AbouEl-Makarim Aboueissa, PhD*

*Professor, Statistics and Data Science*
*Graduate Coordinator*

*Department of Mathematics and Statistics*
*University of Southern Maine*

	[[alternative HTML version deleted]]