st: bootstrapped p-values

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

st: bootstrapped p-values

Richard Harvey
Hi,

Could someone please explain how stata computes the bootstrap p-values?

suppose i issue the following commands

sysuse auto
bootstrap t=r(t), rep(1000)  saving(bsauto, replace): ttest mpg==0

i get


Bootstrap results                                               Number
of obs=        74
Replications       =      1000


Observed   Bootstrap            Normal-based
Coef.   Std. Err.       z       P>z     [95% Conf. Interval]

t    31.66644   2.900212        10.92   0.000     25.98213    37.35075


how does stata compute the P>z?

does it...

look at the proportion of bootstrapped t-values which are less than or
equal to the original t-value ( from
ttest mpg==0) say x and the proportion of bootstrapped t-values which
are greater than or equal to the original t-value ( from ttest mpg==0)
say y then computes p as 2 min(x,y) ?

--
thanks for your time
rich
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

st: RE: bootstrapped p-values

Nick Cox
No.

Look at the results Stata is giving you: you can reconstruct with
-display- the calculation that z = coef / se = 31.66644 /  2.900212 =
10.92 (2 d.p.). Then it's a matter of looking up P for z as a unit
Gaussian, in this example a mind-blowingly small value. That's why it
says "Normal-based".

Nick
[hidden email]

Richard Harvey

Could someone please explain how stata computes the bootstrap p-values?

suppose i issue the following commands

sysuse auto
bootstrap t=r(t), rep(1000)  saving(bsauto, replace): ttest mpg==0

i get


Bootstrap results                                               Number
of obs=        74
Replications       =      1000


Observed   Bootstrap            Normal-based
Coef.   Std. Err.       z       P>z     [95% Conf. Interval]

t    31.66644   2.900212        10.92   0.000     25.98213    37.35075


how does stata compute the P>z?

does it...

look at the proportion of bootstrapped t-values which are less than or
equal to the original t-value ( from
ttest mpg==0) say x and the proportion of bootstrapped t-values which
are greater than or equal to the original t-value ( from ttest mpg==0)
say y then computes p as 2 min(x,y) ?

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

st: R: bootstrapped p-values

Carlo Lazzaro
In reply to this post by Richard Harvey
Dear Richard,
I would refer you to Stata Archives and recommend to take a look at Maarten
Buis' and Martin Weiss' kind replies to one thread I have posted on 2008 Sep
6 (which was not so different from yours, as it would seem):

st: z statistics in bootstrap output

Kind Regards and enjoy your W-E,
Carlo
-----Messaggio originale-----
Da: [hidden email]
[mailto:[hidden email]] Per conto di Richard Harvey
Inviato: venerdì 5 dicembre 2008 18.40
A: [hidden email]
Oggetto: st: bootstrapped p-values

Hi,

Could someone please explain how stata computes the bootstrap p-values?

suppose i issue the following commands

sysuse auto
bootstrap t=r(t), rep(1000)  saving(bsauto, replace): ttest mpg==0

i get


Bootstrap results                                               Number
of obs=        74
Replications       =      1000


Observed   Bootstrap            Normal-based
Coef.   Std. Err.       z       P>z     [95% Conf. Interval]

t    31.66644   2.900212        10.92   0.000     25.98213    37.35075


how does stata compute the P>z?

does it...

look at the proportion of bootstrapped t-values which are less than or
equal to the original t-value ( from
ttest mpg==0) say x and the proportion of bootstrapped t-values which
are greater than or equal to the original t-value ( from ttest mpg==0)
say y then computes p as 2 min(x,y) ?

--
thanks for your time
rich
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

Re: st: R: bootstrapped p-values

Richard Harvey
hi,

thanks nick and carlo..

I understand your point about the p value in the statout now. I am a
bit confused about some things i have read elsewhere

for example  the following

http://compdiag.molgen.mpg.de/ngfn/docs/2003/nov/Exercises_Bootstrap.pdf

discusses non parametric bootstrap on page 4 and on page 5 it talks about

"The two-sided p-values are now calculated by determining the
percentage of situation where the
absolute value of the test statistics calculated from the groups with
permutated values is at least
as large as the absolute value of the test statistics calculated from
the original groups."..

and the following page

westfall.ba.ttu.edu/Finance%20Pres.ppt

talks about something similar on page 11 and page 12

I am not what that means and how that is related to the stata p-value?
and the following page

westfall.ba.ttu.edu/Finance%20Pres.ppt

talks about something similar on page 11 and page 12

I am not what that means and how that is related to the stata p-value?

thanks
rich

2008/12/5 Carlo Lazzaro <[hidden email]>:

> Dear Richard,
> I would refer you to Stata Archives and recommend to take a look at Maarten
> Buis' and Martin Weiss' kind replies to one thread I have posted on 2008 Sep
> 6 (which was not so different from yours, as it would seem):
>
> st: z statistics in bootstrap output
>
> Kind Regards and enjoy your W-E,
> Carlo
> -----Messaggio originale-----
> Da: [hidden email]
> [mailto:[hidden email]] Per conto di Richard Harvey
> Inviato: venerdì 5 dicembre 2008 18.40
> A: [hidden email]
> Oggetto: st: bootstrapped p-values
>
> Hi,
>
> Could someone please explain how stata computes the bootstrap p-values?
>
> suppose i issue the following commands
>
> sysuse auto
> bootstrap t=r(t), rep(1000)  saving(bsauto, replace): ttest mpg==0
>
> i get
>
>
> Bootstrap results                                               Number
> of obs=        74
> Replications       =      1000
>
>
> Observed   Bootstrap            Normal-based
> Coef.   Std. Err.       z       P>z     [95% Conf. Interval]
>
> t    31.66644   2.900212        10.92   0.000     25.98213    37.35075
>
>
> how does stata compute the P>z?
>
> does it...
>
> look at the proportion of bootstrapped t-values which are less than or
> equal to the original t-value ( from
> ttest mpg==0) say x and the proportion of bootstrapped t-values which
> are greater than or equal to the original t-value ( from ttest mpg==0)
> say y then computes p as 2 min(x,y) ?
>
> --
> thanks for your time
> rich
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>



--
thanks for your time
rich

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

Re: st: R: bootstrapped p-values

Richard Harvey
hi all,

Also, Is there any relation between p values and the percentile based
confidence interval?

thanks
rich

2008/12/5 Richard Harvey <[hidden email]>:

> hi,
>
> thanks nick and carlo..
>
> I understand your point about the p value in the statout now. I am a
> bit confused about some things i have read elsewhere
>
> for example  the following
>
> http://compdiag.molgen.mpg.de/ngfn/docs/2003/nov/Exercises_Bootstrap.pdf
>
> discusses non parametric bootstrap on page 4 and on page 5 it talks about
>
> "The two-sided p-values are now calculated by determining the
> percentage of situation where the
> absolute value of the test statistics calculated from the groups with
> permutated values is at least
> as large as the absolute value of the test statistics calculated from
> the original groups."..
>
> and the following page
>
> westfall.ba.ttu.edu/Finance%20Pres.ppt
>
> talks about something similar on page 11 and page 12
>
> I am not what that means and how that is related to the stata p-value?
> and the following page
>
> westfall.ba.ttu.edu/Finance%20Pres.ppt
>
> talks about something similar on page 11 and page 12
>
> I am not what that means and how that is related to the stata p-value?
>
> thanks
> rich
>
> 2008/12/5 Carlo Lazzaro <[hidden email]>:
>> Dear Richard,
>> I would refer you to Stata Archives and recommend to take a look at Maarten
>> Buis' and Martin Weiss' kind replies to one thread I have posted on 2008 Sep
>> 6 (which was not so different from yours, as it would seem):
>>
>> st: z statistics in bootstrap output
>>
>> Kind Regards and enjoy your W-E,
>> Carlo
>> -----Messaggio originale-----
>> Da: [hidden email]
>> [mailto:[hidden email]] Per conto di Richard Harvey
>> Inviato: venerdì 5 dicembre 2008 18.40
>> A: [hidden email]
>> Oggetto: st: bootstrapped p-values
>>
>> Hi,
>>
>> Could someone please explain how stata computes the bootstrap p-values?
>>
>> suppose i issue the following commands
>>
>> sysuse auto
>> bootstrap t=r(t), rep(1000)  saving(bsauto, replace): ttest mpg==0
>>
>> i get
>>
>>
>> Bootstrap results                                               Number
>> of obs=        74
>> Replications       =      1000
>>
>>
>> Observed   Bootstrap            Normal-based
>> Coef.   Std. Err.       z       P>z     [95% Conf. Interval]
>>
>> t    31.66644   2.900212        10.92   0.000     25.98213    37.35075
>>
>>
>> how does stata compute the P>z?
>>
>> does it...
>>
>> look at the proportion of bootstrapped t-values which are less than or
>> equal to the original t-value ( from
>> ttest mpg==0) say x and the proportion of bootstrapped t-values which
>> are greater than or equal to the original t-value ( from ttest mpg==0)
>> say y then computes p as 2 min(x,y) ?
>>
>> --
>> thanks for your time
>> rich
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>
>
>
> --
> thanks for your time
> rich
>



--
thanks for your time
rich

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

R: st: R: bootstrapped p-values

Carlo Lazzaro

Dear Richard,
Nick's yesterday reply about p-value still holds.
Percentile based confidence interval (CI) are simply one out of four methods
(but please note that bias corrected and accelerated (BCa) CI should be set
before starting bootstrap procedure by thicking the apposite box in -
bootstrap options -) ,Stata offers you for calculating bootstrap. You can
simply invoke  - estat bootstrap, all - at the end of your yesterday's Stata
lines.

However, and this is more relevant, despite its computational appealing,
bootstrap percentile CIs are not generally recommended (Barber JA and
Thompson SG. Analysis of cost data in randomized trials: an application of
the non-parametric bootstrap. Statistics in Medicine 2000; 19:3219-3236).

Moreover, for a comprehensive coverage of bootstrap method, the usual
reference is: Efron B, Tibshirani RJ. An Introduction to the Bootstrap.
Chapman and Hall: New York, 1993

HTH,Kind Regards and enjoy your W-E,
Carlo

-----Messaggio originale-----
Da: [hidden email]
[mailto:[hidden email]] Per conto di Richard Harvey
Inviato: sabato 6 dicembre 2008 0.19
A: [hidden email]
Oggetto: Re: st: R: bootstrapped p-values

hi all,

Also, Is there any relation between p values and the percentile based
confidence interval?

thanks
rich

2008/12/5 Richard Harvey <[hidden email]>:

> hi,
>
> thanks nick and carlo..
>
> I understand your point about the p value in the statout now. I am a
> bit confused about some things i have read elsewhere
>
> for example  the following
>
> http://compdiag.molgen.mpg.de/ngfn/docs/2003/nov/Exercises_Bootstrap.pdf
>
> discusses non parametric bootstrap on page 4 and on page 5 it talks about
>
> "The two-sided p-values are now calculated by determining the
> percentage of situation where the
> absolute value of the test statistics calculated from the groups with
> permutated values is at least
> as large as the absolute value of the test statistics calculated from
> the original groups."..
>
> and the following page
>
> westfall.ba.ttu.edu/Finance%20Pres.ppt
>
> talks about something similar on page 11 and page 12
>
> I am not what that means and how that is related to the stata p-value?
> and the following page
>
> westfall.ba.ttu.edu/Finance%20Pres.ppt
>
> talks about something similar on page 11 and page 12
>
> I am not what that means and how that is related to the stata p-value?
>
> thanks
> rich
>
> 2008/12/5 Carlo Lazzaro <[hidden email]>:
>> Dear Richard,
>> I would refer you to Stata Archives and recommend to take a look at
Maarten
>> Buis' and Martin Weiss' kind replies to one thread I have posted on 2008
Sep

>> 6 (which was not so different from yours, as it would seem):
>>
>> st: z statistics in bootstrap output
>>
>> Kind Regards and enjoy your W-E,
>> Carlo
>> -----Messaggio originale-----
>> Da: [hidden email]
>> [mailto:[hidden email]] Per conto di Richard Harvey
>> Inviato: venerdì 5 dicembre 2008 18.40
>> A: [hidden email]
>> Oggetto: st: bootstrapped p-values
>>
>> Hi,
>>
>> Could someone please explain how stata computes the bootstrap p-values?
>>
>> suppose i issue the following commands
>>
>> sysuse auto
>> bootstrap t=r(t), rep(1000)  saving(bsauto, replace): ttest mpg==0
>>
>> i get
>>
>>
>> Bootstrap results                                               Number
>> of obs=        74
>> Replications       =      1000
>>
>>
>> Observed   Bootstrap            Normal-based
>> Coef.   Std. Err.       z       P>z     [95% Conf. Interval]
>>
>> t    31.66644   2.900212        10.92   0.000     25.98213    37.35075
>>
>>
>> how does stata compute the P>z?
>>
>> does it...
>>
>> look at the proportion of bootstrapped t-values which are less than or
>> equal to the original t-value ( from
>> ttest mpg==0) say x and the proportion of bootstrapped t-values which
>> are greater than or equal to the original t-value ( from ttest mpg==0)
>> say y then computes p as 2 min(x,y) ?
>>
>> --
>> thanks for your time
>> rich
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>
>
>
> --
> thanks for your time
> rich
>



--
thanks for your time
rich

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/