Dear Richard,
I suppose that confusion arose because you are implicitly referring to two different things: - the meaning of p-value reported in bootstrap printout (and Nick's yesterday reply still holds): - the bootstrap p-value vs the original one in searching for the significance of the difference of the mean with skewed data. As widely reported in literature (please see for instance, Desgagne A, Castilloux A, Angers J, LeLorier J. The use of the bootstrap statistical method for the pharmacoeconomic cost analysis of skewed data. Pharmacoeconomics 1998; 13:487-497.Barber JA and Thompson SG. Analysis of cost data in randomized trials: an application of the non-parametric bootstrap. Statistics in Medicine 2000; 19:3219-3236), ttest procedure needs the fulfilments of some prerequisite before being invoked, which are systematically violated whenever data are skewed (please, take a look at the following example sketched via Stata 9.2/SE): ---------------------------begin example--------------------------- sysuse auto save "C:\Documents and Settings\carlo\Documenti\Statistiche\Stata\Richard_1.dta", replace ttest mpg, by(foreign) unequal scalar tobs=r(t) sum mpg, mean scalar omean=r(mean) sum mpg if foreign==0, mean replace mpg = mpg-r(mean) + scalar(omean) if foreign==0 sum mpg if foreign==1, mean replace mpg = mpg-r(mean) + scalar(omean) if foreign==1 sort foreign by foreign: sum mpg keep mpg foreign set seed 1 bootstrap r(t), reps(1000) nodots strata(foreign) saving(C:\Documents and Settings\carlo\Documenti\Statistiche\Stata\Richard_2.dta, every(1) replace)verbose : ttest mpg, by(foreign) unequal save "C:\Documents and Settings\carlo\Documenti\Statistiche\Stata\Richard_1.dta", replace use "C:\Documents and Settings\carlo\Documenti\Statistiche\Stata\Richard_2.dta", clear generate indicator =abs(r(t))>=abs(scalar(tobs)) sum indicator, mean display "ASLboot="r(mean) ------------------------------end example---------------------------- HTH, Kind Regards and enjoy your W-E, Carlo -----Messaggio originale----- Da: Richard Harvey [mailto:[hidden email]] Inviato: venerdì 5 dicembre 2008 23.37 A: Carlo Lazzaro Oggetto: Re: R: bootstrapped p-values hi, thanks nick and carlo.. I understand your point about the p value in the statout now. I am a bit confused about some things i have read elsewhere for example the following http://compdiag.molgen.mpg.de/ngfn/docs/2003/nov/Exercises_Bootstrap.pdf discusses non parametric bootstrap on page 4 and on page 5 it talks about "The two-sided p-values are now calculated by determining the percentage of situation where the absolute value of the test statistics calculated from the groups with permutated values is at least as large as the absolute value of the test statistics calculated from the original groups.".. and the following page westfall.ba.ttu.edu/Finance%20Pres.ppt talks about something similar on page 11 and page 12 I am not what that means and how that is related to the stata p-value? and the following page westfall.ba.ttu.edu/Finance%20Pres.ppt talks about something similar on page 11 and page 12 I am not what that means and how that is related to the stata p-value? thanks rich 2008/12/5 Carlo Lazzaro <[hidden email]>: > Dear Richard, > I would refer you to Stata Archives and recommend to take a look at Maarten > Buis' and Martin Weiss' kind replies to one thread I have posted on 2008 Sep > 6 (which was not so different from yours, as it would seem): > > st: z statistics in bootstrap output > > Kind Regards and enjoy your W-E, > Carlo > -----Messaggio originale----- > Da: [hidden email] > [mailto:[hidden email]] Per conto di Richard Harvey > Inviato: venerdì 5 dicembre 2008 18.40 > A: [hidden email] > Oggetto: st: bootstrapped p-values > > Hi, > > Could someone please explain how stata computes the bootstrap p-values? > > suppose i issue the following commands > > sysuse auto > bootstrap t=r(t), rep(1000) saving(bsauto, replace): ttest mpg==0 > > i get > > > Bootstrap results Number > of obs= 74 > Replications = 1000 > > > Observed Bootstrap Normal-based > Coef. Std. Err. z P>z [95% Conf. Interval] > > t 31.66644 2.900212 10.92 0.000 25.98213 37.35075 > > > how does stata compute the P>z? > > does it... > > look at the proportion of bootstrapped t-values which are less than or > equal to the original t-value ( from > ttest mpg==0) say x and the proportion of bootstrapped t-values which > are greater than or equal to the original t-value ( from ttest mpg==0) > say y then computes p as 2 min(x,y) ? > > -- > thanks for your time > rich > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > > > -- thanks for your time rich * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/ |
hi,
carlo thanks for your effort... I am still confused by your example.. especially this bit of the code.. generate indicator =abs(r(t))>=abs(scalar(tobs)) sum indicator, mean display "ASLboot="r(mean) i understand what it is doing is strong the original value of t-stats as a scalar and then after the bootrap process creating a count of the number of bootstrapped t-values greater than the original t and taking the mean of that number However, this always turns out to be 1 when i run the code even when i run it on different samples ..so what does that mean??? how is that related to the p-value.. sorry if tis is really dumb but i have got myself into a mental knot with this. thanks rich 2008/12/6 Carlo Lazzaro <[hidden email]>: > Dear Richard, > I suppose that confusion arose because you are implicitly referring to two > different things: > > - the meaning of p-value reported in bootstrap printout (and Nick's > yesterday reply still holds): > > - the bootstrap p-value vs the original one in searching for the > significance of the difference of the mean with skewed data. > As widely reported in literature (please see for instance, Desgagne A, > Castilloux A, Angers J, LeLorier J. The use of the bootstrap statistical > method for the pharmacoeconomic cost analysis of skewed data. > Pharmacoeconomics 1998; 13:487-497.Barber JA and Thompson SG. Analysis of > cost data in randomized trials: an application of the non-parametric > bootstrap. Statistics in Medicine 2000; 19:3219-3236), ttest procedure needs > the fulfilments of some prerequisite before being invoked, which are > systematically violated whenever data are skewed (please, take a look at the > following example sketched via Stata 9.2/SE): > ---------------------------begin example--------------------------- > sysuse auto > save "C:\Documents and > Settings\carlo\Documenti\Statistiche\Stata\Richard_1.dta", replace > ttest mpg, by(foreign) unequal > scalar tobs=r(t) > sum mpg, mean > scalar omean=r(mean) > sum mpg if foreign==0, mean > replace mpg = mpg-r(mean) + scalar(omean) if foreign==0 > sum mpg if foreign==1, mean > replace mpg = mpg-r(mean) + scalar(omean) if foreign==1 > sort foreign > by foreign: sum mpg > keep mpg foreign > set seed 1 > bootstrap r(t), reps(1000) nodots strata(foreign) saving(C:\Documents and > Settings\carlo\Documenti\Statistiche\Stata\Richard_2.dta, every(1) > replace)verbose : ttest mpg, by(foreign) unequal > save "C:\Documents and > Settings\carlo\Documenti\Statistiche\Stata\Richard_1.dta", replace > use "C:\Documents and > Settings\carlo\Documenti\Statistiche\Stata\Richard_2.dta", clear > generate indicator =abs(r(t))>=abs(scalar(tobs)) > sum indicator, mean > display "ASLboot="r(mean) > ------------------------------end example---------------------------- > > HTH, Kind Regards and enjoy your W-E, > > Carlo > -----Messaggio originale----- > Da: Richard Harvey [mailto:[hidden email]] > Inviato: venerdì 5 dicembre 2008 23.37 > A: Carlo Lazzaro > Oggetto: Re: R: bootstrapped p-values > > hi, > > thanks nick and carlo.. > > I understand your point about the p value in the statout now. I am a > bit confused about some things i have read elsewhere > > for example the following > > http://compdiag.molgen.mpg.de/ngfn/docs/2003/nov/Exercises_Bootstrap.pdf > > discusses non parametric bootstrap on page 4 and on page 5 it talks about > > "The two-sided p-values are now calculated by determining the > percentage of situation where the > absolute value of the test statistics calculated from the groups with > permutated values is at least > as large as the absolute value of the test statistics calculated from > the original groups.".. > > and the following page > > westfall.ba.ttu.edu/Finance%20Pres.ppt > > talks about something similar on page 11 and page 12 > > I am not what that means and how that is related to the stata p-value? > and the following page > > westfall.ba.ttu.edu/Finance%20Pres.ppt > > talks about something similar on page 11 and page 12 > > I am not what that means and how that is related to the stata p-value? > > > thanks > rich > > 2008/12/5 Carlo Lazzaro <[hidden email]>: >> Dear Richard, >> I would refer you to Stata Archives and recommend to take a look at > Maarten >> Buis' and Martin Weiss' kind replies to one thread I have posted on 2008 > Sep >> 6 (which was not so different from yours, as it would seem): >> >> st: z statistics in bootstrap output >> >> Kind Regards and enjoy your W-E, >> Carlo >> -----Messaggio originale----- >> Da: [hidden email] >> [mailto:[hidden email]] Per conto di Richard Harvey >> Inviato: venerdì 5 dicembre 2008 18.40 >> A: [hidden email] >> Oggetto: st: bootstrapped p-values >> >> Hi, >> >> Could someone please explain how stata computes the bootstrap p-values? >> >> suppose i issue the following commands >> >> sysuse auto >> bootstrap t=r(t), rep(1000) saving(bsauto, replace): ttest mpg==0 >> >> i get >> >> >> Bootstrap results Number >> of obs= 74 >> Replications = 1000 >> >> >> Observed Bootstrap Normal-based >> Coef. Std. Err. z P>z [95% Conf. Interval] >> >> t 31.66644 2.900212 10.92 0.000 25.98213 37.35075 >> >> >> how does stata compute the P>z? >> >> does it... >> >> look at the proportion of bootstrapped t-values which are less than or >> equal to the original t-value ( from >> ttest mpg==0) say x and the proportion of bootstrapped t-values which >> are greater than or equal to the original t-value ( from ttest mpg==0) >> say y then computes p as 2 min(x,y) ? >> >> -- >> thanks for your time >> rich >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ >> >> >> > > > > -- > thanks for your time > rich > > > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > -- thanks for your time rich * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/ |
In reply to this post by Carlo Lazzaro
Dear Richard,
your concern is conceivable, since to my real shame I have misreported two bits of the code. Sorry for this. I do hope that everything will work fine this time. Kind Regards and have a Nice Sunday, Carlo ---------------------------begin example--------------------------- sysuse auto save "C:\Documents and Settings\carlo\Documenti\Statistiche\Stata\Richard_1.dta", replace ttest mpg, by(foreign) unequal scalar tobs=r(t) sum mpg, mean scalar omean=r(mean) sum mpg if foreign==0, mean replace mpg = mpg-r(mean) + scalar(omean) if foreign==0 sum mpg if foreign==1, mean replace mpg = mpg-r(mean) + scalar(omean) if foreign==1 sort foreign by foreign: sum mpg keep mpg foreign set seed 1 bootstrap t=r(t), reps(1000) nodots strata(foreign) saving(C:\Documents and Settings\carlo\Documenti\Statistiche\Stata\Richard_2.dta, every(1) replace)verbose : ttest mpg, by(foreign) unequal save "C:\Documents and Settings\carlo\Documenti\Statistiche\Stata\Richard_1.dta", replace use "C:\Documents and Settings\carlo\Documenti\Statistiche\Stata\Richard_2.dta", clear generate indicator =abs(t)>=abs(scalar(tobs)) sum indicator, mean display "ASLboot="r(mean) ------------------------------end example---------------------------- * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/ |
Free forum by Nabble | Edit this page |