st: significance test between two kendalls' tau coefficents ?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

st: significance test between two kendalls' tau coefficents ?

Marcus Fischer
I would like to compare two Kenadalls' tau coefficients. Is it possible to test whether one coefficient is significantly higher than the other one ?
Thanks.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

Re: st: significance test between two kendalls' tau coefficents ?

JOHN ANTONAKIS
Why don't you simply construct a 95% confidence interval around the tau
estimate and check if estimates overlap? STATA reports the ASE for tau
with the tab y x, all command.

HTH,
John Antonakis


On 04.12.2008 12:34, Marcus Fischer wrote:
 > I would like to compare two Kenadalls' tau coefficients. Is it
possible to test whether one coefficient is significantly higher than
the other one ?
 > Thanks.
 >
 >
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

Re: st: significance test between two kendalls' tau coefficents ?

Steven Samuels-2
-somersd- by Roger Newson can compute the difference between two  
Kendalls Tau coefficients. It is downloadable from SSC.

On Dec 4, 2008, at 8:39 AM, JOHN ANTONAKIS wrote:

> Why don't you simply construct a 95% confidence interval around the  
> tau estimate and check if estimates overlap? STATA reports the ASE  
> for tau with the tab y x, all command.
>
> HTH,
> John Antonakis
>
>
> On 04.12.2008 12:34, Marcus Fischer wrote:
> > I would like to compare two Kenadalls' tau coefficients. Is it  
> possible to test whether one coefficient is significantly higher  
> than the other one ?
> > Thanks.
> >
> >
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

st: RE: significance test between two kendalls' tau coefficents ?

Roger Newson
In reply to this post by Marcus Fischer
Yes, it is possible. However, how to do it depends on whether the
Kendall's tau coefficients are from the same sample, or from different
samples.

In Stata, confidence intervals for Kendall's tau-a statistics and their
differences can be calculated using the -somersd- package, downloadable
from SSC using the -ssc- command. This package is described in Newson
(2002), Newson (2006a) and Newson (2006b), and calculates confidence
intervals for a wide range of rank statistics, including some with
sample weighting, clustering and/or stratification.

Often, given a single outcome variable Y and 2 positive predictors X and
W, we want to know which of the 2 positive predictors predicts Y better.
This is because, as proved in Newson (2006a) and Newson (2002), if a
positive association of Y with X is caused by a positive association of
both variables with W, then we must have the inequalities

tau(Y,X) <= tau (W,X)
tau(X,Y) >= tau(W,Y)

where tau(.,.) is Kendall's tau-a. Typically, in Stata, we type

somersd Y X W, taua tdist
lincom X-W

The -somersd- command, with the -taua- option, calculates confidence
intervals for the Kendall's tau-a parameters tau(Y,X) and tau(Y,W). The
-lincom- command calculates a confidence interval for the difference
tau(Y,X)-tau(Y,W).

Sometimes, by contrast, we may want to estimate the difference between 2
tau(Y,X) statistics in different subpopulations, from which we have
taken 2 different subsamples. In this case, we can use the -somersd-
package with the -parmest- package, also downloadable from SSC. The
-parmest- package creates and processes output datasets (or resultssets)
with 1 observation per estimated parameter and data on estimates,
standard errors, confidence limits and P-values, and is described in
Newson (2003), Newson (2004), and Newson(2008). If the variable -subpop-
indicates which of 2 subpopulations an observation belongs to, then we
may type in Stata

parmby "somersd Y X, taua tdist", by(subpop) norestore
list
metaparm [iwei=(subpop==2)-(subpop==1)] if parm=="X", list(,)

The -parmby- command creates, in the memory, an output dataset (or
resultsset), with 1 observation per fitted tau-a parameter and data on
their estimates, standard errors, confidence limits, and P-values. (This
resultsset overwrites the original dataset.) The tau-a parameters, are
tau(Y,Y) in subpopulation 1, tau(Y,X) in subpopulation 1, tau(Y,Y) in
subpopulation 2, and tau(Y,X) in subpopulation 2. We want to compare the
2 tau(Y,X) parameters. (The tau(Y,Y) parameters are simply probabilities
that 2 randomly-chosen Y-values are unequal.) The list command lists all
the estimated parameters. The metaparm command calculates and lists a
confidence interval for the difference between the tau(Y,X) parameter in
subpopulation 2 and the tau(Y,X) parameter in subpopulation 1.

I hope this helps. Let me know if you have any queries.

Best wishes

Roger


References

Newson R. B. parmest and extensions. Presented at the 14th UK Stata User
Meeting, 8-9 September, 2008. Download presentation from
http://www.imperial.ac.uk/nhli/r.newson/usergp.htm

Newson R. 2006b. Confidence intervals for rank statistics: Percentile
slopes, differences, and ratios. The Stata Journal 6(4): 497-520. Buy
the article from
http://www.stata-journal.com/archives.html
or download a pre-publication draft from
http://www.imperial.ac.uk/nhli/r.newson/papers.htm

Newson R. 2006a. Confidence intervals for rank statistics: Somers' D and
extensions. The Stata Journal 6(3): 309-334. Buy the article from
http://www.stata-journal.com/archives.html
or download a pre-publication draft from
http://www.imperial.ac.uk/nhli/r.newson/papers.htm

Newson R. From datasets to resultssets in Stata. Presented at the 10th
UK Stata User Meeting, 28-29 June, 2004. Download presentation and
handouts from
http://www.imperial.ac.uk/nhli/r.newson/usergp.htm

Newson R. 2003. Confidence intervals and p-values for delivery to the
end user. The Stata Journal 3(3): 245-269. Download the article from
http://www.stata-journal.com/archives.html
or download a pre-publication draft from
http://www.imperial.ac.uk/nhli/r.newson/papers.htm

Newson R. 2002. Parameters behind "nonparametric" statistics: Kendall's
tau, Somers' D and median differences. The Stata Journal 2(1): 45-64.
Download the article from
http://www.stata-journal.com/archives.html


Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: [hidden email]
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/pop
genetics/reph/

Opinions expressed are those of the author, not of the institution.

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Marcus
Fischer
Sent: 04 December 2008 11:35
To: [hidden email]
Subject: st: significance test between two kendalls' tau coefficents ?

I would like to compare two Kenadalls' tau coefficients. Is it possible
to test whether one coefficient is significantly higher than the other
one ?
Thanks.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

Re: st: significance test between two kendalls' tau coefficents ?

Ronan Conroy
In reply to this post by JOHN ANTONAKIS
On 4 Dec 2008, at 13:39, JOHN ANTONAKIS wrote:

> Why don't you simply construct a 95% confidence interval around the  
> tau
> estimate and check if estimates overlap? STATA reports the ASE for tau
> with the tab y x, all command.

Warning: overlapping confidence intervals do not necessarily mean that  
the two are not significantly different.

You need the confidence interval for the difference.


Ronan Conroy
=================================

[hidden email]
Royal College of Surgeons in Ireland
Epidemiology Department,
Beaux Lane House, Dublin 2, Ireland
+353 (0)1 402 2431
+353 (0)87 799 97 95
+353 (0)1 402 2764 (Fax - remember them?)
http://rcsi.academia.edu/RonanConroy

P    Before printing, think about the environment




*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

Re: st: significance test between two kendalls' tau coefficents ?

Mike Lacy
In reply to this post by Marcus Fischer

A
 >Date: Thu, 4 Dec 2008 16:13:01 +0000
 >From: Ronan Conroy <[hidden email]>
 >Subject: Re: st: significance test between two kendalls' tau coefficents ?
 >
 >On 4 Dec 2008, at 13:39, JOHN ANTONAKIS wrote:
 >
 >> Why don't you simply construct a 95% confidence interval around the
 >> tau
 >> estimate and check if estimates overlap? STATA reports the ASE for tau
 >> with the tab y x, all command.
 >
 >Warning: overlapping confidence intervals do not necessarily mean that
 >the two are not significantly different.
 >
 >You need the confidence interval for the difference.

Moreover, if you truly want a test as opposed to a CI, it is
worth keeping in mind that the SE of the sampling distribution,
and in fact all aspects of the sampling distribution,  is in general
not the same under the null (test situation) as it is under the
alternative (CI situation.)I have seen cases with measures like
Tau in which the two sampling distributions differed radically.
An instructive exercise is to compare the distribution one gets
from -bootstrap- (distribution under the alternative) vs. -permute-
(under the null)

Regards,
Mike Lacy
Fort Collins CO


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

RE: st: significance test between two kendalls' tau coefficents ?

Roger Newson
Mike has raised an important issue here, because a "significant" difference between 2 Kendall's taus seems to imply that at least one of them is non-zero. A possible response is to use the hyperbolic arctangent transformation (also known as Fisher's z transformation), recommended by Edwardes (1995), as a Normalizing and variance-stabilizing transformation. This transformation is available in Stata by using the -somersd- package with the -transf(z)- option. If this is used, then -lincom- and -metaparm- will give confidence intervals for the difference between the z-transformed Kendall's taus, instead of confidence intervals for the difference between the original Kendall's taus. However, both differences have the same sign. It is also possible to use an arcsine transformation, as suggested by Daniels and Kendall (1947).

I hope this helps.

Best wishes

Roger


References

Daniels, H. E., and M. G. Kendall. 1947. The significance of rank correlation where parental correlation exists. Biometrika 34: 197–208.

Edwardes, M. D. d. B. 1995. A confidence interval for Pr(X < Y) − Pr(X > Y) estimated from simple cluster samples. Biometrics 51: 571–578.


Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: [hidden email]
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/

Opinions expressed are those of the author, not of the institution.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Mike Lacy
Sent: 05 December 2008 18:30
To: [hidden email]
Subject: Re: st: significance test between two kendalls' tau coefficents ?


A
 >Date: Thu, 4 Dec 2008 16:13:01 +0000
 >From: Ronan Conroy <[hidden email]>
 >Subject: Re: st: significance test between two kendalls' tau coefficents ?
 >
 >On 4 Dec 2008, at 13:39, JOHN ANTONAKIS wrote:
 >
 >> Why don't you simply construct a 95% confidence interval around the
 >> tau
 >> estimate and check if estimates overlap? STATA reports the ASE for tau
 >> with the tab y x, all command.
 >
 >Warning: overlapping confidence intervals do not necessarily mean that
 >the two are not significantly different.
 >
 >You need the confidence interval for the difference.

Moreover, if you truly want a test as opposed to a CI, it is
worth keeping in mind that the SE of the sampling distribution,
and in fact all aspects of the sampling distribution,  is in general
not the same under the null (test situation) as it is under the
alternative (CI situation.)I have seen cases with measures like
Tau in which the two sampling distributions differed radically.
An instructive exercise is to compare the distribution one gets
from -bootstrap- (distribution under the alternative) vs. -permute-
(under the null)

Regards,
Mike Lacy
Fort Collins CO


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

st: RE: RE: significance test between two kendalls' tau coefficents ?

Feiveson, Alan H. (JSC-SK311)
In reply to this post by Roger Newson
Roger Newson wrote:

-----------------------------------------------------------------------
Sometimes, by contrast, we may want to estimate the difference between 2
tau(Y,X) statistics in different subpopulations, from which we have
taken 2 different subsamples. In this case, we can use the -somersd-
package with the -parmest- package, also downloadable from SSC. The
-parmest- package creates and processes output datasets (or resultssets)
with 1 observation per estimated parameter and data on estimates,
standard errors, confidence limits and P-values, and is described in
Newson (2003), Newson (2004), and Newson(2008). If the variable -subpop-
indicates which of 2 subpopulations an observation belongs to, then we
may type in Stata
-----------------------------------------------------------------------

But what about the case of clustering? In this case, there may be two
distinct subsets of data on which tau(X, Y) is estimated, but there is a
common pool of subjects. So, for example, Subject 1 contributed to tau(X
,Y) on both subsets. The within-data comparison of tau's in -somersd-
won't work, because it only applies to common observations in the data
set. The between-sets comparison is invalid as it ignores the clustering
due to subjects.

Al Feiveson

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

st: RE: RE: RE: significance test between two kendalls' tau coefficents ?

Roger Newson
Al states, correctly, that -somersd- does not yet calculate confidence
limits for the difference between 2 within-stratum tau(X,Y) parameters
specific to 2 strata that may both appear in the same cluster. To make
this possible, I would probably have to upgrade -somersd- to create an
output dataset (or resultsset), with 1 observation per cluster and data
on the delta-jackknife influence functions or pseudovalues of that
cluster for the various tau-a or Somers' D statistics. If we created 2
such resultssets, one for each stratum, then we could merge the 2
resultssets using the cluster() variable as the key, and then use the
-mean- command to estimate a covariance matrix for the tau-a or Somers'
D statistics, and then use -lincom- to estimate the difference. Such a
method would be a rank-methods analog of -suest-.

Best wishes

Roger


Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: [hidden email]
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/pop
genetics/reph/

Opinions expressed are those of the author, not of the institution.

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Feiveson,
Alan H. (JSC-SK311)
Sent: 08 December 2008 18:45
To: [hidden email]
Subject: st: RE: RE: significance test between two kendalls' tau
coefficents ?

Roger Newson wrote:

-----------------------------------------------------------------------
Sometimes, by contrast, we may want to estimate the difference between 2
tau(Y,X) statistics in different subpopulations, from which we have
taken 2 different subsamples. In this case, we can use the -somersd-
package with the -parmest- package, also downloadable from SSC. The
-parmest- package creates and processes output datasets (or resultssets)
with 1 observation per estimated parameter and data on estimates,
standard errors, confidence limits and P-values, and is described in
Newson (2003), Newson (2004), and Newson(2008). If the variable -subpop-
indicates which of 2 subpopulations an observation belongs to, then we
may type in Stata
-----------------------------------------------------------------------

But what about the case of clustering? In this case, there may be two
distinct subsets of data on which tau(X, Y) is estimated, but there is a
common pool of subjects. So, for example, Subject 1 contributed to tau(X
,Y) on both subsets. The within-data comparison of tau's in -somersd-
won't work, because it only applies to common observations in the data
set. The between-sets comparison is invalid as it ignores the clustering
due to subjects.

Al Feiveson

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/