Re:Re: st: GLLAMM multinomial: tremendous instability

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re:Re: st: GLLAMM multinomial: tremendous instability

K Konstantaras
Dear statalisters and personally Stas, Maarten and J. Verkuilen:

Apologies for my late reply to your helpful suggestions -due to a strong (non PC) virus.

The problem has been resolved by Sophia Rabe-Hesqueth a few days ago; it has been the result of reversing the order of the expand() option variables, namely patt and chosen. With the correct order, instability has stopped.

As a matter of fact, the random effects correlation equal to 1 problem has not ceased to exist and I would be much obliged if you could post me any references you might have come accross on this.

Much obliged,

Dino Konstantaras

....................................................................................................................................
On 11/24/08, Stas Kolenikov [[hidden email]] wrote:
Dino,

Give your -mlogit- results, too; did you feed them as starting values to -gllamm-? Keep in mind that introducing random effects changes the scaling of your coefficients; see http://www.citeulike.org/user/ctacmo/article/3057661.

I'd fully agree with earlier reactions that your model does not appear to be identified well. Correlation of 1 are troublesome; in your particular case this may mean that there is no difference between two of the three alternatives -- is this making sense? If one of the categories is quite rare, or present in some weird patterns wrt units producing random  effects, it may create those numeric problems, too.

On 11/24/08, KONSTANTARAS KONSTANTINOS <[hidden email]> wrote:

> Dear statalisters,
>
>  In my model runs (using Intercooled Stata 9.2) I experience
> tremendous instability in gllamm results, higher than has been
> elsewhere reported at
> statalist: I run a simple multinomial logit model with random effects,
> using the results as initial matrix for exactly the same model, to get
> as a result an extremely different log likelihood and coefficients
> etc. The resulting loglikelihood differs by large margins (3-9%). I
> have also tried to use exactly the same initial values from mlogit but
> not only does the overall loglikelihood continues to be unstable but
> also the same happens with the loglikelihood obtained without random
> effects –running gllamm with init option-. Nevertheless the model
> invariably converges, albeit to an entirely different estimate.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

RE: Re: st: GLLAMM multinomial: tremendous instability

Verkuilen, Jay
>As a matter of fact, the random effects correlation equal to 1 problem has not ceased to exist and I would be much obliged if you could post me any references you might have come accross on this.<

I would guess that you don't have an identified model. I'm not sure if it's mathematically identified or not or simply empirically, i.e., your data don't really tell you anything about the parameter. It is very easy to write a plausible model that is only weakly identified---I've done it many times and I'm sure everyone else has.

Alternatively, you could have a random effect term that really "wants" to be 0 and thus has a profile likelihood piled up on 0. This can often cause odd things to happen in the random effects covariance matrix if it's been constrained to be positive semi-definite. Estimation breaks down in the presence of really small random effects.

Start deleting random effects and see if it goes away. An alternative (and maybe better) strategy would be to start from a model you know is identified, e.g., the multinomial choice model with no random effects, and add RE terms in an order determined by theory.

JV


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

RE: Re: st: GLLAMM multinomial: tremendous instability

K Konstantaras
In reply to this post by K Konstantaras
Thanks a lot for your advice.

It seems that I do fall into the category you have described of a very small random effect for one category, because estimating the model with only this one random effect does not produce LR significant results for it or, in some other specifications, estimation breaks down completely.

The other random effect does not have this problem and is significant from an LR test standpoint.

I believe I can live with one RE not existent, but I would like to ask you once again if there is any reference on the matter in order to figure out whether I can indeed draw any safe conclusions from this model, assuming only one significant random effect. The theoretical explanation for this might be that individuals choosing to belong in the non-existent random effects category do not have a sufficiently dense -deep- panel time dimension, hence failing to generate sufficient depth of time-independent observations to estimate their unobserved heterogeneity through time (they behave more like a single cross section and not a panel).  

Thanking you in advance,

Dino K.
................................................................
Verkuilen, Jay has sent on: Wednesday, December 03, 2008 9:36 PM
>Alternatively, you could have a random effect term that really "wants"
>to
be 0 and thus has a profile likelihood piled up on 0. This can often cause odd things to happen in the random effects covariance matrix if it's been constrained to be positive semi-definite. Estimation breaks down in the presence of really small random effects.

Start deleting random effects and see if it goes away. An alternative (and maybe better) strategy would be to start from a model you know is identified, e.g., the multinomial choice model with no random effects, and add RE terms in an order determined by theory.
-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

RE: Re: st: GLLAMM multinomial: tremendous instability

Verkuilen, Jay
In reply to this post by K Konstantaras
I am not sure I have any citations that would be persuasive for an econometric audience, but you might look thru Kenneth Train's book (Discrete Choice Methods with Simulation, Cambridge, 2003). Random variation is not guaranteed---in a sense if it *doesn't* happen that's good because it says the world is simpler.


-----Original Message-----
From: "KONSTANTARAS KONSTANTINOS" <[hidden email]>
To: [hidden email]
Sent: 12/4/2008 11:21 AM
Subject: RE: Re: st: GLLAMM multinomial: tremendous instability

Thanks a lot for your advice.

It seems that I do fall into the category you have described of a very small random effect for one category, because estimating the model with only this one random effect does not produce LR significant results for it or, in some other specifications, estimation breaks down completely.

The other random effect does not have this problem and is significant from an LR test standpoint.

I believe I can live with one RE not existent, but I would like to ask you once again if there is any reference on the matter in order to figure out whether I can indeed draw any safe conclusions from this model, assuming only one significant random effect. The theoretical explanation for this might be that individuals choosing to belong in the non-existent random effects category do not have a sufficiently dense -deep- panel time dimension, hence failing to generate sufficient depth of time-independent observations to estimate their unobserved heterogeneity through time (they behave more like a single cross section and not a panel).  

Thanking you in advance,

Dino K.
................................................................
Verkuilen, Jay has sent on: Wednesday, December 03, 2008 9:36 PM
>Alternatively, you could have a random effect term that really "wants"
>to
be 0 and thus has a profile likelihood piled up on 0. This can often cause odd things to happen in the random effects covariance matrix if it's been constrained to be positive semi-definite. Estimation breaks down in the presence of really small random effects.

Start deleting random effects and see if it goes away. An alternative (and maybe better) strategy would be to start from a model you know is identified, e.g., the multinomial choice model with no random effects, and add RE terms in an order determined by theory.
-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/