more questions on multicollinearity

Discussion:

(too old to reply)

sriram Narayanan

2006-01-21 22:30:45 UTC

Hi All:

I have been seeing the archives for a discussion of multicollinearity.
I think I want to divide this into two parts.

1. Diagnose if there is a multicollinearity problem
2. Address the issue if multicollinearity is an issue.

I am dividing this because I want to bring some clarity into my head
about this issue, had some questions on them. I probably may be saying
something very silly but I just want to learn more about this problem
because I am facing such issues with my model that I have been
thinking. My model contains an X term and an X^2 term. (.97
correlation)

On diagnosis: There are two ways I know till now:

Method 1) Some earlier messages (I cant trace this message again
except I read it otherwise I would acknowledge the source) seemed to
suggest that one should look at the level of correlation between the
estimators. In lisrel I think PC option gives this. Very high
estimator correlations indicate that there is a problem with
multicollinearity. For example my model with AMOS converges with a
very bad fit, and I have problems with correlations between the
estimates involving the paths from x to the latent variable and x^2 to
the latent variable. However my model in LISREL converges fine.

Method 2) If the completely standardized coefficient is greater than
1 it can indicate multicollinearity as in one of Prof Joreskog's
article.
http://www.ssicentral.com/lisrel/techdocs/HowLargeCanaStandardizedCoefficientbe.pdf

Even though the correlations are high my completely standardized
correlations are less than 1.

Questions:

What happens if one has a completely standardized coefficient of <1
but high correlations between the estimates. ?

Having a completely standardized coefficient of grater than 1 could be
a strong indicator of multicollinearity but can one also say that
having a less than 1 guarantees that there is no multicollinearity?

What is the theory behind the reasons for high estimator correlations
leading to multicollinearity. I will appreciate if there are any
references.

In my model diagnostic 2 seems fine even if the two indicators are
highly correlated in my case (the correlation between them is .97).
however the estimates are strongly correlated *,95 upwards)

Would I think that the collinearity problem is not serious?

On correcting multicollinearity:

Ed's suggested that one can do centering as has been suggested by
authors like Chronbach (1987, Psychological Bulletin) in this
listserv.

I have tried that strategy and it seemed to me that somehow in my data
it does not work I have a .96 correlation before centering and a .76
after centering. It does go down but I am not sure if this is low
enough. When I use this method diagnostic 1 still has some problems
while disgnstic 2 is fine in my model.

The other way is standardization that I found in an article by

Dunlap, William P. AND Edward R. Kemery (1988), "Effects of Predictor ...
INTERACTIONS AND Moderator Effects," Psychological Bulletin, 114 (2), 376-390

I have done log transformations to get my variable to be normal. If I
mean center this would it not affect my interpretation ?

Some clarity on this will be highly welcome. I will be very grateful.

Thanks

Sriram

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

Bert Weijters

2006-01-23 09:37:58 UTC

Permalink

As for mean centering, I found this working paper by Gatignon and
Vosgerau that dismisses the usefulness of this approach (in multiple
regression) under the title "The myth of mean centering".
http://flora.insead.edu/insead/jsp/system/win_main.jsp

Regards,
Bert

Bert Weijters - PhD Candidate ICM
Vlerick Leuven Gent Management School
Reep 1, B9000 Gent, Belgium
tel: +32 9 210 98 76
mailto: ***@vlerick.be

-----Original Message-----
From: Structural Equation Modeling Discussion Group
[mailto:***@BAMA.UA.EDU] On Behalf Of sriram Narayanan
Sent: Saturday, January 21, 2006 11:31 PM
To: ***@BAMA.UA.EDU
Subject: more questions on multicollinearity

Hi All:

I have been seeing the archives for a discussion of multicollinearity.
I think I want to divide this into two parts.

1. Diagnose if there is a multicollinearity problem
2. Address the issue if multicollinearity is an issue.

I am dividing this because I want to bring some clarity into my head
about this issue, had some questions on them. I probably may be saying
something very silly but I just want to learn more about this problem
because I am facing such issues with my model that I have been
thinking. My model contains an X term and an X^2 term. (.97
correlation)

On diagnosis: There are two ways I know till now:

Method 1) Some earlier messages (I cant trace this message again
except I read it otherwise I would acknowledge the source) seemed to
suggest that one should look at the level of correlation between the
estimators. In lisrel I think PC option gives this. Very high
estimator correlations indicate that there is a problem with
multicollinearity. For example my model with AMOS converges with a
very bad fit, and I have problems with correlations between the
estimates involving the paths from x to the latent variable and x^2 to
the latent variable. However my model in LISREL converges fine.

Method 2) If the completely standardized coefficient is greater than
1 it can indicate multicollinearity as in one of Prof Joreskog's
article.
http://www.ssicentral.com/lisrel/techdocs/HowLargeCanaStandardizedCoeffi
cientbe.pdf

Even though the correlations are high my completely standardized
correlations are less than 1.

Questions:

What happens if one has a completely standardized coefficient of <1
but high correlations between the estimates. ?

Having a completely standardized coefficient of grater than 1 could be
a strong indicator of multicollinearity but can one also say that
having a less than 1 guarantees that there is no multicollinearity?

What is the theory behind the reasons for high estimator correlations
leading to multicollinearity. I will appreciate if there are any
references.

In my model diagnostic 2 seems fine even if the two indicators are
highly correlated in my case (the correlation between them is .97).
however the estimates are strongly correlated *,95 upwards)

Would I think that the collinearity problem is not serious?

On correcting multicollinearity:

Ed's suggested that one can do centering as has been suggested by
authors like Chronbach (1987, Psychological Bulletin) in this
listserv.

I have tried that strategy and it seemed to me that somehow in my data
it does not work I have a .96 correlation before centering and a .76
after centering. It does go down but I am not sure if this is low
enough. When I use this method diagnostic 1 still has some problems
while disgnstic 2 is fine in my model.

The other way is standardization that I found in an article by

Dunlap, William P. AND Edward R. Kemery (1988), "Effects of Predictor
...
INTERACTIONS AND Moderator Effects," Psychological Bulletin, 114 (2),
376-390

I have done log transformations to get my variable to be normal. If I
mean center this would it not affect my interpretation ?

Some clarity on this will be highly welcome. I will be very grateful.

Thanks

Sriram

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

Bert Weijters

2006-01-23 10:46:49 UTC

Permalink

Correction: the paper I referred to is accessible via
http://www.insead.edu/facultyresearch/research/output_search.htm
You have to log in as a guest and do a search (using the info below).
Bert

-----Original Message-----
From: Structural Equation Modeling Discussion Group
[mailto:***@BAMA.UA.EDU] On Behalf Of Bert Weijters
Sent: Monday, January 23, 2006 10:37 AM
To: ***@BAMA.UA.EDU
Subject: Re: more questions on multicollinearity

As for mean centering, I found this working paper by Gatignon and
Vosgerau that dismisses the usefulness of this approach (in multiple
regression) under the title "The myth of mean centering".
http://flora.insead.edu/insead/jsp/system/win_main.jsp

Regards,
Bert

Bert Weijters - PhD Candidate ICM
Vlerick Leuven Gent Management School
Reep 1, B9000 Gent, Belgium
tel: +32 9 210 98 76
mailto: ***@vlerick.be

-----Original Message-----
From: Structural Equation Modeling Discussion Group
[mailto:***@BAMA.UA.EDU] On Behalf Of sriram Narayanan
Sent: Saturday, January 21, 2006 11:31 PM
To: ***@BAMA.UA.EDU
Subject: more questions on multicollinearity

Hi All:

I have been seeing the archives for a discussion of multicollinearity.
I think I want to divide this into two parts.

1. Diagnose if there is a multicollinearity problem
2. Address the issue if multicollinearity is an issue.

I am dividing this because I want to bring some clarity into my head
about this issue, had some questions on them. I probably may be saying
something very silly but I just want to learn more about this problem
because I am facing such issues with my model that I have been
thinking. My model contains an X term and an X^2 term. (.97
correlation)

On diagnosis: There are two ways I know till now:

Method 1) Some earlier messages (I cant trace this message again
except I read it otherwise I would acknowledge the source) seemed to
suggest that one should look at the level of correlation between the
estimators. In lisrel I think PC option gives this. Very high
estimator correlations indicate that there is a problem with
multicollinearity. For example my model with AMOS converges with a
very bad fit, and I have problems with correlations between the
estimates involving the paths from x to the latent variable and x^2 to
the latent variable. However my model in LISREL converges fine.

Method 2) If the completely standardized coefficient is greater than
1 it can indicate multicollinearity as in one of Prof Joreskog's
article.
http://www.ssicentral.com/lisrel/techdocs/HowLargeCanaStandardizedCoeffi
cientbe.pdf

Even though the correlations are high my completely standardized
correlations are less than 1.

Questions:

What happens if one has a completely standardized coefficient of <1
but high correlations between the estimates. ?

Having a completely standardized coefficient of grater than 1 could be
a strong indicator of multicollinearity but can one also say that
having a less than 1 guarantees that there is no multicollinearity?

What is the theory behind the reasons for high estimator correlations
leading to multicollinearity. I will appreciate if there are any
references.

In my model diagnostic 2 seems fine even if the two indicators are
highly correlated in my case (the correlation between them is .97).
however the estimates are strongly correlated *,95 upwards)

Would I think that the collinearity problem is not serious?

On correcting multicollinearity:

Ed's suggested that one can do centering as has been suggested by
authors like Chronbach (1987, Psychological Bulletin) in this
listserv.

I have tried that strategy and it seemed to me that somehow in my data
it does not work I have a .96 correlation before centering and a .76
after centering. It does go down but I am not sure if this is low
enough. When I use this method diagnostic 1 still has some problems
while disgnstic 2 is fine in my model.

The other way is standardization that I found in an article by

Dunlap, William P. AND Edward R. Kemery (1988), "Effects of Predictor
...
INTERACTIONS AND Moderator Effects," Psychological Bulletin, 114 (2),
376-390

I have done log transformations to get my variable to be normal. If I
mean center this would it not affect my interpretation ?

Some clarity on this will be highly welcome. I will be very grateful.

Thanks

Sriram

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

Stas Kolenikov

2006-01-23 17:19:01 UTC

Permalink

There are two separate issues here.

(i) Multicollinearity, at least in regression analysis, means that in
your data set, there are variables with high correlations, and due to
(X'X)^{-1} character of the variance estimator in linear regression,
those high correlations lead to big standard errors on (some) linear
combinations of the estimated coefficients, and big standard errors on
the variables themselves. There is a number of diagnostic methods
outlined in say a quarter century old Belsley, Kuh and Welsch
regression book, and in some other books; the most advanced methods,
to my own liking, are based on singular value decomposition (or PCA of
the regressor matrix, if you like), as they pinpoint the naughty
variables and quantify their contribution to multicollinearity.

(ii) In more complex models, and SEMs are certainly in this group, you
can have high correlation between your estimates for thousands of
other reasons. Some of them may be intrinsic and pretty much
incurable, like you always are going to have correlations between say
variance of a latent factor and variances of its indicators: given
factor loadings, a greater factor variance means there is less
variance left to be explained by error terms, so I would expect this
correlation to be negative. Some other kinds of estimate correlations
can be somewhat fixed by reparameterization of the model, and
centering is one of such tricks. It comes from the aforementioned
linear regression theory, where centering means you are breaking the
correlation between your variable and the constant term. Likewise, you
can make your x^2 term orthogonal to both the constant term and the
linear term {insted of x^2, you use x^2+ax+b, where you choose a and b
in such a way that sum_i (x_i^2+ax_i+b)=0 and sum_i
(x_i^2+ax_i+b)x_i=0}, and may improve correlations of the parameter
estimates. It is very likely that high correlations of the original
varialbes will give rise to the high correlations of the estimates,
but that needs to be checked for every particular case. Just looking
at the correlation matrix of the original variables may not give you
the best picture, especially given that some of those correlations are
going to be explained by the structural model itself.

Now, what are the consequences of multicollinearity, and is it really
that bad? I personally view those as somewhat inavoidable thing, at
least after you did your work on centering and such, and hence I would
rather sigh and say, "OK, this is how much multicollinearity I have".
In regression context, there are tons of shrinkage estimators that
admit some bias, but reduce the mean squared error of your estimates;
ridge regression, PCA regression, Stein, lasso and other methods come
under this rubric. Again, to my liking, this is the best you can go
about multicollinearity in the regression world. I don't think those
methods are easily transferrable to the SEMs, although say ridge
regression has a Bayesian justification, so if you are willing to dive
into the Bayesian world, then you can play around with that.

This brings me to another important consequence of high parameter
correlations (that may be due to multicollinearity in the variables,
or to other reasons). It is an irregularity of the likelihood surface
(or whatever your objective function is) near the maximum: it may have
a nice maximum in some directions, but it is almost flat in others. It
is terrible for numeric reasons. Any numeric optimization book will
tell you exactly why; basically, the iterative maximization procedures
are making steps in somewhat wrong directions even when you are close
to the true maximum. It is explicit in the methods that use the second
derivatives, and need to invert the Hessian matrix that is close to
singular (and hence the inverse may go south); but it is a complex
numeric problem, anyway. LISREL may have a better maximization
algorith implemented than AMOS; it may not even be documented very
well in the software manuals, so if you are really curious, you should
ask the developers. If your correlations are getting so high that the
condition numbers (the ratio of the max to min eigenvalues of the
information matrix/Hessian) are getting above 10^8, then it means that
you cannot find a maximum in single precision; if they are getting
higher than 10^15, you cannot really find a maximum in double
precision, and this is what most of the software uses as the standard
precisioin of the core engine, I believe. A serious maximizer routine
should give you condition numbers at the max, or you can compute them
yourself by doing the eigenproblem on the estimated
variance-covariance matrix of the estimates.

Note that going Bayesian does not save you from this curse: high
parameter correlations in MCMC world are getting reflected in slow
mixing, so you need to lengthen your MCMCs to get reasonable coverage
of the whole parameter space. Bayesians have their reparameterization
tricks for such purposes, too.

Hope that adds to your confusion :)))

Oh, and one more thing: I don't think the standardized coefficients
greater than one are very good indicators of multicollinearity. They
are necessarily accompanied by negative error variances, and that may
be due to outliers, or due to lack of (empirical) identification, or
due to sampling variability around the true (small) value of the
variance in population, or due to structural misspecificaiton. The
negative error variances are one of my favorite topics, thanks to the
work Ken Bollen and I are doing on this, so I can talk about them on
and on and on...

Post by sriram Narayanan
I have been seeing the archives for a discussion of multicollinearity.
I think I want to divide this into two parts.
1. Diagnose if there is a multicollinearity problem
2. Address the issue if multicollinearity is an issue.
I am dividing this because I want to bring some clarity into my head
about this issue, had some questions on them. I probably may be saying
something very silly but I just want to learn more about this problem
because I am facing such issues with my model that I have been
thinking. My model contains an X term and an X^2 term. (.97
correlation)
Method 1) Some earlier messages (I cant trace this message again
except I read it otherwise I would acknowledge the source) seemed to
suggest that one should look at the level of correlation between the
estimators. In lisrel I think PC option gives this. Very high
estimator correlations indicate that there is a problem with
multicollinearity. For example my model with AMOS converges with a
very bad fit, and I have problems with correlations between the
estimates involving the paths from x to the latent variable and x^2 to
the latent variable. However my model in LISREL converges fine.
Method 2) If the completely standardized coefficient is greater than
1 it can indicate multicollinearity as in one of Prof Joreskog's
article.
http://www.ssicentral.com/lisrel/techdocs/HowLargeCanaStandardizedCoefficientbe.pdf
Even though the correlations are high my completely standardized
correlations are less than 1.
What happens if one has a completely standardized coefficient of <1
but high correlations between the estimates. ?
Having a completely standardized coefficient of grater than 1 could be
a strong indicator of multicollinearity but can one also say that
having a less than 1 guarantees that there is no multicollinearity?
What is the theory behind the reasons for high estimator correlations
leading to multicollinearity. I will appreciate if there are any
references.
In my model diagnostic 2 seems fine even if the two indicators are
highly correlated in my case (the correlation between them is .97).
however the estimates are strongly correlated *,95 upwards)
Would I think that the collinearity problem is not serious?
Ed's suggested that one can do centering as has been suggested by
authors like Chronbach (1987, Psychological Bulletin) in this
listserv.
I have tried that strategy and it seemed to me that somehow in my data
it does not work I have a .96 correlation before centering and a .76
after centering. It does go down but I am not sure if this is low
enough. When I use this method diagnostic 1 still has some problems
while disgnstic 2 is fine in my model.
The other way is standardization that I found in an article by
Dunlap, William P. AND Edward R. Kemery (1988), "Effects of Predictor ...
INTERACTIONS AND Moderator Effects," Psychological Bulletin, 114 (2), 376-390
I have done log transformations to get my variable to be normal. If I
mean center this would it not affect my interpretation ?
Some clarity on this will be highly welcome. I will be very grateful.
Thanks
Sriram
--------------------------------------------------------------
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

sriram Narayanan

2006-01-24 07:22:00 UTC

Permalink

Stas:

Thanks a lot for your reply. I think it is very useful. I have a very
simple question on orthogonolization that is more of a followup. How
does then one interpret the ciefficients that one gets by
orthogonalizing the variables. The other issue is that if the
nonlinear hypotheses are a part of a larger model then is it the right
strategy to orthogonalize only the x and the x^2 term?

Sriram

Post by Stas Kolenikov
There are two separate issues here.
(i) Multicollinearity, at least in regression analysis, means that in
your data set, there are variables with high correlations, and due to
(X'X)^{-1} character of the variance estimator in linear regression,
those high correlations lead to big standard errors on (some) linear
combinations of the estimated coefficients, and big standard errors on
the variables themselves. There is a number of diagnostic methods
outlined in say a quarter century old Belsley, Kuh and Welsch
regression book, and in some other books; the most advanced methods,
to my own liking, are based on singular value decomposition (or PCA of
the regressor matrix, if you like), as they pinpoint the naughty
variables and quantify their contribution to multicollinearity.
(ii) In more complex models, and SEMs are certainly in this group, you
can have high correlation between your estimates for thousands of
other reasons. Some of them may be intrinsic and pretty much
incurable, like you always are going to have correlations between say
variance of a latent factor and variances of its indicators: given
factor loadings, a greater factor variance means there is less
variance left to be explained by error terms, so I would expect this
correlation to be negative. Some other kinds of estimate correlations
can be somewhat fixed by reparameterization of the model, and
centering is one of such tricks. It comes from the aforementioned
linear regression theory, where centering means you are breaking the
correlation between your variable and the constant term. Likewise, you
can make your x^2 term orthogonal to both the constant term and the
linear term {insted of x^2, you use x^2+ax+b, where you choose a and b
in such a way that sum_i (x_i^2+ax_i+b)=0 and sum_i
(x_i^2+ax_i+b)x_i=0}, and may improve correlations of the parameter
estimates. It is very likely that high correlations of the original
varialbes will give rise to the high correlations of the estimates,
but that needs to be checked for every particular case. Just looking
at the correlation matrix of the original variables may not give you
the best picture, especially given that some of those correlations are
going to be explained by the structural model itself.
Now, what are the consequences of multicollinearity, and is it really
that bad? I personally view those as somewhat inavoidable thing, at
least after you did your work on centering and such, and hence I would
rather sigh and say, "OK, this is how much multicollinearity I have".
In regression context, there are tons of shrinkage estimators that
admit some bias, but reduce the mean squared error of your estimates;
ridge regression, PCA regression, Stein, lasso and other methods come
under this rubric. Again, to my liking, this is the best you can go
about multicollinearity in the regression world. I don't think those
methods are easily transferrable to the SEMs, although say ridge
regression has a Bayesian justification, so if you are willing to dive
into the Bayesian world, then you can play around with that.
This brings me to another important consequence of high parameter
correlations (that may be due to multicollinearity in the variables,
or to other reasons). It is an irregularity of the likelihood surface
(or whatever your objective function is) near the maximum: it may have
a nice maximum in some directions, but it is almost flat in others. It
is terrible for numeric reasons. Any numeric optimization book will
tell you exactly why; basically, the iterative maximization procedures
are making steps in somewhat wrong directions even when you are close
to the true maximum. It is explicit in the methods that use the second
derivatives, and need to invert the Hessian matrix that is close to
singular (and hence the inverse may go south); but it is a complex
numeric problem, anyway. LISREL may have a better maximization
algorith implemented than AMOS; it may not even be documented very
well in the software manuals, so if you are really curious, you should
ask the developers. If your correlations are getting so high that the
condition numbers (the ratio of the max to min eigenvalues of the
information matrix/Hessian) are getting above 10^8, then it means that
you cannot find a maximum in single precision; if they are getting
higher than 10^15, you cannot really find a maximum in double
precision, and this is what most of the software uses as the standard
precisioin of the core engine, I believe. A serious maximizer routine
should give you condition numbers at the max, or you can compute them
yourself by doing the eigenproblem on the estimated
variance-covariance matrix of the estimates.
Note that going Bayesian does not save you from this curse: high
parameter correlations in MCMC world are getting reflected in slow
mixing, so you need to lengthen your MCMCs to get reasonable coverage
of the whole parameter space. Bayesians have their reparameterization
tricks for such purposes, too.
Hope that adds to your confusion :)))
Oh, and one more thing: I don't think the standardized coefficients
greater than one are very good indicators of multicollinearity. They
are necessarily accompanied by negative error variances, and that may
be due to outliers, or due to lack of (empirical) identification, or
due to sampling variability around the true (small) value of the
variance in population, or due to structural misspecificaiton. The
negative error variances are one of my favorite topics, thanks to the
work Ken Bollen and I are doing on this, so I can talk about them on
and on and on...

--
Stas Kolenikov
http://stas.kolenikov.name
--------------------------------------------------------------
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

Stas Kolenikov

2006-01-24 15:32:41 UTC

Permalink

I won't bother interpreting the coefficients, those are just some
technical steps along your estimation procedure. If you have a fixed
exogeneous regressor like age, then this is just the design issue (in
the sense of design of experiments issue, analysis of covariance as a
part of ANOVA, etc.), i.e., taking the data known to you before
observing the outcome and making most out of it. And if you are
absolutely crazy about multicollinearity, you can orthogonalize
everything in the exogeneous variable subspace, including say
interactions; probably there is no huge point in doing so except for
improving convergence and somewhat improving the standard errors, as
your essential estimates, the maximum of the likelihood and fit
statistics should be the same. If your nonlinear terms are a part of a
larger model, then the significance of the (orthogonalized) squared
term will have the same meaning as before: that there is a significant
second order effect (change in slope).

Regarding the nonlinearity issue -- I am brought up on the idea of
using splines rather than polynomials to model nonlinearity. They
allow for a richer class of models and don't exhibit the overshooting
behavior at the extremes of the ranges, as polynomials always do.

Post by sriram Narayanan
Thanks a lot for your reply. I think it is very useful. I have a very
simple question on orthogonolization that is more of a followup. How
does then one interpret the ciefficients that one gets by
orthogonalizing the variables. The other issue is that if the
nonlinear hypotheses are a part of a larger model then is it the right
strategy to orthogonalize only the x and the x^2 term?
Sriram

sriram Narayanan

2006-01-24 18:51:38 UTC

Permalink

Thanks a lot for your mail and suggestions. really appreciate it.

Regards

Sriram

Post by Stas Kolenikov
I won't bother interpreting the coefficients, those are just some
technical steps along your estimation procedure. If you have a fixed
exogeneous regressor like age, then this is just the design issue (in
the sense of design of experiments issue, analysis of covariance as a
part of ANOVA, etc.), i.e., taking the data known to you before
observing the outcome and making most out of it. And if you are
absolutely crazy about multicollinearity, you can orthogonalize
everything in the exogeneous variable subspace, including say
interactions; probably there is no huge point in doing so except for
improving convergence and somewhat improving the standard errors, as
your essential estimates, the maximum of the likelihood and fit
statistics should be the same. If your nonlinear terms are a part of a
larger model, then the significance of the (orthogonalized) squared
term will have the same meaning as before: that there is a significant
second order effect (change in slope).
Regarding the nonlinearity issue -- I am brought up on the idea of
using splines rather than polynomials to model nonlinearity. They
allow for a richer class of models and don't exhibit the overshooting
behavior at the extremes of the ranges, as polynomials always do.

Michael Gillespie

2006-01-24 23:37:59 UTC

Permalink

Stas,

Where can we get a more or less accessible discussion of splines?

Mike Gillespie

________________________________

From: Structural Equation Modeling Discussion Group on behalf of Stas Kolenikov
Sent: Tue 1/24/2006 9:56 AM
To: ***@bama.ua.edu
Subject: Re: more questions on multicollinearity

I won't bother interpreting the coefficients, those are just some
technical steps along your estimation procedure. If you have a fixed
exogeneous regressor like age, then this is just the design issue (in
the sense of design of experiments issue, analysis of covariance as a
part of ANOVA, etc.), i.e., taking the data known to you before
observing the outcome and making most out of it. And if you are
absolutely crazy about multicollinearity, you can orthogonalize
everything in the exogeneous variable subspace, including say
interactions; probably there is no huge point in doing so except for
improving convergence and somewhat improving the standard errors, as
your essential estimates, the maximum of the likelihood and fit
statistics should be the same. If your nonlinear terms are a part of a
larger model, then the significance of the (orthogonalized) squared
term will have the same meaning as before: that there is a significant
second order effect (change in slope).

Regarding the nonlinearity issue -- I am brought up on the idea of
using splines rather than polynomials to model nonlinearity. They
allow for a richer class of models and don't exhibit the overshooting
behavior at the extremes of the ranges, as polynomials always do.

--
Stas Kolenikov
http://stas.kolenikov.name

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

--------------------------------------------------------------
To unsubscribe from SEMNET, send email to ***@bama.ua.edu
with the body of the message as: SIGNOFF SEMNET
Search the archives at http://bama.ua.edu/archives/semnet.html

Urvashi

2006-01-25 04:19:51 UTC

Permalink

For example my model with AMOS converges with a
very bad fit, and I have problems with correlations between the
estimates involving the paths from x to the latent variable and x^2 to
the latent variable. However my model in LISREL converges fine.

Hi Sriram,

Paths leading to latent variable are indicative of formative indicators. Is it about the multicollinearity between formative indicators contributing to the latent variable? In that case, you need to consider the covariances amongst such formative indicators as free parameters. Following papers may be of some help:

Bollen Kenneth and Lennox Richard, "Conventional wisdom on measurement: A structural equation perspective", Psychological Bulletin, Vol. 110, No. 2, 1991, pp. 305 - 314.

MacCallum Robert C. and Browne Michael W., "The use of causal indicators in covariance structure models: Some practical issues", Psychological Bulletin, Vol. 114, No. 3, 1993, pp. 533 - 541.

Jarvis Cheryl Burke, Mackenzie Scott B. and Podsakoff Philip M., "A critical review of construct indicators and measurement model misspecification in marketing and consumer research", Journal of Consumer Research, Inc., Vol. 30, September 2003, pp. 199 - 218.

Diamantopoulos Adamantios and Winklhofer Heidi M., "Index construction with formative indicators: An alternative to scale development", Journal of Marketing Research, Vol. 38, pp. 269 - 277, May 2001.

Urvashi

----- Original Message -----
From: "sriram Narayanan" <***@GMAIL.COM>
To: <***@BAMA.UA.EDU>
Sent: Sunday, January 22, 2006 4:00 AM
Subject: more questions on multicollinearity