520 likes | 778 Vues
Testing Model Fit. SOC 681 James G. Anderson, PhD. Limitations of Fit Indices. Values of fit indices indicate only the average or overall fit of a model. It is thus possible that some parts of the model may poorly fit the data.
E N D
Testing Model Fit SOC 681 James G. Anderson, PhD
Limitations of Fit Indices • Values of fit indices indicate only the average or overall fit of a model. It is thus possible that some parts of the model may poorly fit the data. • Becauwe a single index reflects only a particul;ar aspect oof model fit, a favorable value of that indesx does not by itself indicate good fit. That is why model fit is assessed based on the values of multiple indices.
Limitations of Fit Statistics • Fit indices do not indicate whether the results are theoretically meaningful. • Values of fit indices that suggest adequate fit do not indicate the predictive power of the model is also high. • The sampling distribution of many fit indices are unknown.
Assessment of Model Fit • Examine the parameter estimates • Examine the standard errors and significance of the parameter estimates. • Examine the squared multiple correlation coefficients for the equations • Examine the fit statistics • Examine the standardized residuals • Examine the modification indices
Measures of Fit • Measures of fit are provided for three models: • Default Model – this is the model that you specified • Saturated Model – This is the most general model possible. No constraints are placed on the population moments It is guaranteed to fit any set of data perfectly. • Independence Model – The observed variables are assumed to be uncorrelated with one another.
Overall measures of Fit • NPAR is the number of parameters being estimated (q) • CMIN is the minimum value of the discrepancy function between the sample covariance matrix and the estimated covariance matrix. • DF is the number of degrees of freedom and equals the p-q • p=the number of sample moments • q= the number of parameters estimated
Overall measures of Fit • CMIN is distributed as chi square with df=p-q • P is the probability of getting as large a discrepancy with the present sample • CMIN/DF is the ratio of the minimum discrepancy to degrees of freedom. Values should be close to 1.0 for correct models.
Chi Square: 2 • Best for models with N=75 to N=100 • For N>100, chi square is almost always significant since the magnitude is affected by the sample size • Chi square is also affected by the size of correlations in the model: the larger the correlations, the poorer the fit
Chi Square to df Ratio: 2/df • There are no consistent standards for what is considered an acceptable model • Some authors suggest a ratio of 2 to 1 • In general, a lower chi square to df ratio indicates a better fitting model
Transforming Chi Square to Z Z = (2*2) - (2*df-1)
RMR, GFI • RMR is the Root Mean Square Residual. It is the square root of the average amount that the sample variances and covariances differ from their estimates. Smaller values are better • GFI is the Goodness of Fit Index. GFI is between 0 and 1 where 1 indicates a perfect fit. Acceptable values are above 0.90.
GFI and AGFI (LISREL measures) • The AGFI takes into consideration the df available to test the model. • Values close to .90 reflect a good fit. • These indices are affected by sample size and can be large for poorly specified models. • These are usually not the best measures to use.
RMR, GFI • AGFI is the Adjusted Goodness of Fit Index. It takes into account the degrees of freedom available for testing the model. Acceptable values are above 0.90. • PGFI is the Parsimony Goodness of Fit Index. It takes into account the degrees of freedom available for testing the model. Acceptable values are above 0.90.
Comparisons to a Baseline Model • NFI is the Normed Fit Index. It compares the improvement in the minimum discrepancy for the specified (default) model to the discrepancy for the Independence model. A value of the NFI below 0.90 indicates that the model can be improved.
Bentler-Bonett Index or Normed Fit Index (NFI) • Define null model in which all correlations are zero: 2 (Null Model) - 2 (Proposed Model) 2 (Null Model) • Value between .90 and .95 is acceptable; above .95 is good • A disadvantage of this index is that the more parameters, the larger the index.
Comparisons to a Baseline Model • RFI is the Relative Fit Index This index takes the degrees of freedom for the two models into account. • IFI is the incremental fit index. Values close to 1.0 indicate a good fit. • TLI is the Tucker-Lewis Coefficient and also is known as the Bentler-Bonett non-normed fit index (NNFI). Values close to 1.0 indicate a good fit. • CFI is the Comparative Fit Index and also the Relative Noncentrality Iindex (RNI). Values close to 1.0 indicate a good fit.
Tucker Lewis Index or Non-normed Fit Index (NNFI) • Value: 2/df(Null Model) - 2/df(Proposed Model) 2/df(Null Model) • If the index is greater than one, it is set to1. • Values close to .90 reflects a good model fit. • For a given model, a lower chi-square to df ratio (as long as it is not less than one) implies a better fit.
Comparative Fit Index (CFI) • If D= 2 - df, then: D(Null Model) - D(Proposed Model) D(Null Model) • If index > 1, it is set to 1; if index <0, it is set to 0 • A lower value for D implies a better fit • If the CFI < 1, then it is always greater than the TLI • The CFI pays a penalty of one for every parameter estimated
Parsimony Adjusted Measures • PRATIO is the Parsimony Ratio. It is the number of constraints in the model being evaluated as a fraction of the number of constraints in the independence model. • PNFI is the result of applying the PRATIO to the NFI. • PCFI is the result of applying parsimony adjustments to the CFI.
Measures Based on the Population Discrepancy • NCP is an estimate of the noncentrality parameter obtained by fitting a model to the population moments rather than to the sample moments. • The 90% confidence interval is also computed for the NCP.
The Minimum Sample Discrepancy Function • FMIN is the minimum value of the discrepancy .
Root Mean Square Error of Approximation (RMSEA) • Value: [(F0pop-F0est)/n]/df • F0 is the minimum vale of the discrepancy function. • If 2 < df for the model, RMSEA is set to 0 • Good models have values of < .05; values of > .10 indicate a poor fit. • It is a parsimony-adjusted measure. • Amos provides upper and lower limits of a 90% confidence interval for the RMSEA
PCLOSE • PCLOSE is the probability for testing the null hypothesis that the population RMSEA is no greater than 0.05.
Information Theoretic Measures • These indices are composite measures of badness of fit and complexity. • Simple models that fit well receive low scores. Complicated poorly fitting models get high scores. • These indices are used for model comparison not to evaluate a single model.
Akaike Information Criterion (AIC) • Value: 2 + k(k-1) - 2(df) where k= number of variables in the model • A better fit is indicated when AIC is smaller • Not standardized and not interpreted for a given model. • For two models estimated from the same data, the model with the smaller AIC is preferred.
Information Theoretic Measures • BCC is the Browne-Cudeck Criterion • BIC is Bayes Information Criterion. • CAIC is the consistent AIC • ECVI except for a constant scale factor it is the same as AIC. • MECVI except for a scale factor is the same as the BCC.
Difference in Chi Square Value: X2diff = X2model 1 -X2 model 2 DFdiff = DF model 1 –DFmodel 2
Miscellaneous Measures • HOELTER is the largest sample size for which one would accept the hypothesis that a model is correct.
Hoelter Index • Value: (N-1)*2(crit) + 1 2 Where 2 (crit) is the critical value for the chi-square statistic • The index should only be calculated if chi square isstatistically significant.
Hoelter Index (2) • If the critical value is unknown, can approximate: [ (1.645 + (2df-1) ]2 + 1 2 2/ (N-1) • For both formulas, one rounds down to the nearest integer • The index states the sample size at which the chi square would not be significant
Hoelter Index (3) • In other words, how small one’s sample size would have to be for chi square to no longer be significant • Hoelter Recommends values of at least 200 • Values < 75 indicate poor fit