Optimum Sample Size and Methods of Estimation for Multilevel Modeling

Abstract

An important problem in Multilevel Modeling (MLM) is to find sufficient sample size for accurate estimation purposes. In MLM apart from the general factors of sample size estimation i.e. the test size, the effect size, SE (standard error) of the effect size and power of the test, additional factors like, magnitude of the ICC (Intra Class Correlation), total number of clusters, the number of parameters to be estimated, and the information whether the design is balanced or unbalanced may play a significant role. In this study, the significance of these factors in the context of MLMs is evaluated and their mutual relationship is explored through simulation study. It is found that little problem will arise in estimation of sample size for fitting multi-level model if standardized effect size δ is 0.2 or lower and intra-class correlation ρ is also low (0.05 or less), here substantial power (0.80) can be achieved when J (Number of clusters) is 50 or more. A small upward shift in the ρ (0.1), causes alarming increase in the total number of clusters to be sampled. Secondly, it is observed that the number of clusters to be sampled is playing a greater role in power enhancement as compared to cluster size. Thirdly, significant changes in the power of estimation are observed when the effect size δ increases from 0.2 to 0.4. Fourthly, the level II covariate (with R2= 0.3, 0.6) if added in the model in the model it can significantly increase power of the multilevel models even in the presence of small number of clusters. Two methods of estimation commonly used for multilevel modeling, IGLS (Iterative Generalized Least Square Method) and MCMC (Monte Carlo Markov Chain) are compared with variety of models using limited and extended simulations. The study reveals that the MCMC estimates for the fixed effect is superior to its counterpart IGLS on all the three grounds i.e. unbiasedness, efficiency, and the proportion of the true value captured by the confidence interval/credible interval. As far as estimation of the random effect is concerned, MCMC is better on the proportion of the true value covered, however, IGLS gain considerable lead on the unbiased and efficiency criterion. Further, it is observed that MCMC respond more positively to the extended simulations as compared to IGLS.