Background: Given the growing availability of multilevel data from national surveys, researchers interested in contextual effects may find themselves with a small number of individuals per group. Although there is a growing body of literature on sample size in multilevel modelling, few have explored the impact of group sizes of less than five.
Methods: In a simulated analysis of real data, the impact of a group size of less than five was examined on both a continuous and dichotomous outcome in a simple two-level multilevel model. Models with group sizes one to five were compared with models with complete data. Four different linear and logistic models were examined: empty models; models with a group-level covariate; models with an individual-level covariate and models with an aggregated group-level covariate. The study evaluated further whether the impact of small group size differed depending on the total number of groups.
Results: When the number of groups was large (N=459), neither fixed nor random components were affected by small group size, even when 90% of tracts had only one individual per tract and even when an aggregated group-level covariate was examined. As the number of groups decreased, the SE estimates of both fixed and random effects were inflated. Furthermore, group-level variance estimates were more affected than were fixed components.
Conclusions: Datasets in which there is a small to moderate number of groups, with the majority of very small group size (n<5), size may fail to find or even consider a group-level effect when one may exist and also may be underpowered to detect fixed effects.