An empirical examination of respondent driven sampling design effects among HIV risk groups from studies conducted around the world

For studies using respondent driven sampling (RDS), the current practice of collecting a sample twice as large as that used in simple random sampling (SRS) (i.e. design effect of 2.00) may not be sufficient. This paper provides empirical evidence of sample-to-sample variability in design effects using data from nine studies in six countries among injecting drug users, female sex workers, men who have sex with men and male-to-female transgender (MTF) persons. We computed the design effect as the variance under RDS divided by the variance under SRS for a broad range of demographic and behavioral variables in each study. We also estimated several measures for each variable in each study that we hypothesized might be related to design effect: the number of waves needed for equilibrium, homophily, and mean network size. Design effects for all studies ranged from 1.20 to 5.90. Mean design effects among all studies ranged from 1.50 to 3.70. A particularly high design effect was found for employment status (design effect of 5.90) of MTF in Peru. This may be explained by a "bottleneck" - defined as the occurrence of a relatively small number of recruitment ties between two groups in the population. A design effect of two for RDS studies may not be sufficient. Since the mean design effect across all studies was 2.33, an effect slightly above 2.00 may be adequate; however, an effect closer to 3.00 or 4.00 might be more appropriate.