Child Care and Early Education Research Connections

Skip to main content

Statistical analysis for multisite trials using instrumental variables with random coefficients

Multisite trials can clarify the average impact of a new program and the heterogeneity of impacts across sites. Unfortunately, in many applications, compliance with treatment assignment is imperfect. For these applications, we propose an instrumental variable (IV) model with person-specific and site-specific random coefficients. Site-specific IV coefficients can be interpreted as site-average effects of program participation or as site-average effects of participation for the compliers. The validity of these interpretations depends on the analyst's assumptions. Within the framework of a two-level hierarchical linear model, we propose three ways to estimate the mean and variance of these site-specific effects: (a) estimate the impact of program participation and its standard error in each site, then combine these site-specific statistics to estimate the mean and variance of the true site effects; (b) estimate the mean and variance of the effect of treatment assignment on the outcome and the mean and variance of the effect of treatment assignment on program participation; then combine these results to obtain estimates of the mean and variance of the effect of program participation; and (c) use Site by Treatment interactions as multiple instruments. If we assume the IV coefficients to be homogenous across sites, the three approaches are equivalent to variants of familiar two-stage least squares estimates with site fixed effects. Estimates based on our model are valid under a weaker assumption: that site-average levels of compliance are independent of site-average effects of program participation. To illustrate our approach, we evaluate a district-wide policy intended to increase math instructional time in math for low-achieving students. Finally, we discuss how Method (c) can be extended to incorporate multiple mediators. (author abstract)
Resource Type:
Reports & Papers
United States

- You May Also Like

These resources share similarities with the current selection.

The effects of an undergraduate programme of preschool teaching on preservice teachers' attitudes towards early mathematics education in Turkey: A longitudinal study

Reports & Papers

Early childhood educators working with children who have lesbian, gay, bisexual and transgender parents: What does the literature tell us?

Literature Review

Developing leadership for early childhood professionals through oral inquiry: Strengthening equity through making particulars visible in dilemmas of practice

Reports & Papers

Improving QRISs through the use of existing data: A virtual pilot of the California QRIS

Reports & Papers
Release: 'v1.25.0' | Built: 2023-02-16 16:36:50 EST