Measures used in quality rating and improvement systems (QRIS) validation studies
Investments in the development or revision of state and local Quality Rating and Improvement Systems (QRIS) for early care and education have increased in recent years, due in part to the Race to the Top - Early Learning Challenge (RTT-ELC) grants that were awarded to 20 states beginning in Fiscal Year 2011. The RTT-ELC grants included a requirement that states conduct evaluation activities to validate the rating process used by the QRIS to designate program quality levels and to assess the extent to which quality rating levels correlate with children's developmental outcomes. Though a number of QRIS had conducted evaluation studies prior to 2011, the RTT-ELC grants resulted in multiple QRIS validation studies that were launched in 2012-2015. These validation studies assess the extent to which the QRIS rating process produces meaningfully distinct ratings of program quality and if quality rating levels correlate with children's developmental outcomes. This research brief addresses the need for an aggregate picture of the methods, measures, and analytic strategies being used in QRIS validation studies by summarizing the measures that researchers are currently using or plan to use. To gather information on current QRIS validation study measures, Child Trends compiled data used to inform the Quality Initiatives Research and Evaluation Consortium (INQUIRE) validation workgroup, and reviewed published reports and research plans for RTT-ELC validation studies. A total of 19 QRIS validation studies were examined. All 19 of the validation studies are conducting analyses to assess how well items on the rating tool are working; 18 of the studies are assessing whether program quality ratings are different in meaningful ways; and, 16 studies are assessing whether quality ratings are related to measures of children's development. The Environment Rating Scales (ERS) and the Classroom Assessment Scoring System (CLASS) are the most commonly used observation tools in validation studies; in contrast, the set of child development measures used in validation studies varied across state. The brief also summarizes the structural quality indicators collected in the studies, covariates used in validation analyses, and some of the challenges that have been encountered in choosing a school readiness battery. The research brief was developed to inform discussions in the INQUIRE validation subgroup meetings but is being shared widely to facilitate a common understanding among QRIS stakeholders about QRIS validation studies. (author abstract)
Related resources include summaries, versions, measures (instruments), or other resources in which the current document plays a part. Research products funded by the Office of Planning, Research, and Evaluation are related to their project records.
Child Care and Early Education Policy and Research Analysis Project (CCEEPRA)
Administration for Children and Families/OPRE Projects