Child Care and Early Education Research Connections

Skip to main content

Fighting bias with statistics: Detecting gender differences in responses to items on a preschool science assessment

Share
Description:
Differential item functioning (DIF) and differential distractor functioning (DDF) are methods used to screen for item bias (Camilli & Shepard, 1994; Penfield, 2008). Using an applied empirical example, this mixed-methods study examined the congruency and relationship of DIF and DDF methods in screening multiple-choice items. Data for Study I were drawn from item responses of 271 female and 236 male low-income children on a preschool science assessment. Item analyses employed a common statistical approach of the Mantel-Haenszel log-odds ratio (MH-LOR) to detect DIF in dichotomously scored items (Holland & Thayer, 1988), and extended the approach to identify DDF (Penfield, 2008). Findings demonstrated that the using MH-LOR to detect DIF and DDF supported the theoretical relationship that the magnitude and form of DIF and are dependent on the DDF effects, and demonstrated the advantages of studying DIF and DDF in multiple-choice items. A total of 4 items with DIF and DDF and 5 items with only DDF were detected. Study II incorporated an item content review, an important but often overlooked and under-published step of DIF and DDF studies (Camilli & Shepard). Interviews with 25 female and 22 male low-income preschool children and an expert review helped to interpret the DIF and DDF results and their comparison, and determined that a content review process of studied items can reveal reasons for potential item bias that are often congruent with the statistical results. Patterns emerged and are discussed in detail. The quantitative and qualitative analyses were conducted in an applied framework of examining the validity of the preschool science assessment scores for evaluating science programs serving low-income children, however, the techniques can be generalized for use with measures across various disciplines of research. (author abstract)
Resource Type:
Reports & Papers
Author(s):
Country:
United States
State(s):
Florida

Related resources include summaries, versions, measures (instruments), or other resources in which the current document plays a part. Research products funded by the Office of Planning, Research, and Evaluation are related to their project records.

- You May Also Like

These resources share similarities with the current selection. They are found by comparing the topic, author, and resource type of the currently selected resource to the rest of the library’s publications.

Promoting positive parent-child relationships through Early Head Start home visits

Reports & Papersview

Understanding Head Start children and families in spring 2020: FACES 2019 descriptive data tables and study design

Reports & Papersview

Navigating remote delivery of assessments for Head Start children during the COVID-19 pandemic

Reports & Papersview
Release: 'v1.9.0' | Built: 2022-06-13 15:23:18 EDT