Human-coded benchmark
Established codebooks and adjudicated human codes define the constructs and provide reference data for evaluation.
Program overview
Scale is useful only if the meaning of behavior survives.
Fine-grained behavioral coding makes parent–child interaction research possible, but it is time intensive and depends on contextual judgment. This ongoing methods program asks where language models and other machine-coding approaches can assist without turning nuanced behavior into unexamined labels.
The work begins with existing human behavioral coding systems and previously coded data from the Early Mathematics Learning Project, Technology-Mediated Learning project, and Etch-a-Sketch project. Those human judgments provide the basis for training, testing, correcting, and validating machine-generated codes.
Methodological principles
Human expertise remains the reference point
Machine assistance
A model proposes candidate labels or helps prioritize cases; its output is treated as a hypothesis, not a final judgment.
Review and accountability
Researchers examine disagreement, correct errors, and document reliability before machine-supported codes enter analysis.
Data foundations
Building from behavioral systems developed in earlier projects
EMLP
Early Mathematics Learning Project
Structured parent–child mathematics interactions coded in repeated behavioral intervals.
Available coding systems
- Parent mathematics prompts and statements
- Children’s mathematics strategies and engagement
- Parental autonomy support and control
TML
Technology-Mediated Learning
Online mathematics interactions coded at the sentence and minute levels.
Available coding systems
- Parent and child mathematics talk
- Parental autonomy support and control
- Parent and child positive and negative affect
EAS
Etch-a-Sketch Interactions
Sentence-level coding of talk during a shared spatial problem-solving task.
Available coding systems
- Parent and child spatial talk
- Parent and child mathematics talk
- Mixed or ambiguous spatial–mathematical utterances flagged for human review
Public status: Ongoing methods development. Welcome collaborations.
Development workflow
From human definitions to validated machine-supported codes
Human benchmark
Translate established behavioral definitions into testable machine-coding tasks. Use human-coded data as training and evaluation material.
Machine candidate
Generate candidate codes for comparison with the human benchmark, treating each output as a proposal rather than a final judgment.
Human review
Inspect disagreements to locate ambiguity, missing context, and systematic error. Retain human correction and documentation throughout the pipeline.
Validation
Evaluate reliability and validity before using machine-supported codes in substantive research.