TY - JOUR

T1 - AI-experiments in education: An AI-driven randomized controlled trial for higher education research

AU - Cingillioglu, Ilker

AU - Gal , Uri

AU - Прохоров, Артем Борисович

PY - 2024/10/1

Y1 - 2024/10/1

N2 - This study presents a novel approach contributing to our understanding of the design, development, and implementation AI-based systems for conducting double-blind online randomized controlled trials (RCTs) for higher education research. The process of the entire interaction with the participants (n = 1193) and their allocation to test and control groups was executed seamlessly by our AI system, without human intervention. In this fully automated experiment, we systematically examined eight hypotheses. The AI-experiment strengthened five of these hypotheses, while not accepting three of the factors previously acknowledged in the literature as influential in students’ choices of universities. We showcased how AI can efficiently interview participants and collect their input, offering robust evidence through an RCT (Gold standard) to establish causal relationships between interventions and their outcomes. This approach may enable researchers and industry practitioners to collect data from large samples on which such experiments can be conducted with and by AI to produce statistically reproducible, reliable, and generalizable results in an efficient, rigorous and ethical way.

AB - This study presents a novel approach contributing to our understanding of the design, development, and implementation AI-based systems for conducting double-blind online randomized controlled trials (RCTs) for higher education research. The process of the entire interaction with the participants (n = 1193) and their allocation to test and control groups was executed seamlessly by our AI system, without human intervention. In this fully automated experiment, we systematically examined eight hypotheses. The AI-experiment strengthened five of these hypotheses, while not accepting three of the factors previously acknowledged in the literature as influential in students’ choices of universities. We showcased how AI can efficiently interview participants and collect their input, offering robust evidence through an RCT (Gold standard) to establish causal relationships between interventions and their outcomes. This approach may enable researchers and industry practitioners to collect data from large samples on which such experiments can be conducted with and by AI to produce statistically reproducible, reliable, and generalizable results in an efficient, rigorous and ethical way.

KW - AI experiments

KW - AI-based chatbots

KW - AI-led RCT

KW - Social online experiments

UR - https://www.mendeley.com/catalogue/3d3b01c2-2d0b-3cb7-9a6d-646ba105a12f/

U2 - 10.1007/s10639-024-12633-y

DO - 10.1007/s10639-024-12633-y

M3 - Article

VL - 29

SP - 19649

EP - 19677

JO - Education and Information Technologies

JF - Education and Information Technologies

SN - 1360-2357

IS - 15

ER -