Thinking Like a Scientist: Can Interactive Simulations Foster Critical AI Literacy?

As AI systems shape individual and societal decisions, fostering critical AI literacy is essential. Traditional approaches--such as blog articles, static lessons, and social media discussions--often fail to support deep conceptual understanding and critical engagement. This study examines whether interactive simulations can help learners "think like a scientist" by engaging them in hypothesis testing, experimentation, and direct observation of AI behavior. In a controlled study with 605 participants, we assess how interactive AI tutorials impact learning of key concepts such as fairness, dataset representativeness, and bias in language models. Results show that interactive simulations effectively enhance AI literacy across topics, supporting greater knowledge transfer and self-reported confidence, though engagement alone does not predict learning. This work contributes to the growing field of AI literacy education, highlighting how interactive, inquiry-driven methodologies can better equip individuals to critically engage with AI in their daily lives.