Personalized for Whom?

In this episode we explore new research on bias in AI writing feedback and what it means for teachers, school leaders, and anyone evaluating AI-powered writing tools in K–12 education. We look at a Stanford preprint on how large language models responded differently to the same student essay when only the attached demographic profile changed, and we connect that to earlier MIT research showing that AI systems can shift tone and quality based on who they think they are talking to. The bigger question: when AI claims to “personalize” feedback, is it actually supporting equity, or quietly automating lower expectations?Topics covered:How a Stanford research team tested AI writing feedback using 600 real eighth-grade persuasive essaysWhy changing only demographic labels changed the feedback students receivedWhat “positive feedback bias” and “feedback withholding bias” can look like in classroom practiceHow AI can give more praise but less useful critique to some studentsWhat the earlier MIT chatbot study revealed about tone, condescension, and perceived vulnerabilityWhy AI “personalization” can slip into profilingWhat teachers should ask before trusting AI-generated writing feedbackHow students can be taught to question, audit, and respond critically to AI feedbackWhat curriculum leaders and district leaders should demand from vendors about inputs, transparency, and equity testingSource:https://arxiv.org/pdf/2603.12471