Khanmigo doesn’t answer student questions directly, but starts with questions of its own, such as asking whether the student has any ideas about how to find an answer. Then it guides them to a solution, step by step, with hints and encouragement.
Notwithstanding Khan’s expansive vision of “amazing” personal tutors for every student on the planet, DiCerbo assigns Khanmigo a more limited teaching role. When students are working independently on a skill or concept but get hung up or caught in a cognitive rut, she says, “we want to help students get unstuck.”
Some 100,000 students and teachers piloted Khanmigo this past academic year in schools nationwide, helping to flag any hallucinations the bot has and providing tons of student-bot conversations for DiCerbo and her team to analyze.
“We look for things like summarizing, providing hints and encouraging,” she explains.
The degree to which Khanmigo has closed AI’s engagement gap is not yet known. Khan Academy plans to release some summary data on student-bot interactions later this summer, according to DiCerbo. Plans for third-party researchers to assess the tutor’s impact on learning will take longer.
AI Feedback Works Both Ways
Since 2021, the nonprofit Saga Education has also been experimenting with AI feedback to help tutors better engage and motivate students. Working with researchers from the University of Memphis and the University of Colorado, the Saga team pilot in 2023 fed transcripts of their math tutoring sessions into an AI model trained to recognize when the tutor was prompting students to explain their reasoning, refine their answers, or initiate a deeper discussion. The AI analyzed how often each tutor took these steps.
Tracking some 2,300 tutoring sessions over several weeks, they found that tutors whose coaches used the AI feedback peppered their sessions with significantly more of these prompts to encourage student engagement.
While Saga is looking into having AI deliver some feedback directly to tutors, it’s doing so cautiously because, according to Brent Milne, the vice president of product research and development at Saga Education, “having a human coach in the loop is really valuable to us.”
Experts expect that AI’s role in education will grow, and its interactions will continue to seem more and more human. Earlier this year, OpenAI and the startup Hume AI separately launched “emotionally intelligent” AI that analyzes tone of voice and facial expressions to infer a user’s mood and respond with calibrated “empathy.” Nevertheless, even emotionally intelligent AI will likely fall short on the student engagement front, according to Brown University computer science professor Michael Littman, who is also the National Science Foundation’s division director for information and intelligent systems.
No matter how humanlike the conversation, he says, students understand at a fundamental level that AI doesn’t really care about them, what they have to say in their writing, or whether they pass or fail subjects. In turn, students will never really care about the bot and what it thinks. A June study in the journal Learning and Instruction found that AI can already provide decent feedback on student essays. What is not clear is whether student writers will put in care and effort, rather than offload the task to a bot, if AI becomes the primary audience for their work.
“There’s incredible value in the human relationship component of learning,” Littman says, “and when you just take humans out of the equation, something is lost.”
This story about AI tutors was produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.