Chatbots are used in higher education for several applications, including increasing student engagement, streamlining the enrollment process and supporting students academically. AI-enabled bots can help universities and colleges improve communications of all types, answering student questions, alerting administration to complex situations and enabling professors to easily share information with students.
Now, empathetic chatbots can also help students succeed and persist. Research from Georgia State University (GSU) found that using periodic, timely text messages actually can help students improve academic performance.
GSU piloted the Virtual Teaching Assistant from Mainstay. The chatbot that uses AI and behavioral intelligence to communicate proactively with students about coursework in an empathetic manner. The Virtual Teaching Assistant alerts students via text messages about course requirements and deadlines, provided customized feedback on students’ progress, offered practice quizzes and gave encouragement along the way. The chatbot provided an extra layer of support that both ensures that fewer things slip through the cracks and also listens and responds to students, so learners get customized support for any issues they may have.
Mainstay has used behavioral intelligence in the past to help colleges increase access, persistence and success. GSU has used the Mainstay solutions for the past six years. In 2016, it launched a chatbot to address summer melt. Every incoming student engaged with the chatbot – named “Pounce” to coordinate with the school’s giant fuzzy panther mascot – which supported enrollment.
Lindsey Fifield, Director of Student Success Communication at GSU coordinates all student-directed communication and manages Pounce. “It had an immediate impact on the summer melt rate, reducing it by 4 percent,” she explained. “We scaled it fully in the admissions space and then decided to explore applications to help students navigate through the university to help our persistent rates or graduation rates just by adding a similar communications strategy for existing students.”
GSU implemented a new chatbot in 2018-2019 for continuing students and with a randomized control trial (RCT), discovered the effort yielded similar results. “Students visited their advisors more often, completed their FAFSA more often and doing more critical tasks at much higher rates than those who didn’t get the texts,” Fifield pointed out. Persistence rates improved about 3 percent, equating to 1300 students who otherwise may not have graduated.
GSU then extended the RCT to students at its embedded Georgia Perimeter College and the technology achieved the same results. “The bot helps encourage positive student behaviors for positive outcomes,” she added.
In the Spring of 2020, the school did a pilot in the academic space, in a political science course, leveraging the relationship that students had built with the admission and retention bot. The school then moved into a full RCT in the fall of 2021. This bot reached out to students with messages from professors that are highly relevant to students, reminding them of upcoming assignment deadlines and checking in to see how they’re progressing and to encourage them to complete tasks.
The research revealed that bot-generated texts do impact student progress. Those who received texts were 8 percent more likely to get a B in the class. More importantly, the technology clearly benefitted one group: First generation college students. This group scored 11 percentage points higher in the class next to counterparts in the group of students who did not received the texts.
The messaging cadence is optimized to ensure that students aren’t receiving more than two or three texts a week. “We’re operating the retention bot and academic bot, so we have to coordinate to make sure students don’t receive messages on the same day,” Fifield explained.
The bot not only provides text messages but converses with students. Drew Magliozzi, CEO of Mainstay, explained that the system is able to have conversations at scale, not only with intelligence but with empathy and context, being deeply aware of the place, the community and the students’ unique situations.
“With each iteration, the tool has improved. “The best part of the product is that it gets better with usage over time,” Magliozzi explained. “Students ask questions, the knowledge base gets built out, engagement and insights you derive from it help you fine-tune the intervention. If you’re operating the tool, you’re always looking at the conversation, seeing what’s on people’s minds, where they’re getting stuck, where they’re asking questions and then what are they saying and providing for feedback.”
During the pandemic, the bot provided a sense of support and connection for students they weren’t getting elsewhere. “We got thousands of messages from students telling us that you were the only one checking on me, the only one I could rely on, and without you, I wouldn’t have made it through this time without you,” Magliozzi said. “Seeing that level of extreme connection is mind boggling. There’s something about the intimacy of a text message, the frequency of proactive outreach and the way and care and empathy we infuse into the product helps people know they’re not alone.”
Right now, Mainstay is exploring a similar approach that we know will work in different courses. While the Virtual Assistant has been successful in humanities course applications Mainstay is beginning to explore its use in STEM courses.