Language learning chatbots are among the most promising AI applications, widely praised for helping learners improve communication, reflexes, and vocabulary in a personalized way. Thanks to their ability to understand natural language and respond through text or voice, chatbots are becoming popular study companions across many educational platforms.
However, the key questions remain: Are learners and teachers truly satisfied with chatbot-based learning experiences? And what factors define an effective learning chatbot?
A recent study published in the International Journal of Educational Technology in Higher Education (IJETHE) offers some answers. Researchers Jose Belda-Medina and Vendula Kokošková surveyed 237 students in Spain and the Czech Republic using four popular English-learning chatbots (Andy, Mondly, John Bot, and Buddy.ai). The goal was to measure satisfaction using the CHISM framework (Chatbot-Human Interaction Satisfaction Model).
The CHISM Model – A New Measure for Evaluating Learning with Chatbots
Unlike general user feedback, the CHISM model evaluates learner satisfaction across three core dimensions:
- Linguistic Experience (LEX): Quality and coherence of responses, sentence length and complexity, speech recognition and pronunciation, vocabulary and grammar explanations.
- Design Experience (DEX): User-friendly interface, multimedia integration, personalization options tailored to individual learners.
- User Experience (UEX): Engagement level, emotional connection during usage, and long-term usefulness.
The results showed that user satisfaction was moderate — revealing significant room for improvement in areas such as language capabilities, design, and interaction quality.
Strengths and Weaknesses Identified from Real User Experiences
Among the tested chatbots, Andy and John Bot scored highest in terms of coherent conversation, language diversity, and adaptive responses for different learning levels. Andy stood out for its ability to adjust language complexity based on user feedback, while also clearly explaining vocabulary and grammar.
Buddy.ai, designed for children, was highly rated for its speech recognition, but lacked flexibility for adult learners.
Common weaknesses reported by students included: repetitive and templated answers, difficulty understanding complex or naturally phrased sentences, and limited ability to handle regional accents or natural speech errors.
Chatbots as Companions — Not Replacements
A key takeaway from the study is how students and future educators perceive chatbots: not as teacher replacements, but as valuable learning companions that:
- Allow for language practice anytime, anywhere
- Reduce communication anxiety without fear of judgment
- Suggest learning content tailored to each user’s level
That said, chatbots truly add value only when integrated into a smart learning ecosystem — one that tracks progress, personalizes content, and connects technology with human guidance.
Opportunities in Vietnam – ICSC’s Perspective
At ICSC, we see strong potential for language learning chatbots in Vietnam — a country where demand for English and other foreign languages is growing fast, but qualified teaching resources remain limited.
AI-integrated chatbots (AICs) can act as intelligent learning assistants, especially when developed based on models like CHISM that put learner experience at the center. ICSC AI is currently researching localized AI Agents for Vietnamese students, capable of bilingual interaction (Vietnamese–English) and personalized learning support.
Language learning chatbots are quickly becoming an inevitable trend in modern education. Yet, for them to be truly effective, a blend of technology, pedagogy, and user experience is essential. The CHISM model provides a reliable framework for building and refining chatbot systems that deliver impactful learning.
If your organization or educational institution is exploring chatbot-based learning solutions, get in touch with ICSC — our team of AI and digital education experts is ready to help you succeed.