Student-Artificial Intelligence Conversations and Sentiments: A Process Data Analysis and Reciprocal Effects
Research on academic outcomes has extensively explored students’ emotions, motivation, and learning behavior. While further research highlights .
- Pub. date: June 15, 2025
- Pages: 69-82
- 49 Downloads
- 196 Views
- 0 Citations
- #AI assistants
- # campus-management-system chats
- # hierarchical Bayesian continuous time dynamic modeling
- # higher education
- # sentiment analysis
- # university students.
Abstract:
Research on academic outcomes has extensively explored students’ emotions, motivation, and learning behavior. While further research highlights the role of individual interactions with generative artificial intelligence (AI), a gap exists in understanding the longitudinal dynamic associations between university students' interactions with AI assistants and the sentiment expressed during these conversations over time. Based on a theoretical framework, we expected reciprocal effects between students’ interactions with AI assistants and sentiment within their conversation, considering students’ gender, age, and type of AI assistant used. Over four months, students at a university had access to three AI assistants: Alix for motivational-emotional support, Robyn for reflective discussions, and Dr. Melly for exam discussions. All AI assistants are integrated into a campus management system, ensuring compliance with data protection regulations. The sample for this process data analysis comprised n = 5,262 interactions of 422 students (56% female students, MAge = 30.42, SDAge = 9.08). Time series analysis with hierarchical Bayesian continuous time dynamic modeling revealed significant reciprocal effects between the students’ question length and the sentiment during these conversations: Students’ relatively short questions at the beginning of a conversation determined a later positive sentiment. A positive sentiment at the beginning of a conversation determined relatively short questions from the students later on, in particular from female students. Another result is that the type of AI assistance moderated the reciprocal effects. Thus, the students perceived the AI assistants as helpful tools for addressing questions during the day and nighttime in higher education.
ai assistants campus management system chats hierarchical bayesian continuous time dynamic modeling higher education sentiment analysis university students
Keywords: AI assistants, campus-management-system chats, hierarchical Bayesian continuous time dynamic modeling, higher education, sentiment analysis, university students.
0
References
Al-Samarraie, H., Muthana Sarsam, S., & Alzahrani, A. I. (2023). Haptic technology in society: A sentiment analysis of public engagement. Computers in Human Behavior, 147, Article 107862. https://doi.org/10.1016/j.chb.2023.107862
Al-Sharafi, M. A., Al-Emran, M., Arpaci, I., Iahad, N. A., AlQudah, A. A., Iranmanesh, M., & Al-Qaysi, N. (2023). Generation Z use of artificial intelligence products and its impact on environmental sustainability: A cross-cultural comparison. Computers in Human Behavior, 143, Article 107708. https://doi.org/10.1016/j.chb.2023.107708
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.testingstandards.net/open-access-files.html
Bai, S., Hew, K. F., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30, Article 100322. https://doi.org/10.1016/j.edurev.2020.100322
Balabdaoui, F., Dittmann-Domenichini, N., Grosse, H., Schlienger, C., & Kortemeyer, G. (2024). A survey on students’ use of AI at a technical university. Discover Education, 3(51), Article 51. https://doi.org/10.1007/s44217-024-00136-4
Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117(3), 497–529. https://doi.org/10.1037/0033-2909.117.3.497
Bieletzke, S. (2023). Ai-chatbot-integration in campus-management-systems. EDULEARN23 Proceedings, 3574–3583. https://doi.org/10.21125/edulearn.2023.0971
Boehm, U., Marsman, M., Matzke, D., & Wagenmakers, E.-J. (2018). On the importance of avoiding shortcuts in applying cognitive models to hierarchical data. Behavior Research Methods, 50(4), 1614–1631. https://doi.org/10.3758/s13428-018-1054-3
Buder, J., Rabl, L., Feiks, M., Badermann, M., & Zurstiege, G. (2021). Does negatively toned language use on social media lead to attitude polarization? Computers in Human Behavior, 116, Article 106663. https://doi.org/10.1016/j.chb.2020.106663
Carneros-Prado, D., Villa, L., Johnson, E., Dobrescu, C. C., Barragán, A., & García-Martínez, B. (2023). Comparative study of large language models as emotion and sentiment analysis systems: A case-specific analysis of GPT vs. IBM Watson. In J. Bravo & G. Urzáiz, (Eds.), Proceedings of the 15th International Conference on Ubiquitous Computing & Ambient Intelligence (UCAmI 2023) (Vol. 842, pp. 229–239). Springer. https://doi.org/10.1007/978-3-031-48642-5_22
Castro-Alonso, J. C., Wong, R. M., Adesope, O. O., & Paas, F. (2021). Effectiveness of multimedia pedagogical agents predicted by diverse theories: A meta-analysis. Educational Psychology Review, 33, 989–1015. https://doi.org/10.1007/s10648-020-09587-1
Chen, Z., Zhang, R., Xu, T., Yang, Y., Wang, J., & Feng, T. (2020). Emotional attitudes towards procrastination in people: A large-scale sentiment-focused crawling analysis. Computers in Human Behavior, 110, Article 106391. https://doi.org/10.1016/j.chb.2020.106391
Ching, C. C., Basham, J. D., & Jang, E. (2005). The legacy of the digital divide: Gender, socioeconomic status, and early exposure as predictors of full-spectrum technology use among young adults. Urban Education, 40(4), 394–411. https://doi.org/10.1177/0042085905276389
Chiu, T. K. F. (2021). Digital support for student engagement in blended learning based on self-determination theory. Computers in Human Behavior, 124, Article 106909. https://doi.org/10.1016/j.chb.2021.106909
Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: the state of the field. International Journal of Educational Technology in Higher Education, 20(1), Article 22. https://doi.org/10.1186/s41239-023-00392-8
Dogan, M. E., Goru Dogan, T., & Bozkurt, A. (2023). The use of artificial intelligence (AI) in online learning and distance education processes: A systematic review of empirical studies. Applied Sciences, 13(5) Article 3056. https://doi.org/10.3390/app13053056
Driver, C. C., Oud, J. H. L., & Voelkle, M. C. (2017). Continuous time structural equation modeling with R package ctsem. Journal of Statistical Software, 77(5), 1–35. https://doi.org/10.18637/jss.v077.i05
Driver, C. C., & Voelkle, M. C. (2018). Hierarchical Bayesian continuous time dynamic modeling. Psychological Methods, 23, 774–799. https://doi.org/10.1037/met0000168
Duan, R., Li, D., Tong, Q., Yang, T., Liu, X., & Liu, X. (2021). A survey of few-shot learning: An effective method for intrusion detection. Security and Communication Networks, 2021, Article 4259629. https://doi.org/10.1155/2021/4259629
Eckert, M., Ebert, D. D., Lehr, D., Sieland, B., & Berking, M. (2016). Overcome procrastination: Enhancing emotion regulation skills reduce procrastination. Learning and Individual Differences, 52, 10–18. https://doi.org/10.1016/j.lindif.2016.10.001
Ge, Y., Guo, Y., Das, S., Al-Garadi, M. A., & Sarker, A. (2023). Few-shot learning for medical text: A review of advances, trends, and opportunities. Journal of Biomedical Informatics, 144, Article 104458. https://doi.org/10.1016/j.jbi.2023.104458
Gegenfurtner, A., Quesada‐Pallarès, C., & Knogler, M. (2014). Digital simulation‐based training: A meta‐analysis. British Journal of Educational Technology, 45(6), 1097–1114. https://doi.org/10.1111/bjet.12188
Gillath, O., Ai, T., Branicky, M. S., Keshmiri, S., Davison, R. B., & Spaulding, R. (2021). Attachment and trust in artificial intelligence. Computers in Human Behavior, 115, Article 106607. https://doi.org/10.1016/j.chb.2020.106607
Goel, A., Creeden, B., Kumble, M., Salunke, S., Shetty, A., & Wiltgen, B. (2015). Using Watson for enhancing human-computer co-creativity. In C. Dede, J. Richards, & B. Saxberg (Eds.), Proceedings of the 2015 AAAI Fall Symposium (No:2, pp. 22-29). Association for the Advancement of Artificial Intelligence. https://bit.ly/4kgvP4F
Goel, A., & Polepeddi, L. (2018). Jill Watson: A virtual teaching assistant for online education. In Learning engineering for online education (pp. 120–143). Routledge. https://doi.org/10.4324/9781351186193-7
Guzsvinecz, T., & Szűcs, J. (2023). Length and sentiment analysis of reviews about top-level video game genres on the steam platform. Computers in Human Behavior, 149, Article 107955. https://doi.org/10.1016/j.chb.2023.107955
Heidig, S., Müller, J., & Reichelt, M. (2015). Emotional design in multimedia learning: Differentiation on relevant design features and their effects on emotions and learning. Computers in Human Behavior, 44, 81–95. https://doi.org/10.1016/j.chb.2014.11.009
Julian, M. W. (2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling, 8(3), 325–352.
Kahu, E., Stephens, C., Leach, L., & Zepke, N. (2015). Linking academic emotions and student engagement: mature-aged distance students’ transition to university. Journal of Further and Higher Education, 39(4), 481–497. https://doi.org/10.1080/0309877X.2014.895305
Koenka, A. C. (2020). Academic motivation theories revisited: An interactive dialog between motivation scholars on recent contributions, underexplored issues, and future directions. Contemporary Educational Psychology, 61, Article 101831. https://doi.org/10.1016/j.cedpsych.2019.101831
Krath, J., Schürmann, L., & von Korflesch, H. F. O. (2021). Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games and game-based learning. Computers in Human Behavior, 125, Article 106963. https://doi.org/10.1016/j.chb.2021.106963
Langer, M., & Landers, R. N. (2021). The future of artificial intelligence at work: A review on effects of decision automation and augmentation on workers targeted by algorithms and third-party observers. Computers in Human Behavior, 123, Article 106878. https://doi.org/10.1016/j.chb.2021.106878
Leathwood, C., & Hey, V. (2009). Gender/ed discourses and emotional sub-texts: Theorising emotion in UK higher education. Teaching in Higher Education, 14(4), 429–440. https://doi.org/10.1080/13562510903050194
Lerche, T., & Kiel, E. (2018). Predicting student achievement in learning management systems by log data analysis. Computers in Human Behavior, 89, 367–372. https://doi.org/10.1016/j.chb.2018.06.015
Liang, J.-C., Hwang, G.-J., Chen, M.-R. A., & Darmawansah, D. (2023). Roles and research foci of artificial intelligence in language education: an integrated bibliographic analysis and systematic review approach. Interactive Learning Environments, 31(7), 4270–4296. https://doi.org/10.1080/10494820.2021.1958348
Lim, L., Bannert, M., van der Graaf, J., Singh, S., Fan, Y., Surendrannair, S., Rakovic, M., Molenaar, I., Moore, J., & Gašević, D. (2023). Effects of real-time analytics-based personalized scaffolds on students’ self-regulated learning. Computers in Human Behavior, 139, Article 107547. https://doi.org/10.1016/j.chb.2022.107547
Liu, B., & Zhang, L. (2012). A survey of opinion mining and sentiment analysis. In C. C. Aggarwal & C. Zhai (Eds.), Mining text data (pp. 415–463). Springer. https://doi.org/10.1007/978-1-4614-3223-4_13
Lv, X., Yang, Y., Qin, D., Cao, X., & Xu, H. (2022). Artificial intelligence service recovery: The role of empathic response in hospitality customers’ continuous usage intention. Computers in Human Behavior, 126, Article 106993. https://doi.org/10.1016/j.chb.2021.106993
Manasi, A., Panchanadeswaran, S., Sours, E., & Lee, S. J. (2022). Mirroring the bias: gender and artificial intelligence. Gender, Technology and Development, 26(3), 295–305. https://doi.org/10.1080/09718524.2022.2128254
Marciano, L., Driver, C. C., Schulz, P. J., & Camerini, A.-L. (2022). Dynamics of adolescents’ smartphone use and well-being are positive but ephemeral. Scientific Reports, 12, Article 1316. https://doi.org/10.1038/s41598-022-05291-y
Martı́nez-Miranda, J., & Aldea, A. (2005). Emotions in human and artificial intelligence. Computers in Human Behavior, 21(2), 323–341. https://doi.org/10.1016/j.chb.2004.02.010
Marx, E., Leonhardt, T., & Bergner, N. (2023). Secondary school students’ mental models and attitudes regarding artificial intelligence - A scoping review. Computers and Education: Artificial Intelligence, 5, Article 100169. https://doi.org/10.1016/j.caeai.2023.100169
Ng, Y.-L., & Lin, Z. (2022). Exploring conversation topics in conversational artificial intelligence–based social mediated communities of practice. Computers in Human Behavior, 134, Article 107326. https://doi.org/10.1016/j.chb.2022.107326
Nistor, N., Lerche, T., Weinberger, A., Ceobanu, C., & Heymann, O. (2014). Towards the integration of culture into the Unified Theory of Acceptance and Use of Technology. British Journal of Educational Technology, 45(1), 36–55. https://doi.org/10.1111/j.1467-8535.2012.01383.x
Pekrun, R. (2019). Inquiry on emotions in higher education: progress and open problems. Studies in Higher Education, 44(10), 1806–1811. https://doi.org/10.1080/03075079.2019.1665335
Pekrun, R., Elliot, A. J., & Maier, M. A. (2006). Achievement goals and discrete achievement emotions: A theoretical model and prospective test. Journal of Educational Psychology, 98(3), 583–597. https://doi.org/10.1037/0022-0663.98.3.583
Pentina, I., Xie, T., Hancock, T., & Bailey, A. (2023). Consumer–machine relationships in the age of artificial intelligence: Systematic literature review and research directions. Psychology & Marketing, 40(8), 1593–1614. https://doi.org/10.1002/mar.21853
Pychyl, T. A., & Sirois, F. M. (2016). Procrastination, emotion regulation, and well-being. In F. M. Sirois & T. A. Pychyl (Eds.), Procrastination, health, and well-being. (pp. 163–188). Elsevier Academic Press. https://doi.org/10.1016/B978-0-12-802862-9.00008-6
Ranger, J., & Kuhn, J.-T. (2017). Detecting unmotivated individuals with a new model-selection approach for Rasch models. Psychological Test and Assessment Modeling, 59(3), 269–295.
Ranger, J., Kuhn, J.-T., & Pohl, S. (2021). Effects of motivation on the accuracy and speed of responding in tests: The speed-accuracy tradeoff revisited. Measurement: Interdisciplinary Research and Perspectives, 19(1), 15–38. https://doi.org/10.1080/15366367.2020.1750934
R Development Core Team. (2021). R: A language and environment for statistical computing). R Foundation for Statistical Computing. https://www.R-project.org
Ribino, P. (2023). The role of politeness in human-machine interactions: a systematic literature review and future perspectives. Artificial Intelligence Review, 56, 445–482. https://doi.org/10.1007/s10462-023-10540-1
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68-78. https://doi.org/10.1037/0003-066X.55.1.68
Ryan, R. M., & Deci, E. L. (2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, Article 101860. https://doi.org/10.1016/j.cedpsych.2020.101860
Schuenemann, L., Scherenberg, V., von Salisch, M., & Eckert, M. (2022). “I’ll worry about it tomorrow”– Fostering emotion regulation skills to overcome procrastination. Frontiers in Psychology, 13, Article 780675. https://doi.org/10.3389/fpsyg.2022.780675
Shin, D., Zhong, B., & Biocca, F. A. (2020). Beyond user experience: What constitutes algorithmic experiences? International Journal of Information Management, 52, Article 102061. https://doi.org/10.1016/j.ijinfomgt.2019.102061
Stine, R. A. (2019). Sentiment analysis. Annual Review of Statistics and Its Application, 6, 287–308. https://doi.org/10.1146/annurev-statistics-030718-105242
Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55, 5731–5780. https://doi.org/10.1007/s10462-022-10144-1
Wolgast, A., Schmidt, N., & Ranger, J. (2020). Test-taking motivation in education students: Task battery order affected within-test-taker effort and importance. Frontiers in Psychology, 11, Article 3268. https://doi.org/10.3389/fpsyg.2020.559683
Xu, K. M., Cunha-Harvey, A. R., King, R. B., de Koning, B. B., Paas, F., Baars, M., Zhang, J., & de Groot, R. (2023). A cross-cultural investigation on perseverance, self-regulated learning, motivation, and achievement. Compare: A Journal of Comparative and International Education, 53(3), 361–379. https://doi.org/10.1080/03057925.2021.1922270
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), Article 39. https://doi.org/10.1186/s41239-019-0171-0
Zhoc, K. C. H., King, R. B., Chung, T. S. H., & Chen, J. (2020). Emotionally intelligent students are more engaged and successful: examining the role of emotional intelligence in higher education. European Journal of Psychology of Education, 35, 839–863. https://doi.org/10.1007/s10212-019-00458-0