logo logo European Journal of Psychology and Educational Research

EJPER is a leading, peer-reviewed research journal based in the Netherlands that provides an online forum for studies in educational psychology.

Subscribe to

Receive Email Alerts

for special events, calls for papers, and professional development opportunities.

Subscribe

Publisher (HQ)

Eurasian Society of Educational Research
Eurasian Society of Educational Research
Christiaan Huygensstraat 44, Zipcode:7533XB, Enschede, THE NETHERLANDS
Eurasian Society of Educational Research
Headquarters
Christiaan Huygensstraat 44, Zipcode:7533XB, Enschede, THE NETHERLANDS
Research Article

Student-Artificial Intelligence Conversations and Sentiments: A Process Data Analysis and Reciprocal Effects

Anett Wolgast , Stefan Bieletzke

Research on academic outcomes has extensively explored students’ emotions, motivation, and learning behavior. While further research highlights .


  • Pub. date: June 15, 2025
  • Pages: 69-82
  • 49 Downloads
  • 196 Views
  • 0 Citations

How to Cite

Abstract:

R

Research on academic outcomes has extensively explored students’ emotions, motivation, and learning behavior. While further research highlights the role of individual interactions with generative artificial intelligence (AI), a gap exists in understanding the longitudinal dynamic associations between university students' interactions with AI assistants and the sentiment expressed during these conversations over time. Based on a theoretical framework, we expected reciprocal effects between students’ interactions with AI assistants and sentiment within their conversation, considering students’ gender, age, and type of AI assistant used. Over four months, students at a university had access to three AI assistants: Alix for motivational-emotional support, Robyn for reflective discussions, and Dr. Melly for exam discussions. All AI assistants are integrated into a campus management system, ensuring compliance with data protection regulations. The sample for this process data analysis comprised n = 5,262 interactions of 422 students (56% female students, MAge = 30.42, SDAge = 9.08). Time series analysis with hierarchical Bayesian continuous time dynamic modeling revealed significant reciprocal effects between the students’ question length and the sentiment during these conversations: Students’ relatively short questions at the beginning of a conversation determined a later positive sentiment. A positive sentiment at the beginning of a conversation determined relatively short questions from the students later on, in particular from female students. Another result is that the type of AI assistance moderated the reciprocal effects. Thus, the students perceived the AI assistants as helpful tools for addressing questions during the day and nighttime in higher education.

Keywords: AI assistants, campus-management-system chats, hierarchical Bayesian continuous time dynamic modeling, higher education, sentiment analysis, university students.

description PDF
file_save XML
Article Metrics
Views
49
Download
196
Citations
Crossref
0

References

Al-Samarraie, H., Muthana Sarsam, S., & Alzahrani, A. I. (2023). Haptic technology in society: A sentiment analysis of public engagement. Computers in Human Behavior, 147, Article 107862. https://doi.org/10.1016/j.chb.2023.107862

Al-Sharafi, M. A., Al-Emran, M., Arpaci, I., Iahad, N. A., AlQudah, A. A., Iranmanesh, M., & Al-Qaysi, N. (2023). Generation Z use of artificial intelligence products and its impact on environmental sustainability: A cross-cultural comparison. Computers in Human Behavior, 143, Article 107708. https://doi.org/10.1016/j.chb.2023.107708

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.testingstandards.net/open-access-files.html

Bai, S., Hew, K. F., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30, Article 100322. https://doi.org/10.1016/j.edurev.2020.100322

Balabdaoui, F., Dittmann-Domenichini, N., Grosse, H., Schlienger, C., & Kortemeyer, G. (2024). A survey on students’ use of AI at a technical university. Discover Education, 3(51), Article 51. https://doi.org/10.1007/s44217-024-00136-4

Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117(3), 497–529. https://doi.org/10.1037/0033-2909.117.3.497

Bieletzke, S. (2023). Ai-chatbot-integration in campus-management-systems. EDULEARN23 Proceedings, 3574–3583. https://doi.org/10.21125/edulearn.2023.0971

Boehm, U., Marsman, M., Matzke, D., & Wagenmakers, E.-J. (2018). On the importance of avoiding shortcuts in applying cognitive models to hierarchical data. Behavior Research Methods, 50(4), 1614–1631. https://doi.org/10.3758/s13428-018-1054-3

Buder, J., Rabl, L., Feiks, M., Badermann, M., & Zurstiege, G. (2021). Does negatively toned language use on social media lead to attitude polarization? Computers in Human Behavior, 116, Article 106663. https://doi.org/10.1016/j.chb.2020.106663

Carneros-Prado, D., Villa, L., Johnson, E., Dobrescu, C. C., Barragán, A., & García-Martínez, B. (2023). Comparative study of large language models as emotion and sentiment analysis systems: A case-specific analysis of GPT vs. IBM Watson. In J. Bravo & G. Urzáiz, (Eds.), Proceedings of the 15th International Conference on Ubiquitous Computing & Ambient Intelligence (UCAmI 2023) (Vol. 842, pp. 229–239). Springer. https://doi.org/10.1007/978-3-031-48642-5_22

Castro-Alonso, J. C., Wong, R. M., Adesope, O. O., & Paas, F. (2021). Effectiveness of multimedia pedagogical agents predicted by diverse theories: A meta-analysis. Educational Psychology Review33, 989–1015. https://doi.org/10.1007/s10648-020-09587-1 

Chen, Z., Zhang, R., Xu, T., Yang, Y., Wang, J., & Feng, T. (2020). Emotional attitudes towards procrastination in people: A large-scale sentiment-focused crawling analysis. Computers in Human Behavior, 110, Article 106391. https://doi.org/10.1016/j.chb.2020.106391

Ching, C. C., Basham, J. D., & Jang, E. (2005). The legacy of the digital divide: Gender, socioeconomic status, and early exposure as predictors of full-spectrum technology use among young adults. Urban Education, 40(4), 394–411. https://doi.org/10.1177/0042085905276389

Chiu, T. K. F. (2021). Digital support for student engagement in blended learning based on self-determination theory. Computers in Human Behavior, 124, Article 106909. https://doi.org/10.1016/j.chb.2021.106909

Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: the state of the field. International Journal of Educational Technology in Higher Education, 20(1), Article 22. https://doi.org/10.1186/s41239-023-00392-8

Dogan, M. E., Goru Dogan, T., & Bozkurt, A. (2023). The use of artificial intelligence (AI) in online learning and distance education processes: A systematic review of empirical studies. Applied Sciences, 13(5) Article 3056. https://doi.org/10.3390/app13053056

Driver, C. C., Oud, J. H. L., & Voelkle, M. C. (2017). Continuous time structural equation modeling with R package ctsem. Journal of Statistical Software, 77(5), 1–35. https://doi.org/10.18637/jss.v077.i05

Driver, C. C., & Voelkle, M. C. (2018). Hierarchical Bayesian continuous time dynamic modeling. Psychological Methods, 23, 774–799. https://doi.org/10.1037/met0000168

Duan, R., Li, D., Tong, Q., Yang, T., Liu, X., & Liu, X. (2021). A survey of few-shot learning: An effective method for intrusion detection. Security and Communication Networks, 2021, Article 4259629. https://doi.org/10.1155/2021/4259629

Eckert, M., Ebert, D. D., Lehr, D., Sieland, B., & Berking, M. (2016). Overcome procrastination: Enhancing emotion regulation skills reduce procrastination. Learning and Individual Differences, 52, 10–18. https://doi.org/10.1016/j.lindif.2016.10.001

Ge, Y., Guo, Y., Das, S., Al-Garadi, M. A., & Sarker, A. (2023). Few-shot learning for medical text: A review of advances, trends, and opportunities. Journal of Biomedical Informatics, 144, Article 104458. https://doi.org/10.1016/j.jbi.2023.104458

Gegenfurtner, A., Quesada‐Pallarès, C., & Knogler, M. (2014). Digital simulation‐based training: A meta‐analysis. British Journal of Educational Technology45(6), 1097–1114. https://doi.org/10.1111/bjet.12188

Gillath, O., Ai, T., Branicky, M. S., Keshmiri, S., Davison, R. B., & Spaulding, R. (2021). Attachment and trust in artificial intelligence. Computers in Human Behavior115, Article 106607. https://doi.org/10.1016/j.chb.2020.106607

Goel, A., Creeden, B., Kumble, M., Salunke, S., Shetty, A., & Wiltgen, B. (2015). Using Watson for enhancing human-computer co-creativity. In C. Dede, J. Richards, & B. Saxberg (Eds.), Proceedings of the 2015 AAAI Fall Symposium (No:2, pp. 22-29). Association for the Advancement of Artificial Intelligence. https://bit.ly/4kgvP4F

Goel, A., & Polepeddi, L. (2018). Jill Watson: A virtual teaching assistant for online education. In Learning engineering for online education (pp. 120–143). Routledge. https://doi.org/10.4324/9781351186193-7

Guzsvinecz, T., & Szűcs, J. (2023). Length and sentiment analysis of reviews about top-level video game genres on the steam platform. Computers in Human Behavior, 149, Article 107955. https://doi.org/10.1016/j.chb.2023.107955

Heidig, S., Müller, J., & Reichelt, M. (2015). Emotional design in multimedia learning: Differentiation on relevant design features and their effects on emotions and learning. Computers in Human Behavior, 44, 81–95. https://doi.org/10.1016/j.chb.2014.11.009

Julian, M. W. (2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling, 8(3), 325–352.

Kahu, E., Stephens, C., Leach, L., & Zepke, N. (2015). Linking academic emotions and student engagement: mature-aged distance students’ transition to university. Journal of Further and Higher Education, 39(4), 481–497. https://doi.org/10.1080/0309877X.2014.895305

Koenka, A. C. (2020). Academic motivation theories revisited: An interactive dialog between motivation scholars on recent contributions, underexplored issues, and future directions. Contemporary Educational Psychology, 61, Article 101831. https://doi.org/10.1016/j.cedpsych.2019.101831

Krath, J., Schürmann, L., & von Korflesch, H. F. O. (2021). Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games and game-based learning. Computers in Human Behavior, 125, Article 106963. https://doi.org/10.1016/j.chb.2021.106963

Langer, M., & Landers, R. N. (2021). The future of artificial intelligence at work: A review on effects of decision automation and augmentation on workers targeted by algorithms and third-party observers. Computers in Human Behavior123, Article 106878. https://doi.org/10.1016/j.chb.2021.106878

Leathwood, C., & Hey, V. (2009). Gender/ed discourses and emotional sub-texts: Theorising emotion in UK higher education. Teaching in Higher Education, 14(4), 429–440. https://doi.org/10.1080/13562510903050194

Lerche, T., & Kiel, E. (2018). Predicting student achievement in learning management systems by log data analysis. Computers in Human Behavior, 89, 367–372. https://doi.org/10.1016/j.chb.2018.06.015

Liang, J.-C., Hwang, G.-J., Chen, M.-R. A., & Darmawansah, D. (2023). Roles and research foci of artificial intelligence in language education: an integrated bibliographic analysis and systematic review approach. Interactive Learning Environments, 31(7), 4270–4296. https://doi.org/10.1080/10494820.2021.1958348

Lim, L., Bannert, M., van der Graaf, J., Singh, S., Fan, Y., Surendrannair, S., Rakovic, M., Molenaar, I., Moore, J., & Gašević, D. (2023). Effects of real-time analytics-based personalized scaffolds on students’ self-regulated learning. Computers in Human Behavior, 139, Article 107547. https://doi.org/10.1016/j.chb.2022.107547

Liu, B., & Zhang, L. (2012). A survey of opinion mining and sentiment analysisIn C. C. Aggarwal & C. Zhai (Eds.), Mining text data (pp. 415–463). Springer. https://doi.org/10.1007/978-1-4614-3223-4_13

Lv, X., Yang, Y., Qin, D., Cao, X., & Xu, H. (2022). Artificial intelligence service recovery: The role of empathic response in hospitality customers’ continuous usage intention. Computers in Human Behavior, 126, Article 106993. https://doi.org/10.1016/j.chb.2021.106993

Manasi, A., Panchanadeswaran, S., Sours, E., & Lee, S. J. (2022). Mirroring the bias: gender and artificial intelligence. Gender, Technology and Development, 26(3), 295–305. https://doi.org/10.1080/09718524.2022.2128254

Marciano, L., Driver, C. C., Schulz, P. J., & Camerini, A.-L. (2022). Dynamics of adolescents’ smartphone use and well-being are positive but ephemeral. Scientific Reports, 12, Article 1316. https://doi.org/10.1038/s41598-022-05291-y

Martı́nez-Miranda, J., & Aldea, A. (2005). Emotions in human and artificial intelligence. Computers in Human Behavior, 21(2), 323–341. https://doi.org/10.1016/j.chb.2004.02.010

Marx, E., Leonhardt, T., & Bergner, N. (2023). Secondary school students’ mental models and attitudes regarding artificial intelligence - A scoping review. Computers and Education: Artificial Intelligence, 5, Article 100169. https://doi.org/10.1016/j.caeai.2023.100169

Ng, Y.-L., & Lin, Z. (2022). Exploring conversation topics in conversational artificial intelligence–based social mediated communities of practice. Computers in Human Behavior, 134, Article 107326. https://doi.org/10.1016/j.chb.2022.107326

Nistor, N., Lerche, T., Weinberger, A., Ceobanu, C., & Heymann, O. (2014). Towards the integration of culture into the Unified Theory of Acceptance and Use of Technology. British Journal of Educational Technology, 45(1), 36–55. https://doi.org/10.1111/j.1467-8535.2012.01383.x

Pekrun, R. (2019). Inquiry on emotions in higher education: progress and open problems. Studies in Higher Education, 44(10), 1806–1811. https://doi.org/10.1080/03075079.2019.1665335

Pekrun, R., Elliot, A. J., & Maier, M. A. (2006). Achievement goals and discrete achievement emotions: A theoretical model and prospective test. Journal of Educational Psychology, 98(3), 583–597. https://doi.org/10.1037/0022-0663.98.3.583

Pentina, I., Xie, T., Hancock, T., & Bailey, A. (2023). Consumer–machine relationships in the age of artificial intelligence: Systematic literature review and research directions. Psychology & Marketing, 40(8), 1593–1614. https://doi.org/10.1002/mar.21853

Pychyl, T. A., & Sirois, F. M. (2016). Procrastination, emotion regulation, and well-being. In F. M. Sirois & T. A. Pychyl (Eds.), Procrastination, health, and well-being. (pp. 163–188). Elsevier Academic Press. https://doi.org/10.1016/B978-0-12-802862-9.00008-6

Ranger, J., & Kuhn, J.-T. (2017). Detecting unmotivated individuals with a new model-selection approach for Rasch models. Psychological Test and Assessment Modeling, 59(3), 269–295.

Ranger, J., Kuhn, J.-T., & Pohl, S. (2021). Effects of motivation on the accuracy and speed of responding in tests: The speed-accuracy tradeoff revisited. Measurement: Interdisciplinary Research and Perspectives, 19(1), 15–38. https://doi.org/10.1080/15366367.2020.1750934

R Development Core Team. (2021). R: A language and environment for statistical computing). R Foundation for Statistical Computing. https://www.R-project.org

Ribino, P. (2023). The role of politeness in human-machine interactions: a systematic literature review and future perspectives. Artificial Intelligence Review, 56, 445–482. https://doi.org/10.1007/s10462-023-10540-1

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68-78. https://doi.org/10.1037/0003-066X.55.1.68

Ryan, R. M., & Deci, E. L. (2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, Article 101860. https://doi.org/10.1016/j.cedpsych.2020.101860

Schuenemann, L., Scherenberg, V., von Salisch, M., & Eckert, M. (2022). “I’ll worry about it tomorrow”– Fostering emotion regulation skills to overcome procrastination. Frontiers in Psychology, 13, Article 780675. https://doi.org/10.3389/fpsyg.2022.780675

Shin, D., Zhong, B., & Biocca, F. A. (2020). Beyond user experience: What constitutes algorithmic experiences? International Journal of Information Management, 52, Article 102061. https://doi.org/10.1016/j.ijinfomgt.2019.102061

Stine, R. A. (2019). Sentiment analysis. Annual Review of Statistics and Its Application, 6, 287–308. https://doi.org/10.1146/annurev-statistics-030718-105242

Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55, 5731–5780. https://doi.org/10.1007/s10462-022-10144-1

Wolgast, A., Schmidt, N., & Ranger, J. (2020). Test-taking motivation in education students: Task battery order affected within-test-taker effort and importance. Frontiers in Psychology, 11, Article 3268. https://doi.org/10.3389/fpsyg.2020.559683

Xu, K. M., Cunha-Harvey, A. R., King, R. B., de Koning, B. B., Paas, F., Baars, M., Zhang, J., & de Groot, R. (2023). A cross-cultural investigation on perseverance, self-regulated learning, motivation, and achievement. Compare: A Journal of Comparative and International Education, 53(3), 361–379. https://doi.org/10.1080/03057925.2021.1922270

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), Article 39. https://doi.org/10.1186/s41239-019-0171-0

Zhoc, K. C. H., King, R. B., Chung, T. S. H., & Chen, J. (2020). Emotionally intelligent students are more engaged and successful: examining the role of emotional intelligence in higher education. European Journal of Psychology of Education, 35, 839–863. https://doi.org/10.1007/s10212-019-00458-0

...