DETEKSI EKSPRESI AFEKTIF MAHASISWA PADA SISTEM PEMBELAJARAN PEMROGRAMAN MENGGUNAKAN YOLOV13N
DOI:
https://doi.org/10.36080/skanika.v9i2.3895Keywords:
Affective Expressions, Facial Emotion Recognition, YOLOv13n, Learning Analytics, Programming LearningAbstract
Programming learning requires repeated problem-solving processes that may influence students’ affective states. However, these changes are difficult to continuously observe through manual observation or self-reporting. This study develops a YOLOv13n-based Facial Emotion Recognition system to detect four affective states—engagement, confusion, frustration, and boredom—in real time and integrate the detected results with learning activities in a web-based application. Detection is performed while students read learning modules and complete assessments through multiple-choice quizzes or interactive coding exercises. The model was fine-tuned using a combined dataset of 1,660 images, consisting of 953 images from Roboflow Universe and 707 hard samples, with stratified training, validation, and testing splits of 80:10:10. Evaluation on 173 test images achieved a precision of 0.994, recall of 0.982, F1-score of 0.988, mAP@0.5 of 0.994, and mAP@0.5:0.95 of 0.982, with an inference time of 6.2 ms per image. The application of confidence filtering, sliding window, and majority voting reduced label changes by 77.20%, improving temporal stability. Black-box testing across 27 scenarios confirmed that all application functions operated as designed. The system provides descriptive indicators rather than psychological diagnosis, demonstrating potential for monitoring affective expression patterns during programming learning.
Downloads
References
[1] Z. Atiq and R. Batra, “How Do First-Year Engineering Students’ Emotions Change while Working on Programming Problems?,” ACM Transactions on Computing Education, vol. 24, no. 2, pp. 1-30, 2024, doi: 10.1145/3643865.
[2] R. Pekrun, “Control-Value Theory: From Achievement Emotion to a General Theory of Human Emotions,” Educ. Psychol. Rev., vol. 36, no. 3, pp. 1-36, 2024, doi: 10.1007/s10648-024-09909-7.
[3] X. Tang, et al., “Facial Expression Recognition for Probing Students’ Emotional Engagement in Science Learning,” J. Sci. Educ. Technol., vol. 34, no. 1, pp. 13–30, 2025, doi: 10.1007/s10956-024-10143-7.
[4] X. Hu and J. Gao, “Facial expression recognition reveals students’ engagement in online class: Correlations with six engagement measurements,” PLoS One, vol. 20, no. 10, 2025, doi: 10.1371/journal.pone.0334232.
[5] K. Mangaroska, K. Sharma, D. Gašević, and M. Giannakos, “Exploring students’ cognitive and affective states during problem solving through multimodal data: Lessons learned from a programming activity,” J. Comput. Assist. Learn., vol. 38, no. 1, pp. 40–59, 2022, doi: 10.1111/jcal.12590.
[6] L. Yan, X. Wu, and Y. Wang, “Student engagement assessment using multimodal deep learning,” PLoS One, vol. 20, no. 6, pp. e0325377, 2025, [Online]. Available: 10.1371/journal.pone.0325377.
[7] A. Gupta, A. D’Cunha, K. Awasthi, and V. Balasubramanian, “DAiSEE: Towards User Engagement Recognition in the Wild,” vol. 14, no. 8, pp. 1-12, 2022, doi: 10.48550/arXiv.1609.01885.
[8] Y. Xu, Y. Li, Y. Chen, H. Bao, and Y. Zheng, “Spontaneous visual database for detecting learning-centered emotions during online learning,” Image Vis. Comput., vol. 136, p. 104739, 2023, doi: 10.1016/j.imavis.2023.104739.
[9] E. B. Cloude, et al., “Exploring Confusion and Frustration as Non-linear Dynamical Systems,” in ACM International Conference Proceeding Series, Association for Computing Machinery, 2024, pp. 241–252. doi: 10.1145/3636555.3636875.
[10] A. K. Ulandari, F. Bimantoro, and I. G. P. S. Wijaya, “Real Time Student Emotion Detection using Yolov5,” Edumatic: Jurnal Pendidikan Informatika, vol. 8, no. 1, pp. 222–231, 2024, doi: 10.29408/edumatic.v8i1.25726.
[11] D. Tresnawati, S. Nurhidayanti, and N. Lestari, “A Comparison of YOLOv8 Series Performance in Student Facial Expressions Detection on Online Learning,” Jurnal Online Informatika, vol. 10, no. 1, pp. 93–104, 2025, doi: 10.15575/join.v10i1.1390.
[12] A. I. Aulia, A. J. Hutapea, A. M. Siregar, and Surjandy, “Lightweight YOLO Models for Robust Facial Expression Detection,” Jurnal Teknologi Informasi dan Pendidikan, vol. 19, no. 2, 2026, doi: 10.24036/jtip.v19i2.1120.
[13] M. Lei et al., “YOLOv13: Real-Time Object Detection with Hypergraph-Enhanced Adaptive Visual Perception,” pp. 1-12, 2025, doi: 10.48550/arXiv.2506.17733.
[14] A. S. Kusuma, A. I. Pradana, and B. W. Pamekas, “Pengembangan Sistem Perhitungan Jumlah Kendaraan Berdasarkan Jenis Kendaraan Menggunakan Algoritma Yolo Secara Realtime,” SKANIKA: Sistem Komputer dan Teknik Informatika, vol. 7, no. 2, pp. 166–179, 2024, doi: 10.36080/skanika.v7i2.3201.
[15] M. N. Hasnine, et al., “A Real-Time Learning Analytics Dashboard for Automatic Detection of Online Learners’ Affective States,” Sensors, vol. 23, no. 9, pp. 1-15, 2023, doi: 10.3390/s23094243.
[16] A. Harb et al., “Diverse distant-students deep emotion recognition and visualization,” Computers and Electrical Engineering, vol. 111, 2023, doi: 10.1016/j.compeleceng.2023.108963.
[17] A. B. Amjoud and M. Amrouch, “Object Detection Using Deep Learning, CNNs and Vision Transformers: A Review,” IEEE Access, vol. 11, pp. 35479–35516, 2023, doi: 10.1109/ACCESS.2023.3266093.
[18] Y. ELsayed, A. ELSayed, and M. A. Abdou, “An automatic improved facial expression recognition for masked faces,” Neural Comput. Appl., vol. 35, no. 20, pp. 14963–14972, 2023, doi: 10.1007/s00521-023-08498-w.
[19] I. Dominguez-Catena, D. Paternain, and M. Galar, “Metrics for Dataset Demographic Bias: A Case Study on Facial Expression Recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 8, pp. 5209–5226, 2024, doi: 10.1109/TPAMI.2024.3361979.
[20] M. Lukac, G. Zhambulova, K. Abdiyeva, and M. Lewis, “Study on emotion recognition bias in different regional groups,” Sci. Rep., vol. 13, no. 1, 2023, doi: 10.1038/s41598-023-34932-z.
[21] L. D. Vreugd, A. V. Leeuwen, R. Jansen, and M. V. D. Schaaf, “Learning Analytics Dashboard Design and Evaluation to Support Student Self-Regulation of Study Behaviour,” Journal of Learning Analytics, vol. 11, no. 3, pp. 249–262, 2024, doi: 10.18608/jla.2024.8529.
[22] L. Paulsen and E. Lindsay, “Learning analytics dashboards are increasingly becoming about learning and not just analytics - A systematic review,” Educ. Inf. Technol. (Dordr)., vol. 29, no. 11, pp. 14279–14308, 2024, doi: 10.1007/s10639-023-12401-4.
[23] A. Kaveri, A. Silvola, and H. Muukkonen, “Supporting Student Agency with a Student-Facing Learning Analytics Dashboard: Perceptions of an Interdisciplinary Development Team,” Journal of Learning Analytics, vol. 10, no. 2, pp. 85–99, 2023, doi: 10.18608/jla.2023.7729.
[24] H. Bichri, A. Chergui, and M. Hain, “Investigating the Impact of Train / Test Split Ratio on the Performance of Pre-Trained Models with Custom Datasets,” International Journal of Advanced Computer Science and Applications (IJACSA), vol. 15, no. 2, pp. 331–339, 2024, doi: 10.14569/IJACSA.2024.0150235.
[25] Z. Wang, A. Li, Z. Li, and Q. Dou, “Efficient Transferability Assessment for Selection of Pre-Trained Detectors,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024, pp. 1679–1689, doi: 10.1109/WACV57701.2024.00170.
[26] A. Psaroudakis and D. Kollias, “MixAugment & Mixup: Augmentation Methods for Facial Expression Recognition,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2022, pp. 2366–2374, doi: 10.1109/CVPRW56347.2022.00264.
[27] D. Felicio, J. Simao, and N. Datia, “Rapitest: Continuous black-box testing of restful web apis,” in Procedia Computer Science, Elsevier B.V., 2023, pp. 537–545. doi: 10.1016/j.procs.2023.01.322.
[28] C. Herodotou et al., “A Participatory Approach to Designing a Student-Facing Dashboard for Online and Distance Education,” Journal of Learning Analytics, vol. 12, no. 2, pp. 158–174, 2025, doi: 10.18608/jla.2025.8481.
[29] A. L. C. Barczak, A. Mathrani, B. Han, and N. H. Reyes, “Automated assessment system for programming courses: a case study for teaching data structures and algorithms,” Educational Technology Research and Development, vol. 71, no. 6, pp. 2365–2388, 2023, doi: 10.1007/s11423-023-10277-2.
[30] P. Utamachant, C. Anutariya, and S. Pongnumkul, “i-Ntervene: applying an evidence-based learning analytics intervention to support computer programming instruction,” Smart Learning Environments, vol. 10, no. 1, pp. 1-30, 2023, doi: 10.1186/s40561-023-00257-7.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Verell Hermawan, Sawali Wahyu
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
CC BY-SA 4.0
Creative Commons Attribution-ShareAlike 4.0 International
This license requires that reusers give credit to the creator. It allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, even for commercial purposes. If others remix, adapt, or build upon the material, they must license the modified material under identical terms.
BY: Credit must be given to you, the creator.
SA: Adaptations must be shared under the same terms.ng
