Enhancing Assessment and Feedback in Game Design Programs: Leveraging Generative AI for Efficient and Meaningful Evaluation

James Hutson; Ben Fulcher; Jeremiah Ratican

doi:10.46661/ijeri.11038

Autores/as

James Hutson Lindenwood University https://orcid.org/0000-0002-0578-6052
Ben Fulcher Lindenwood University https://orcid.org/0009-0005-4338-6067
Jeremiah Ratican Lindenwood University

DOI:

https://doi.org/10.46661/ijeri.11038

Palabras clave:

IA Generativa, Educación en Diseño de Videojuegos, Evaluación Automatizada, Eficiencia en la Retroalimentación, Compromiso Estudiantil

Resumen

La integración de herramientas de IA generativa en la educación de diseño de videojuegos ofrece formas prometedoras de optimizar los procesos de calificación, evaluación y retroalimentación, que suelen ser intensivos en mano de obra. En los programas de diseño de videojuegos, el profesorado a menudo maneja formatos de archivo variados, incluidos modelos 3D, prototipos ejecutables, videos y documentos complejos de diseño de juegos. Los métodos tradicionales de evaluación y retroalimentación, principalmente basados en texto, tienen dificultades para proporcionar a los estudiantes ideas oportunas y accionables. Además, solo un pequeño porcentaje de los estudiantes más destacados revisa y aplica de manera constante la retroalimentación, lo que genera ineficiencias. Este artículo explora cómo las herramientas de IA generativa pueden mejorar estos procesos mediante la automatización de aspectos de la calificación, la generación de retroalimentación más personalizada y significativa, y la resolución de la naturaleza intensiva en tiempo de la revisión de formatos de archivo diversos. Se discuten estrategias clave, incluido el uso de rúbricas adaptadas para la evaluación basada en IA, indicaciones automatizadas para asignaciones impulsadas por la narrativa y la aplicación de la IA en la revisión de construcciones de proyectos complejos. El objetivo es crear más tiempo para que el profesorado se involucre en la mentoría en vivo y actividades de aprendizaje práctico, que la investigación demuestra ser más efectivas. Se proporcionan ejemplos prácticos de diversas tareas de diseño de videojuegos, incluidas revisiones de construcciones y evaluaciones de documentos, para ilustrar estos nuevos enfoques. Este cambio promete mejorar el compromiso estudiantil y mejorar los resultados de aprendizaje.

Descargas

Los datos de descargas todavía no están disponibles.

Citas

Adams, E. (2014). Fundamentals of game design. Pearson Education.

Alam, A. (2023). Developing a Curriculum for Ethical and Responsible AI: A University Course on Safety, Fairness, Privacy, and Ethics to Prepare Next Generation of AI Professionals. In Intelligent Communication Technologies and Virtual Mobile Networks (pp. 879-894). Springer Nature Singapore. https://doi.org/10.1007/978-981-99-1767-9_64

Anjum, A., Li, Y., Law, N., Charity, M., & Togelius, J. (2024, May). The Ink Splotch Effect: A case study on ChatGPT as a co-creative game designer. In Proceedings of the 19th International Conference on the Foundations of Digital Games (pp. 1-15). https://doi.org/10.1145/3649921.3650010

Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in education: principles, policy & practice, 18(1), 5-25. https://doi.org/10.1080/0969594X.2010.513678

Black, R.W. (2008). Adolescents and online fan fiction. Peter Lang.

Brunnberg, J. (2020). How tools shape the game authoring process. https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1449483&dswid=2939

Burke, K. (2010). From standards to rubrics in six steps: Tools for assessing student learning. Corwin Press.

Chen, Y., Clayton, E. W., Novak, L. L., Anders, S., & Malin, B. (2023). Human-centered design to address biases in artificial intelligence. Journal of medical Internet research, 25, e43251. https://doi.org/10.2196/43251

Couper, J. (2018). The struggle is real: Investigating the challenge of assigning a failing clinical grade. Nursing Education Perspectives, 39(3), 132-138. https://doi.org/10.1097/01.NEP.0000000000000295

Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: the state of the field. International Journal of Educational Technology in Higher Education, 20(1), 22. https://doi.org/10.1186/s41239-023-00392-8

Dastyar, S. (2019). The Investigation Of The Effectiveness of Participatory Learning Education On Students Motivation And Academic Achievement. International Journal of Advanced Research and Publications, 3 (8), 165-170.

Easterday, M. W., Rees Lewis, D., & Gerber, E. M. (2017). Designing crowdcritique systems for formative feedback. International Journal of Artificial Intelligence in Education, 27, 623-663. https://doi.org/10.1007/s40593-016-0125-9

Ewell, P. T., Paulson, K., & Kinzie, J. (2011). Down and in: Assessment practices at the program level. National Institute for Learning Outcomes Assessment. Retrieved from http://www.learningoutcomesassessment.org

Exterman, D. (2021). The Gamification of Everything. DevOps.com. https://devops.com/the-gamification-of-everything/

Ferrara, E. (2023). Fairness and Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, and Mitigation Strategies. Sci, 6, 3. https://doi.org/10.3390/sci6010003

Fitch, S. (2023). Art, assessment and uncertainty. Art, Design & Communication in Higher Education, 22(2), 263-275. https://doi.org/10.1386/adch_00077_1

Gui, Y., Cai, Z., Yang, Y., Kong, L., Fan, X., & Tai, R. H. (2023). Effectiveness of digital educational game and game design in STEM learning: a meta-analytic review. International Journal of STEM Education, 10(1), 36. https://doi.org/10.1186/s40594-023-00424-9

Hall, P., & Ellis, D. (2023). A systematic review of socio-technical gender bias in AI algorithms. Online Information Review, 47 (7), 1264-1279. https://doi.org/10.1108/OIR-08-2021-0452

Heil, J., & Ifenthaler, D. (2023). Online Assessment in Higher Education: A Systematic Review. Online Learning, 27(1), 187-218. https://doi.org/10.24059/olj.v27i1.3398

Hutson, J., & Plate, D. (2023). Enhancing institutional assessment and reporting through conversational technologies: exploring the potential of AI-powered tools and natural language processing. Journal of Artificial Intelligence and Robotics, 1(1), 11-22. https://doi.org/10.59232/AIR-V1I1P102

Islam, S. R., Russell, I., Eberle, W., & Dicheva, D. (2022, March). Incorporating the Concepts of Fairness and Bias into an Undergraduate Computer Science Course to Promote Fair Automated Decision Systems. In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 2 (pp. 1075-1075). https://doi.org/10.1145/3478432.3499043

Jacobs, M., Kurtz, C., Simon, J., & Böhmann, T. (2021). Value sensitive design and power in socio-technical ecosystems. Internet Policy Review, 10(3), 1-26. https://doi.org/10.14763/2021.3.1580

Jenkins, H. (2006). Convergence culture. University Press.

Jin, X., Jiang, Q., Xiong, W., Feng, Y., & Zhao, W. (2024). Effects of student engagement in peer feedback on writing performance in higher education. Interactive Learning Environments, 32(1), 128-143. https://doi.org/10.1080/10494820.2022.2081209

Kapp, Karl (2012). The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. Pfeiffer. https://doi.org/10.1145/2207270.2211316

Kordzadeh, N., & Ghasemaghaei, M. (2022). Algorithmic bias: review, synthesis, and future research directions. European Journal of Information Systems, 31(3), 388-409. https://doi.org/10.1080/0960085X.2021.1927212

Kramarzewski, A., & De Nucci, E. (2023). Practical Game Design: A modern and comprehensive guide to video game design. Packt Publishing Ltd.

Kritek, P. A. (2015). Strategies for effective feedback. Annals of the American Thoracic Society, 12(4), 557-560. https://doi.org/10.1513/AnnalsATS.201411-524FR

Leavy, S. (2018, May). Gender bias in artificial intelligence: The need for diversity and gender theory in machine learning. In Proceedings of the 1st international workshop on gender equality in software engineering (pp. 14-16). https://doi.org/10.1145/3195570.3195580

Levin, Douglas & Arafeh, Sousan. (2003). The Digital Disconnect: The Widening Gap Between Internet-Savvy Students and Their Schools. Internet and American Life. ii-30.

Liu, H., Choi, M., Kao, D., & Mousas, C. (2023). Synthesizing game levels for collaborative gameplay in a shared virtual environment. ACM Transactions on Interactive Intelligent Systems, 13(1), 1-36. https://doi.org/10.1145/3558773

Mashhadi, A., Zolyomi, A., & Quedado, J. (2022, April). A Case Study of Integrating Fairness Visualization Tools in Machine Learning Education. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (pp. 1-7). https://doi.org/10.1145/3491101.3503568

McGinness, H. T., Caldwell, P. H., Gunasekera, H., & Scott, K. M. (2020). An educational intervention to increase student engagement in feedback. Medical Teacher, 42(11), 1289-1297. https://doi.org/10.1080/0142159X.2020.1804055

Moreno-Ger, P., Burgos, D., Martínez-Ortiz, I., Sierra, J. L., & Fernández-Manjón, B. (2008). Educational game design for online education. Computers in Human Behavior, 24(6), 2530-2540. https://doi.org/10.1016/j.chb.2008.03.012

Morton, J. K., Northcote, M., Kilgour, P., & Jackson, W. A. (2021). Sharing the construction of assessment rubrics with students: A Model for collaborative rubric construction. Journal of University Teaching & Learning Practice, 18(4). https://doi.org/10.53761/1.18.4.9

Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M. E., ... & Staab, S. (2020). Bias in data‐driven artificial intelligence systems—An introductory survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(3), e1356. https://doi.org/10.1002/widm.1356

Petrova, E. (2020). Checklists and Rubrics: Do They Really Promote Learning? In ICERI2020 Proceedings (pp. 4906-4911). IATED. https://doi.org/10.21125/iceri.2020.1069

Seering, J., Mayol, R., Harpstead, E., Chen, T., Cook, A., & Hammer, J. (2019, October). Peer feedback processes in the game industry. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (pp. 427-438). https://doi.org/10.1145/3311350.3347176

Serrano-Laguna, Á., Manero, B., Freire, M., & Fernández-Manjón, B. (2018). A methodology for assessing the effectiveness of serious games and for inferring player learning outcomes. Multimedia Tools and applications, 77, 2849-2871. https://doi.org/10.1007/s11042-017-4467-6

Sezen, T. I. (2024). Analog prototyping for digital game design. In Encyclopedia of Computer Graphics and Games (pp. 102-104). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-23161-2_135

Squire, Kurt (2011). Video Games and Learning: Teaching and Participatory Culture in the Digital Age. Columbia University.

Stãnescu, I. A., Stefan, A., Kravcik, M., Lim, T., & Bidarra, R. (2013). Interoperability strategies for serious games development. Internet Learning, 2(1), 373-378. https://doi.org/10.18278/il.2.1.4

Stevens, D. D., & Levi, A. J. (2023). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning. Routledge.

Sudhakaran, S., González-Duque, M., Glanois, C., Freiberger, M., Najarro, E., & Risi, S. (2023, July). Prompt-guided level generation. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation (pp. 179-182). https://doi.org/10.1145/3583133.3590656

Srivastava, O., Tennant, M., Grewal, P., Rubin, U., & Seamone, M. (2023). Artificial intelligence and machine learning in ophthalmology: A review. Indian Journal of Ophthalmology, 71(1), 11-17. https://doi.org/10.4103/ijo.IJO_1569_22

Tang, L. M., & Kay, J. (2014, July). Gamification: metacognitive scaffolding towards long term goals? In UMAP workshops (pp. 7-11).

Udeozor, C., Chan, P., Russo Abegão, F., & Glassey, J. (2023). Game-based assessment framework for virtual reality, augmented reality and digital game-based learning. International Journal of Educational Technology in Higher Education, 20 (36). https://doi.org/10.1186/s41239-023-00405-6

Umbrello, S. (2020). Combinatory and complementary practices of values and virtues in design: A reply to Reijers and Gordijn. Filosofia, (65), 107-121.

Wang, T., Lund, B. D., Marengo, A., Pagano, A., Mannuru, N. R., Teel, Z. A., & Pange, J. (2023). Exploring the potential impact of artificial intelligence (AI) on international students in higher education: Generative AI, chatbots, analytics, and international student success. Applied Sciences, 13(11), 6716. https://doi.org/10.3390/app13116716

Winstone, N., & Carless, D. (2019). Designing effective feedback processes in higher education: A learning-focused approach. Routledge. https://doi.org/10.4324/9781351115940

Winstone, N. E., & Boud, D. (2022). The need to disentangle assessment and feedback in higher education. Studies in higher education, 47(3), 656-667. https://doi.org/10.1080/03075079.2020.1779687

Wolf, K. D., Maya, F., & Heilmann, L. (2023). Explainable Feedback for Learning Based on Rubric-Based Multimodal Assessment Analytics with AI. https://aisop.de/EPEPLA/Feedback.pdf

Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are we there yet? -A systematic literature review on chatbots in education. Frontiers in artificial intelligence, 4, https://doi.org/10.3389/frai.2021.654924

Wolz, U., Ault, C., & Nakra, T. M. (2007). Teaching game design through cross-disciplinary content and individualized student deliverables. Journal of Game Development, 2(2), 19-32.

Zarkoob, H. (2024). AI-powered methods for academic assessment: overcoming scalability challenges in large university classrooms and conference review. [Doctoral dissertation], University of British Columbia.

Zouhaier, S. (2023). The impact of Artificial intelligence on higher education: An empirical study. European Journal of Educational Sciences, 10(1), 17-33. https://doi.org/10.19044/ejes.v10no1a17

Zytko, D., J. Wisniewski, P., Guha, S., PS Baumer, E., & Lee, M. K. (2022, April). Participatory design of AI systems: opportunities and challenges across diverse users, relationships, and application domains. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (pp. 1-4). https://doi.org/10.1145/3491101.3516506