An intuitive number conversion system employing gestural recognition for students with special educational needs in Computer Science

Renjith P. R.; Jeeva Jose

doi:10.46661/ijeri.10880

Authors

Renjith P. R. Department of General Education - Higher Secondary https://orcid.org/0009-0001-8107-8744
Jeeva Jose Department of Computer Applications, Baselios Paulose II Catholicos College https://orcid.org/0000-0002-6919-1381

DOI:

https://doi.org/10.46661/ijeri.10880

Keywords:

Artificial Intelligence, differently abled, hand gesture recognition, inclusive learning, number conversion

Abstract

Duty of a teacher is to effectively engage the different types of students in the classroom into the teaching-learning environment. Each class has a variety of students varying in intelligence, communication, understanding levels, problem solving skills, learning and mathematical abilities. It is the need of the hour to make the differently abled learners with learning or intellectual disabilities to compete with regular students to avoid a social divide in class. Computer science curriculum has many topics/areas that pose learning difficulty for the differently abled. The problems faced by such learners were not taken care of sufficiently, which leaves them behind in the learning process. The paper proposes a hand gesture method, implemented using AI for number conversions, which can be used by teachers and differently abled learners. The efficiency of the hand gesture tool to understand hand gestures is tested for various classifiers and an average accuracy of 96.97% is obtained across various number systems. The hand gesture based method is evaluated among learners and proved efficient in retention and recovery of concepts. Learners showed greater improvement from pre-test to post-test scores across all categories. This method proved very helpful in engaging students, enhancing understanding, enjoyable learning, notable clarity, effectiveness, and ease of use, making it a valuable educational tool.

Downloads

Download data is not yet available.

References

Alibali, M. W., & DiRusso, A. A. (1999). The function of gesture in learning to count: More than keeping track. Cognitive Development, 14(1), 37–56. https://doi.org/10.1016/S0885-2014(99)80017-3

Alibali, M. W., & Nathan, M. J. (2012). Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. Journal of Learning Science, 21(2), 247–286. https://doi.org/10.1080/10508406.2011.611446

Chang V, Eniola RO, Golightly L, Xu QA. An Exploration into Human-Computer Interaction: Hand Gesture Recognition Management in a Challenging Environment. SN Comput Sci. 2023, 4(5), 441. https://doi.org/10.1007/s42979-023-01751-y

Chang, Y. J., Chen, S. F., & Chuang, A. F. (2011). A gesture recognition system to transition autonomously through vocational tasks for individuals with cognitive impairments. Research in Developmental Disabilities, 32(6), 2064–2068. https://doi.org/10.1016/j.ridd.2011.08.010

Cook, S. W., Yip, T. K., & Goldin-Meadow, S. (2010). Gesturing makes memories that last. Journal of Memory and Language, 63(4), 465–475. https://doi.org/10.1016/j.jml.2010.07.002

De Koning, B. B., & Tabbers, H. K. (2013). Gestures in instructional animations: A helping hand to understanding non-human movements? Applied Cognitive Psychology, 27(5), 683–689. https://doi.org/10.1002/acp.2937

Hashiyana, V., Veiko, V., Suresh, N., Ujakpa, M.M. & Sverdlik, W. (2022). Assistive Technology for Children with Learning Disabilities as an Artificial Intelligence Application. In Assistive Technologies for Differently Abled Students, 1st ed. (113-125). IGI Global. https://doi.org/10.4018/978-1-7998-4736-6.ch007

Johnson-Glenberg, M. C., & Megowan-Romanowicz, C. (2017). Embodied science and mixed reality: How gesture and motion capture affect physics education. Cognitive Research: Principles and Implications, 2(1), 24. https://doi.org/10.1186/s41235-017-0060-9

Ladner, R.E. (2011). Accessible Technology and Models of Disability. In: Oishi, M., Mitchell, I., Van der Loos, H. (eds) Design and Use of Assistive Technology. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7031-2_3

Lee, W., Huang, C., Wu, C., Huang, S., & Chen, G. (2012). The effects of using embodied interactions to improve learning performance. Advanced Learning Technologies (ICALT), 2012 IEEE 12th International Conference, 4–6 July 2012, 557–559. https://doi.org/10.1109/ICALT.2012.104

Merkouris, A., Chorianopoulou, B., Chorianopoulos, K., & Chrissikopoulos, V. (2019). Understanding the Notion of Friction Through Gestural Interaction with a Remotely Controlled Robot. Journal of Science Education and Technology, 28(3), 209–221. https://doi.org/10.1007/s10956-018-9760-2

Micó-Amigo, E., & Bernal Bravo, C. (2020). Evaluative research on teaching innovation with simulators in the area of Technology in Compulsory Secondary Education. IJERI: International Journal of Educational Research and Innovation, (14), 134–146. https://doi.org/10.46661/ijeri.4855.

Moral-Sánchez, S. N., Ruiz Rey, F. J., & Cebrián-de-la-Serna, M. (2023). Analysis of artificial intelligence chatbots and satisfaction for learning in mathematics education. IJERI: International Journal of Educational Research and Innovation, (20), 1–14. https://doi.org/10.46661/ijeri.8196.

Román-Graván, P., & Fernández-Cerero, J. (2022). Training of university teachers in ICT and disability. The case of the University of Seville. IJERI: International Journal of Educational Research and Innovation, (17), 135–151. https://doi.org/10.46661/ijeri.6168.

Xu, X., & Ke, F. (2021). Learning Number Conversions through Embodied Interactions, Springer Journal of Technology, Knowledge and Learning 28(1), 1-5. https://doi.org/10.1007/s10758-021-09557-8