Usabilidad de un recurso de realidad virtual para la enseñanza de osciloscopios: valoración de expertos
Usability of a virtual reality resource for oscilloscope teaching: expert assessment
Jeanette Chaljub-Hasbún
Instituto Tecnológico de Santo Domingo (INTEC), Santo Domingo, República Dominicana
https://orcid.org/0000-0001-6894-4719
Julio Cabero Almenara
Universidad de Sevilla
https://orcid.org/0000-0002-1133-6031
Pamela Michel-Acosta
Instituto Tecnológico de Santo Domingo (INTEC), Santo Domingo, República Dominicana
https://orcid.org/0000-0001-6198-0020
Rafael Bello
Universidad Católica de Santo Domingo (UCSD), Santo Domingo, República Dominicana
https://orcid.org/0000-0002-2572-4414
Josefina Pepín
Universidad Nacional Pedro Henríquez Ureña (UNPHU), Santo Domingo, República Dominicana
https://orcid.org/0009-0008-8758-8754
RESUMEN
La realidad virtual forma parte del conjunto de tecnologías hápticas que permiten al usuario interactuar con un escenario artificial, distintos al mundo real, pero muy parecido a él. Dentro del campo de la educación, promueve aprendizajes activos y dinámicos, con experiencias inmersivas. El objetivo de este estudio fue evaluar la usabilidad de un objeto en formato de RV para la enseñanza del osciloscopio. Se creó un ambiente virtual de un laboratorio de Física, versión 1.0. La metodología tiene un enfoque cuantitativo, de corte descriptivo. Los datos se recogieron mediante la escala de usabilidad del sistema (SUS, por sus siglas en inglés), diseñada por Brooke (1996), que fue aplicada a expertos, tanto en el área de tecnología como de Física, para un total de 42 participantes. Se obtuvieron los índices de Alfa de Cronbach y Omega de McDonald con valores de 0.924 y 0.901, respectivamente. Los resultados obtenidos reflejan que el recurso elaborado en formato de Realidad Virtual (RV), obtuvo una valoración de 72.58, para un percentil entre 65 y 69, dentro de un abanico de 500 estudios realizados previamente, por diversos autores, en lo referente a la SUS. Esto indica que el objeto creado tiene un buen rango de aceptación, siendo valorado como muy bueno y adecuado para su utilización en la enseñanza.
PALABRAS CLAVE
Usabilidad; aceptabilidad; ambiente virtual; SUS; valoración de expertos.
ABSTRACT
Virtual reality is part of the set of haptic technologies that allow the user to interact with an artificial scenario, different from the real world but very similar to it. Within the field of education, it promotes active and dynamic learning with immersive experiences. The goal of this study was to assess the usability of an object in VR format for oscilloscope instruction. We created a virtual physics lab environment in version 1.0. The methodology has a quantitative, descriptive approach. Brooke (1996) designed the System Usability Scale (SUS), which collected data from experts in both technology and physics, totaling 42 participants. We obtained Cronbach’s alpha and McDonald’s omega indexes with values of 0.924 and 0.901, respectively. The results reveal that the virtual reality (VR) resource received an assessment of 72.58, corresponding to a percentile between 65 and 69, across 500 previous studies by various authors on the SUS. This indicates that the created object has a good acceptance range, being valued as very good and suitable for use in teaching.
KEYWORDS
Usability; acceptability; virtual environment; SUS; expert evaluation.
1. INTRODUCTION
Developing an information and knowledge society involves the incorporation of technological tools into education (Cabero-Almenara & Marín-Díaz, 2012; López-Meneses & Fernández-Cerero, 2020). Learning and Knowledge Technologies (LKT) represent a specialized category of digital tools designed to enhance meaningful educational experiences (Wang et al., 2024; Izquierdo-Álvarez, 2024). Its increasing adoption is directed towards the creation of innovative and immersive learning environments, characterized by their dynamism and interactivity, with the purpose of stimulating students and generating meaningful learning (Weingärtner et al., 2024; Shadiev et al., 2024).
Today, Virtual Reality (VR) and 360 videos are emerging educational technologies. (Menjívar-Valencia et al., 2022). It is about the interaction of a person in a virtual and artificial world, totally different from the real context, although very close to it (Rodríguez, 2023). This technology allows real-life situations to be simulated in real-time through devices that facilitate immersive experiences, through head-mounted (HMD), as well as non-immersive or desktop displays (Korlat et al., 2024). In other words, it is based on the creation of a representation of an artificial environment dominated by sight and sound created by software, resulting in a perception of reality, due to its ability to interact (Bala & Gupta, 2024; Satu et al., 2024; Samala et al., 2024).
VR-based simulation as an educational tool can create unique specific training conditions, allowing students to practice skills in safe and controlled conditions, which helps to reach various dimensions of curiosity, knowledge and experiences (Koolivand et al., 2024; Rani, 2024; Wangb et al., 2024). Experience in virtual environments can have a significant impact on learning outcomes in terms of improved motivation, knowledge acquisition, and use of an appropriate cognitive load, which translates into better practical, manual, and theoretical skills. (Daling & Schlittmeier, 2024; Ma et al., 2024; Marín-Díaz, Sampedro & Vega-Gea, 2022; Menjívar-Valencia et al., 2022; Mukasheva et al., 2023; Palacios-Rodríguez, Cabero-Almenara & Serrano-Hidalgo, 2024; Pedram et al., 2024;). Such is the case of Physics classes, in which complex concepts can be found for students such as the laws of conservation of energy and the relationship of potential and kinetic energy; another interesting topic arises when teaching physical quantities through virtual tours in 3D (Yang, et al., 2024; Yavoruk, 2023).
An important aspect to consider is that the greater affordability of the hardware necessary to design objects in VR, together with the processing power of current software and the use of haptic devices, allow us to transform a traditional simulation model in a more realistic environment (Sudharson et al., 2024; Huang et al., 2023). This facilitates that, when exposing the student in risky contexts, such as chemical reactions or simulating the driving of vehicles, immersion or interaction in non-immersive objects in VR format is safer (Zou & Liang, 2024; Carulli et al., 2024). On the other hand, given the possibilities that this technology is developing, we find more and more experiences of its use in subjects with specific characteristics and needs (Ausín-Villaverde, et al., 2023; López-Belmonte, et al., 2024).
However, to achieve successful implementation of VR applications in teaching-learning processes, it is crucial to integrate them with high-quality educational methodologies and that the teacher has adequate training for its use.
(Bucăţa & Tileagă, 2024; Rodríguez, Sandu & Santana-Perera, 2024). The usefulness and usability of this technology are key aspects in terms of the quality of the digital system. These characteristics are most effectively appreciated when combined harmoniously with clear pedagogical designs; therefore, it is imperative to evaluate the resource, followed by a meticulous design of an educational model that enriches its integration in classroom environments (Bengsch, 2024; Gao et al., 2024; De Felice et al., 2023). Emphasizing usability, it is defined as the quality of a system to achieve specific goals with effectiveness, efficiency and satisfaction in each context (Sharma et al., 2024; Nishchyk et al., 2024; Robles et al., 2016). Various authors indicate that this feature comprises three fundamental dimensions, as indicated in Figure 1, below: a) effectiveness: degree of completion and the accuracy with which users achieve specific goals; b) efficiency: a user’s ability to perform a task in terms of time and effort (physical and mental); and c) satisfaction: the degree to which the physical, cognitive and emotional responses experienced by the user as a result of use (Purwaningsih et al., 2024; Shaban et al., 2024).
Figure 1. Representation of system usability dimensions.
Source: Scheme of own elaboration
2. MATERIALS AND METHODS
The main objective of this study is to evaluate, through expert evaluation, the usability of an object in VR format for the teaching of the oscilloscope. The idea was to create a virtual environment for the Physics Laboratory of the Santo Domingo Institute of Technology (INTEC) (Figure 2). This is to show the description and use of the oscilloscope, which is an electronic device for measuring electrical waveforms, allowing the electrical signals that change over time to be visually displayed as different types of waves (Bansal et al., 2023; Yuan et al., 2023; Herres, 2020). They can be used to analyze the waveforms of the signals to measure parameters such as voltage (magnitude in charge of establishing the differentiation of electrical potential that exists between two points), frequency (number of waves per second) and period (duration of time of each repetitive event, so it is inverse to the frequency) and to solve circuit problems by visualizing the waveforms of the signals therein (Pietrzak et al., 2024; Sychev & Batako, 2024; Pavuluri et al., 2023).
Figure 2. Panoramic view of the Physics Laboratory.
Source: https://refodige.intec.edu.do/recursos/labfisica/. Screenshot.
To design the learning object in VR format, the following steps were carried out:
•Elaboration of the design, the Physics laboratory of the Santo Domingo Institute of Technology (INTEC) was selected, and the oscilloscope was chosen, as an essential element of the object in VR, since it is widely used by engineering students, specifically in the field of electronics. This required a human team consisting of physics professors and audiovisual experts. It should be noted that this activity will serve to introduce the oscilloscope into the practice of students, to know the essential parts and functions of it.
•Test process for the creation of the object, version 1.0, several 3D shots were required, with the use of the Ricoh® Theta® SC2 camera. After this, the KRPANO® application was used which is a tool for virtual tours with a panoramic high resolution, which can be seen on websites or mobile devices, even with glasses holders to facilitate the virtual reality experience (https://krpano.com). Responsible for this process were the coordinator of the Physics area and the head of the Physics Laboratory, together with audiovisual technicians from INTEC, who met on several occasions to define the script to follow, specifically in the selection of hot spots, within a virtual tour and the practice of describing the components of the oscilloscope (See figure 3). In this phase, the resources and the capacity of the developers were tested, to make the necessary adjustments in the programming stage.
•Also indicate that for its design and production we rely on the principles derived from the cognitive theory of multimedia learning (Kartiko et al., 2010). Theory formulated by Mayer (2002 and 2021), and is based on several key principles, as highlighted by Mulder et al. (2020). These principles include reducing superfluous processing, eliminating elements that may distract the student, and considering principles such as information redundancy and the use of signage to direct the student’s attention. In addition, the principles of the modality that establishes that it is preferable to present images accompanied by spoken text instead of written, which facilitates better learning and segmentation that recommends the distribution of complex material in smaller learning units.
•This led to the use of different resources in its production: 360º video, incorporation of hot spots, video clip, locution, segmentation of information, and written texts.
Figure 3. Hot spot sample (active point) on oscilloscope for on/off button.
Source: https://refodige.intec.edu.do/recursos/labfisica/. Screenshot.
2.1 Instrument for collecting information
To evaluate this version 1.0 of the object in VR format, the evaluation strategy was used through expert evaluation that consists of requesting a series of people, the demand for a judgment towards a virtual object or a teaching material, or their opinion regarding a specific aspect (Cabero-Almenara et al., 2021; Welsandt et al., 2024). This technique is common in educational research, since it presents advantages such as: a) the theoretical quality of the answers reached, b) the level of depth of these and c) being able to obtain detailed information (Hsu et al., 2024; Cabero & Barroso, 2013; Adrogué et al., 2020). It has been used in various studies, such as the evaluation of a platform for English teaching, with VR through the metaverse and the application of Augmented Reality developed for geometry material (Santosa & Banjar, 2024; Prasetya & Hidayati, 2024)
To carry out the data collection, the “System Usability Scale” (“System Usability Scale” – SUS) was developed by John Brooke, in 1996, especially to evaluate its engineering programs; allowing an easy and agile mechanism to evaluate products and services (Martins et al. 2015; Ampuan &
Deleña, 2024). It also offers the possibility to evaluate the overall user experience. In the case of technological tools, SUS consists of a standard questionnaire administered to users after they have interacted with the system in question, by which various aspects of usability are classified, such as ease of learning, efficiency of use, user satisfaction and ease of remembering how to use the system (Vlachogianni, & Tselios, 2022; Awang et al., 2024).
Some of the reasons justifying their choice are presented in Table 1.
Table 1. Reasons for choosing the SUS scale.
|
Reasons |
Source of information |
|---|---|
|
Used to measure different types of technology |
Platforms (Lirola & Pérez, 2020) Web (Galuh et al., 2021) RA and RV (Campo-Prieto et al., 2021) |
|
It has been translated and adapted in different languages, with high levels of reliability |
Spanish (Castilla et al. 2023) Portuguese (Silva & Turrini, 2019) Dutch (Ensink et al., 2024) |
|
Used in combination with other instruments |
Technology Acceptance Model (TAM), Unified Theory of Technology Acceptance and Use (UTAUT), version 2, emotion scale (Pincay et al. 2021; Cheah et al. 2022; Ong et al. 2023) |
|
Existence of different research meta-analyzes |
Source: Scheme of own elaboration
On the other hand, it cannot be forgotten that it is easy and quick to apply, and that in all the studies in which it has been applied, no problematic situation has arisen regarding its interpretation by the people who completed it and the way in which they should do it.
The implementation of the SUS was carried out during the period February–April 2024, so it has a cross-cutting scope. It was carried out through the sending, both RV resource of the Physics laboratory and of the questionnaire, to professors and experts in the areas of Technology, as well as professors of Physics of the Technological Institute of Santo Domingo (INTEC), of Universidad Nacional Pedro Henríquez Ureña (UNPHU), Dominican Republic. In addition, they were shared by the platform “The metaverse: extended reality (Virtual and Augmented) in higher education: Design, production, evaluation and training of extended reality programs for university education” - MEREVIA (https://merevia.es/) and the platform “Design, production and evaluation of programs in extended reality format for training in climate change and comprehensive disaster risk management” – REFODIGE (https://refodige.intec.edu.do/). Likewise, masters of the subject Pedagogical Models and Curriculum in Higher Education of the University of Murcia, Spain participated in the study. This is of important value, as responses have been obtained from experts, both from the Dominican Republic and Europe. For a total of 42 participating experts. Number of experts that exceed the recommended for these studies that vary according to different authors between 15-35 (Malla & Zabala, 1978; Witkin & Altschuld, 1995; Landeta, 2002).
The research has a quantitative approach and descriptive scope with non-probabilistic sampling. It is a sampling technique, in which the choice of elements does not depend on probability, but on causes related to the characteristics of the research (Levitt, 2021; Shu et al., 2023).
Indicate that the selected experts were university professors, who usually taught in Educational Technology or in subjects related to the contents referred to in the object of learning produced.
The scale consists of 10 questions (5 with positive statements and 5 with negative statements), each of which is scored according to Likert type system where 1 to 5, where 1 means total disagreement and 5, total agreement. The question items are as follows, as shown:
a)I think I would like to use this system frequently
b)I found the system unnecessarily complex
c)I think the system is easy to use
d)I think I would need help from a person with technical knowledge to use this system
e)I found that the various functions of this system were well integrated
f)I think the system is very inconsistent
g)I imagine most people would learn to use this system very quickly
h)I find the system very difficult to use
i)I felt very safe using the system
j)I needed to learn a lot of things before starting the system
For data analysis, SUS results are measured on the scale from 0 to 100. They were collected in “Statistical Package for the Social Sciences” (SPSS) Version 22. You should avoid modifying the questions or changing their order.
The definitive SUS value is obtained by transforming the scores, depending on the items occupying an even or odd position (Brooke, 1996). For the odd items (1, 3, 5, 7, 9), subtract one from each of the answers offered by the user, and then the sum of the 5 scores. For even items (2, 4, 6, 8, 10), subtract to 25 the sum of all the answers offered by the user in the 5 items. Subsequently, the sum of the scores obtained in the pairs and odd items is performed. And the result is multiplied by 2.5 converts the range of possible values from 0 to 100 instead of 0 to 40. As indicated in the following formulas:
The score obtained in this range from 0 to 100, cannot be defined in terms of percentage and cannot be interpreted as good or bad. However, authors such as Lewis and Sauro (2009) have developed a rating scale, after having analyzed more than 500 studies related to SUS, that allows to obtain a classification of the results to evaluate the digital resource and adjust of place in the cases that are necessary.
This scale is shown in table 2 below, which presents the grades, SUS scores, percentile range and definition of adjectives that indicate the general analysis of the values generated by the results of the responses to the various statements, to normalize the result and adjectivize it by a range of categories. It also presents the level of acceptability of the object in VR format, resulting in an assessment from a qualitative perspective.
Table 2. SUS percentiles, grades and adjectives.
|
Degree |
SUS score |
Percentile Range |
|---|---|---|
|
A |
80.8–84.1 |
90–95 |
|
A- |
78.9–80.7 |
85–89 |
|
B+ |
77.2–78.8 |
80–84 |
|
B |
74.1–77.1 |
70–79 |
|
B- |
72.6–74.0 |
65–69 |
|
C+ |
71.1–72.5 |
60–64 |
|
C |
65.0–71.0 |
41–59 |
|
C- |
62.7–64.9 |
35–40 |
|
D |
51.7–62.6 |
15–34 |
|
F |
0–51.6 |
0-14 |
Source: Lewis and Saurus Adapted Scheme (2009).
Likewise, this scale gives an idea of the ease of the application or resource with which it interacted and allows to visualize whether users would recommend it to achieve certain goals with effectiveness, efficiency and satisfaction in each context (Brooke, 2013; Mata & Hernández-Ruiz, 2019; Marco-Ahulló et al., 2022).
3. RESULTS
Reliability was analyzed using Cronbach’s Alpha and McDonald’s Omega, the results obtained were 0.924 and 0.901, respectively. Values that according to O’Dwyer and Bernauer (2014) denote high levels of reliability.
Regarding the measurement, starting from the results obtained with the SUS valuation, after scaling from 0 to 100, a value of 72.58 was obtained, considered as very good, within a percentile between 65 and 69 and the degree of acceptability is “acceptable” (chart 2). Gimeno (2018) indicates that each score is given an adjective: between 0 and 25, the worst; between 25 and 50, poor; between 50 and 70, good; between 70 and 80 very good; between 80 and 90, excellent; more than 90, unmatched. Having obtained in the evaluation made by the experts a score of 72.58 can be considered with a good usability, and consequently justify in this way its incorporation into teaching (See table 3).
Table 3. SUS score results, for virtual oscilloscope resource.
|
Criteria |
Degree |
SUS Average |
Adjective |
Percentile range |
Acceptability |
|
Oscilloscope virtual resource |
B- |
72.58 |
Very good |
65–69 |
Acceptable |
Source: Scheme of own elaboration.
4. DISCUSSION AND CONCLUSIONS
The conclusions of the work go in different directions, some referring to the instrument itself, and others to the learning object itself considered. Starting with the first, two aspects should be noted: that the level of reliability obtained in the work is high and coincides with those obtained in other works (Lewis, 2018). And with respect to the second, it has been shown an easy scale to apply, to administer and to understand by the people who completed it. These two aspects point out that it is an instrument that can be truly applicable for the evaluation of the usability of different technologies. This fact leads to reinforce the idea expressed by Lewis (2018), which indicates that researchers and professionals who need a measure of perceived usability, should seriously consider the use of SUS.
For the case of the total and average score of the SUS results, after using the algorithm to scale it from 0 to 100, a value of 72.58 was obtained for a rating of very good. It exceeds the minimum acceptable value proposed by Brooke (1996, 2013). As for the adjective valuation, defined by Bangor et al. (2009), by the expert valuation is of ‘very good’.
Another element to highlight is the relationship between the score obtained and the grade rating scale, proposed by Sauro and Lewis (2012), in which this study resulted in B. As well as the percentile yielded of 65 – 69, indicating that the results are above 69 %, in relation to the studies carried out with SUS by these authors. This allows “researchers to compare their results using this questionnaire without having to resort to direct scores” (Lirola & Pérez, 2020, p. 197).
The scores point out that the object produced and the built-in elements of video in 360º format, incorporation of hot spots, video clip, locution, segmentation of information, and written texts. They are perceived by experts with real usability and consequently with possibilities to be incorporated into teaching.
In the work several limitations must be assumed, the first of which may refer to the selection process of experts who evaluated the learning object produced. Although measures for qualified election were adopted, some more refined procedure for the election of experts could have been used, such as the expert competency coefficient (Martínez et al., 2018; Cruz & Martínez, 2018), which further ensured the quality of experts.
On the other hand, the instrument used, although with high levels of reliability and with a very large use of research and studies that use it, is of the type of self-perception, with the limitations that this entails. Therefore, it would be appropriate to replicate, or use, in these studies more qualitative instruments, such as interviews, discussion groups or nominal groups.
This study can benefit Physics laboratory classes about the oscilloscope, as there are not many studies on VR that relate the teaching methodology in this area and this topic, in particular, to analyze the different signals in electrical circuits. The high value of this resource suggests that the application of VR in higher education can extend to other basic sciences like chemistry and biology, as well as engineering and health. However, it’s important to remember that students place a high value on this technology (Menjvar Valencia et al., 2022). Of course, this requires collaboration between different disciplines with clear teaching and learning objectives within a reasonable time in the academic period (Fink et al., 2023). The obtained results underscore the importance of conducting additional studies to evaluate the acceptance patterns of this technology for immersive learning, while also suggesting potential enhancements to the level of interactivity. Hong et al. (2023) highlight that simulating the representation of reality can create a gamified environment, facilitating self-management of knowledge and improving learning outcomes. Virtual Reality (VR) plays a significant role in teaching magnetic fields and quantum waves. By modeling these objects in 3D, students can interact with them and analyze them from various scales and angles, thereby enhancing their understanding of these phenomena.
Future research should first replicate the assessment using an immersive version and compare the outcomes with the findings from the current study. Conversely, researchers could design a study that links the subjects’ evaluation of the object’s usability to other factors like their performance, their acceptance of the subject, their motivation from participating in virtual experiences, or their cognitive load. Finally, we could apply the instrument to students to understand their usability, calculate their score, and compare their evaluation with that of experts.
5. LIMITATIONS OF THE STUDY
One of the limitations encountered during this study was the availability of expert teachers to evaluate the object. Both the resource and the questionnaire were sent to them via email and through the REFODIGE project platform, and to obtain evaluation responses, they had to call and motivate. Therefore, the number of responses was very limited. Another limitation was that this was the work team’s first experience designing the script and producing the object in VR. Therefore, gaining greater technical and didactic mastery of the VR object can enhance the management of equipment and software. Aguilar (2020) refers to the need to strengthen capacities among the teaching team in the use of new trends in education, both in methodologies and in programming and programming logic.
Moreover, we created the object solely to describe the oscilloscope in version 01 format. After analyzing the results, we will adjust the object to version 02 to implement improvements. Ultimately, the evaluated version aligns with both desktop and non-immersive versions. The usability and expert evaluation results indicate the need to continue designing and producing so that virtual reality can become more immersive and inclusive (Segura and Osorio, 2021). Finally, we applied a questionnaire to evaluate the experts, who responded anonymously and without observation. Although semi-structured interviews are more susceptible to bias, they could have enriched the results (Ber-gen & Labonte, 2020). However, we can establish a connection between what we observed and the participants’ best experiences.
FUNDINGS
This study is part of a R & D research project, was funded by the Ministry of Higher Education, Science and Technology of the Dominican Republic through the National Fund for Scientific and Technological Innovation and Development (FONDOCYT- 2022-1C3-062), called “Design, production and evaluation of extended reality programs for training in climate change and comprehensive disaster risk management (REFODIGE)”.
REFERENCES
, , , & (2020). Influence of learning strategies and skills on academic performance. Revista Educación, 45(1), 4-19. https://doi.org/10.15517/revedu.v45i1.41065
(2020). A research-based approach for evaluating resources for transitioning to teaching online. Information and Learning Sciences, 121(5), 301-310. https://doi.org/ 10.1108/ILS-04-2020-0072
, & (2024, 29–30 January). A Quantitative Evaluation of Online Appointment System at Mindanao State University–Main Campus: Employing the System Usability Scale (SUS) and Technology Acceptance Model (TAM) [Paper presentation]. 2024 3rd International Conference on Digital Transformation and Applications (ICDXA), Kuala Lumpur, Malaysia. https://doi.org/10.1109/ICDXA61007.2024.10470770
, , , & (2023). Evaluation of an augmented reality APP for children with dyslexia: a pilot study: [Evaluation of an augmented reality APP for children with dyslexia: a pilot study]. Pixel-bit. Journal of Media and Education, 66(0), 87–111. https://doi.org/10.12795/pixelbit.95632
, , & (2024). Insights on usability testing: The effectiveness of an adaptive e-learning system for secondary school mathematics. International Electronic Journal of Mathematics Education, 19(3), 07–82. https://doi.org/10.29333/iejme/14621
, & (2024). Virtual Reality in Education. In R. Malik, A. Sharma, P. Chaudhary (Eds.), Transforming Education with Virtual Reality (pp. 165–180). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781394200498.ch10
, , & (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114–123. https://uxpajournal.org/wp-content/uploads/sites/7/pdf/JUS_Bangor_May2009.pdf
, , & (2023). A taxonomical review of developments in digital storage oscilloscope (DSO). AIP Conference Proceedings, 2796(1), 18–81. https://doi.org/10.1063/5.0149169
(2024). Redefining Traditional Education: The Integration of Machine Learning in the Contemporary Language Education Classroom. In F. Pan (Ed.), AI in Language Teaching, Learning, and Assessment (pp. 195–221). IGI Global. https://doi.org/10.4018/979-8-3693-0872-1.ch010
, & (2020). Everything Is perfect, and we have no problems: Detecting and limiting social desirability bias in qualitative research. Qualitative Health Research, 30(5), 783–792. https://doi.org/10.1177/1049732319889354
(1996). SUS—A Quick and Dirty Usability Scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & I. L. McClelland (Eds.), Usability Evaluation in Industry (pp. 189-194). Taylor & Francis.
(2013). SUS: A retrospective. Journal of Usability Studies, 8(2), 29–40. https://www.researchgate.net/publication/285811057_SUS_a_retrospective
, & (2024). Digital Renaissance in Education: Unveiling the Transformative Potential of Digitization in Educational Institutions. Land Forces Academy Review, 29(1), 20–37. https://doi.org/10.2478/raft-2024-0003
, & (2013). The use of expert judgment for ICT assessment: the coefficient of expert competence. Embroidery. Journal of Pedagogy, 65(2), 25-38. https://doi.org/10.13042/brp.2013.65202
, & (2012). ICT training of university teachers in a personal learning environment. Project DIPRO 2.0. New Approaches in Educational Research, 1(1), 2-6. https://doi.org/10.7821/naer.1.1.2-6
, , & (2021). Digital competences of educators in Health Sciences: Their relationship with some variables. Medical Education, 22(2), 94-98. https://doi.org/10.1016/j.edumed.2020.11.014
, , , & (2020). Immersive Virtual Reality in older people: a case study (case study). Challenges, 39(0). 101–105. https://doi.org/10.47197/retos.v0i39.78195
, , , , , , & (2024). A Case Study on Digital Training: Benefits and Unexplored Opportunities. Journal of Computing and Information Science in Engineering, 24(05), 25–50. https://doi.org/10.1115/1.4064797
, , , , , , , , & (2023). Psychometric Properties of the Spanish Full and Short Forms of the System Usability Scale (SUS): Detecting the Effect of Negatively Worded Items. International Journal of Human–Computer Interaction, 0(0), 1–7. https://doi.org/10.1080/10447318.2023.2209840
, , , & (2022). Mobile Technology in Medicine: Development and Validation of an Adapted System Usability Scale (SUS). Questionnaire and Modified Technology Acceptance Model (TAM) to Evaluate User Experience and Acceptability of a Mobile Application in MRI Safety Screening. Indian Journal or Radiology and Imaging, 33(1), 37–45. https://doi.org/10.1055/s-0042-1758198
, & (2020). Origin and development of an index of expert competence: The coefficient k. Latin American Journal of Social Research Methodology, 1(19), 40-56. http://www.relmis.com.ar/ojs/index.php/relmis/article/view/origen_desarrollo_indice_competencia_experta/7
, & (2024). Effects of Augmented Reality-, Virtual Reality-, and Mixed Reality–Based Training on Objective Performance Measures and Subjective Evaluations in Manual Assembly Tasks: A Scoping Review. Human Factors, 66(2), 589–626. https://doi.org/10.1177/00187208221105135
, , , , & (2023). How Does Metaverse Shape Education? A Systematic Literature Review. Applied Sciences, 13(9), 9–21. https://doi.org/10.3390/app13095682
, , & (2024). Translation and validation of the System Usability Scale to a Dutch version: D-SUS. Disability and Rehabilitation, 46(2), 395–400. https://doi.org/10.1080/09638288.2022.2160837
, & (2023). What variables are connected with system usability and satisfaction? Computers & Education: X Reality, 3, 100043. https://doi.org/10.1016/j.cexr.2023.100043
, and (2021). Evaluasi Usability Website Shopee Menggunakan System Usability Scale (SUS). Journal of Applied Informatics and Computing (JAIC), 5(2), 146–150. https://doi.org/10.30871/jaic.v5i2.3293
, , , , , & (2024). Hybrid Soft-Rigid Active Prosthetics Laboratory Exercise for Hands-On Biomechanical and Biomedical Engineering Education. Journal of Biomechanical Engineering, 29(1), 1–55. https://doi.org/10.1115/1.4065008
(2018). Five ways to interpret a SUS. Torresburriel. https://torresburriel.com/weblog/cinco-formas-de-interpretar-un-sus/
(2020). Acquiring, Displaying, and Measuring Digital Signals in an Oscilloscope. In D. Herres (Ed.), Oscilloscopes: A Manual for Students, Engineers, and Scientists (pp. 155–180). Springer International Publishing. https://doi.org/10.1007/978-3-030-53885-9_6
, , , , & (2023). VR training program for fire escape: Learning progress predicted by the perception of fire presence, VR operational frustration, and gameplay self-efficacy. Computers & Education: X Reality, 3, 100029. https://doi.org/10.1016/j.cexr.2023.100029
, , , & (2023). Effects of Incorporating an Expert Decision-making Mechanism into Chatbots on Students’ Achievement, Enjoyment, and Anxiety. Educational Technology & Society, 26(1), 218–231. https://www.jstor.org/stable/48707978
, , , , , , , & (2023). Student perceptions toward virtual reality training in dental implant education. PeerJ, 11(1), 14–57. https://doi.org/10.7717/peerj.14857
(2024). Integration of Information and Communication Technologies Education for the Deaf. In M. Khaldi (Eds.), Technological Tools for Innovative Teaching (pp. 103–120), IGI Global. https://doi.org/10.4018/979-8-3693-3132-3.ch006
, & (2010). Learning science in a virtual reality application: The impacts of animated-virtual actors’ visual complexity. Computer & Education, 55(0), 881–891. https://doi.org/10.1016/j.compedu.2010.03.019
, , , , , , , & (2024). Comparison of the effectiveness of virtual reality-based education and conventional teaching methods in dental education: A systematic review. BMC Medical Education, 24(1), 18–34. https://doi.org/10.1186/s12909-023-04954-2
, , , , , & (2024). PhyLab – a virtual reality laboratory for experiments in Physics: a pilot study on intervention effectiveness and gender differences. Frontiers in Psychology, 15(0), 1-13. https://doi.org/10.3389/fpsyg.2024.1284597
(2002). The Delphi method: a technique for forecasting the future. Oh, Ariel.
(2021). Qualitative generalization, not to the population but to the phenomenon: Reconceptualizing variation in qualitative research. Qualitative Psychology, 8(1), 95–110. https://doi.org/10.1037/qup0000184
(2018) The System of Usability Scale: Past, Present, and Future. International Journal of Human-Computer Interaction, 34(7), 577–590. https://doi.org/10.1080/10447318.2018.1455307
, and (2009). The Factor Structure of the System Usability Scale. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 5619(0), 94–103. https://doi.org/10.1007/978-3-642-02806-9_12
, & (2020). The perceived usability of distance vocational training teachers in the Balearic Islands. Pixel-Bit, 59(1), 147–183. https://doi.org/10.12795/pixelbit.76299
, Duo, , & (2024). Effects of augmented and virtual reality on students with ASD (ASD). Pixel-bit. Journal of Media and Education, 70(0), 7–23. https://doi.org/10.12795/pixelbit.103789
, & (2020). Information and Communication Technologies and functional diversity: knowledge and training of teachers in Navarra. IJERI: International Journal of Educational Research and Innovation, 14, 59–75. https://doi.org/10.46661/ijeri.4407
, , , , , , , & (2024). Using immersive virtual reality technology to enhance nursing education: A comparative pilot study to understand efficacy and effectiveness. Applied Ergonomics, 11(5), 104–159. https://doi.org/10.1016/j.apergo.2023.104159
& (1978). Forecasting the future in the company (III): the Delphi method. Business Studies, 39,13–24.
, , , , (2022). Design and usability of ParaSportAPP: an mHealth designed to promote physical activity in people with spinal cord injury. Andalusian Journal of Sports Medicine. 15(2), 65–71, https://ws208.juntadeandalucia.es/ojs/index.php/ramd/article/view/921/1326.
, , & (2022). Promoting learning through use of 360º videos. Innoeduca. International Journal of Technology and Educational Innovation, 8(2), 138–151. https://doi.org/10.24310/innoeduca.2022.v8i2.15120
, , , & (2024). How personalized and effective is immersive virtual reality in education? A systematic literature review for the last decade. Multimedia Tools and Applications, 83(6), 18185–18233. https://doi.org/10.1007/s11042-023-15986-7
, , , & (2018). Identification of the specific competencies of nurses in the care of the neonate in severe condition. Medisan, 22(2).1–84. http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S1029-30192018000200009&lng=es&nrm=iso
, , , , & (2015). Activity of phenolic compounds from plant origin against Candida species. Industrial Crops and Products, 74(0), 648–670. https://doi.org/10.1016/j.indcrop.2015.05.067
& (2019). Usability assessment for an e-commerce site: Development of a methodology and its application to the site crgourmetcoffee.com. In Y. Morales-López (Ed.), Memoirs of the First International Congress of Exact and Natural Sciences of the National University, Costa Rica, 2019 (pp. 1-10). Heredia: Universidad Nacional. https://doi.org/10.15359/cicen.1.9
(2002). Multimedia learning. Psychology of Learning and Motivation, 42, 85–139. https://doi.org/10.1016/S0079-7421(02)80005-6
(2021). Multimedia learning (3rd ed). Cambridge University Press. https://doi.org/10.1017/9781316941355
, , , & (2022). Percepciones de estudiantes universitarios sobre la realidad virtual como recurso didáctico: un estudio preexperimental con un grupo de control y experimental. IJERI: International Journal of Educational Research and Innovation, (17), 152–171. https://doi.org/10.46661/ijeri.5904
, , , , , & (2023). Contextual structure as an approach to the study of virtual reality learning environment. Cogent Education, 10(1), 16–57. https://doi.org/10.1080/2331186X.2023.2165788
, , & (2020). A Framework for the Use of Immersive Virtual Reality in Learning Environments. International Journal of Emerging Technologies in Learning (iJET), 15(24), 208–224. https://doi.org/10.3991/ijet.v15i24.16615
, , & (2024). Elderly-centered usability heuristics for augmented reality design and development. Universal Access in the Information Society, 1(1), 1–21. https://doi.org/10.1007/s10209-023-01084-w
& (2014). Quantitative research for the qualitative researcher. Sage. https://doi.org/10.4135/9781506335674
, , , , , , , , & (2023). Determination of Factors Influencing the Behavioral Intention to Play Mobile Legends: Bang-Bang during the COVID-19 Pandemic: Integrating UTAUT2 and System Usability Scale for a Sustainable E-Sport Business. Sustainability, 15(4), 1–26. https://doi.org/10.3390/su15043170
, , & (2024). Educación Médica y Carga Cognitiva: Estudio de la Interacción con Objetos de Aprendizaje en Realidad Virtual y Vídeo 360º. Revista de Educación a Distancia (RED), 24(79). https://doi.org/10.6018/red.582741
, , , , & (2023). Portable Digital Oscilloscope using Arduino. 2023 International Conference on Inventive Computation Technologies (ICICT), 0(0), 1361–1366. https://doi.org/10.1109/ICICT57646.2023.10134297
, , & (2024). Assessing the validity of VR as a training tool for medical students. Virtual Reality, 28(1), 15-35. https://doi.org/10.1007/s10055-023-00912-x
, , & (2024). Microcontroller-Based Embedded System for the Diagnosis of Stator Winding Faults and Unbalanced Supply Voltage of the Induction Motors. Energies, 17(2), 40–55. https://doi.org/10.3390/en17020387
, , & (2021). The usability and differential scale of emotions in Android apps. A case study. Mikarimin. Multidisciplinary Scientific Journal, 7(1), 79-86. https://revista.uniandes.edu.ec/ojs/index.php/mikarimin/article/view/2246
, & (2024). Implementation of augmented reality application in learning mathematics of elementary school. AIP Conference Proceedings, 2926(1), 20–64. https://doi.org/10.1063/5.0185227
, , , & (2024). Utilizing blockchain technology in enhancing supply chain efficiency and export performance, and its implications on the financial performance of SMEs. Uncertain Supply Chain Management, 12(1), 449–460. https://doi.org/10.5267/j.uscm.2023.9.007
(2024). Impact of Virtual Reality on Immersive Education. In R. Malik, A. Sharma, P. Chaudhary, (eds.), Transforming Education with Virtual Reality (pp. 101–125). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781394200498.ch7
, , , & (2016). International Organization for Standardization (ISO). Education and Health Scientific Bulletin Institute of Health Sciences Universidad Autónoma del Estado de Hidalgo, 4(8), 18-26. https://doi.org/10.29057/icsa.v4i8.277
(2023). Ethics of Virtual Reality. In: F. Lara, J. Deckers, (eds) Ethics of Artificial Intelligence. The International Library of Ethics, Law and Technology (pp. 1–107). Springer, Cham. https://doi.org/10.1007/978-3-031-48135-2_6
, , , , , & (2024). Exploring Campus through Web-Based Immersive Adventures Using Virtual Reality Photography: A Low-Cost Virtual Tour Experience. International Journal of Online & Biomedical Engineering, 20(1), 104–127. https://doi.org/10.3991/ijoe.v20i01.44339
, & (2024). Animals of Nusantara: Virtual Reality-Based English Learning Materials for Secondary Students. Nusantara Science and Technology Proceedings, 2024(38), 9–22. https://doi.org/10.11594/nstp.2024.3802
, , , , , & (2024). Virtual-Reality training solutions for nuclear power plant field operators: A scoping review. Progress in Nuclear Energy, 169(0), 104–125. https://doi.org/10.1016/j.pnucene.2024.105104
, & (2012). Quantifying the user experience: Practical statistics for user research. Elsevier/Morgan Kaufmann. https://doi.org/10.1016/B978-0-12-384968-7.00002-3
& (2021). Usability in Accessible and Inclusive Multi-Scenario Immersive Virtual Reality Applications: Case Study. Investigación e Innovación en Ingenierías, 9 (3), 82-92, 2021. https://doi.org/10.17081/invinno.9.3.5563
, , , & (2024). The invisible hand of touch: Testing a tactile sensation-choice satisfaction model in online shopping. Journal of Sensory Studies, 39(1), 12–37. https://doi.org/10.1111/joss.12897
, , & (2024). Cultivating self-directed learning abilities in K-12 students through immersive online virtual tours. Interactive Learning Environments, 0(0), 1–26. https://doi.org/10.1080/10494820.2024.2312923
, , , , & (2024). Entrepreneurial challenges of COVID-19: Re-thinking entrepreneurship after the crisis. Journal of Small Business Management, 62(2), 824–846. https://doi.org/10.1080/00472778.2022.2089676
, , , , & (2023). Non-Probabilistic Reliability Analysis of Slopes Based on a Multidimensional Parallelepiped Model. Applied Sciences, 13(17), 1–15. https://doi.org/10.3390/app13179874
& (2019). Development of an educational mobile application for patients submitted to orthognathic surgery. Magazine. Latin American. Enfermagem, 27:e3143. https://doi.org/10.1590/1518-8345.2904.3143
, & (2019). Development of an educational mobile application for patients submitted to orthognathic surgery. Revista Latino-Americana de Enfermagem, 27(0), 31-43. https://doi.org/10.1590/1518-8345.2904.3143
, , , , , & (2024). A Novel Adaptive Framework for Immersive Learning Using VR in Education. In R. Malik, A. Sharma, P. Chaudhary (Eds), Transforming Education with Virtual Reality (pp. 1–26). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781394200498.ch1
, & (2024). A Study of Sliding Friction Using an Acoustic Emission and Wavelet-Based Energy Approach. Machines, 12(4), 1–15. https://doi.org/10.3390/machines12040265
& (2022). Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review. Journal of Research on Technology in Education, 54(3), 392–409. https://doi.org/10.1080/15391523.2020.1867938
& (2024). How virtual reality, augmented reality and mixed reality facilitate teacher education: A systematic review. Journal of Computer Assisted Learning, 40(3), 1276–1294. https://doi.org/10.1111/jcal.12949
, , , , , , , & (2024). Examining the Effects of an Immersive Learning Environment in Tertiary AEC Education: CAVE-VR System for Students’ Perception and Technology Acceptance. Journal of Civil Engineering Education, 150(2), 05–23. https://doi.org/10.1061/JCEECD.EIENG-1995
, , & (2024). Exploring factors influencing students’ willingness to use translation technology. Education and Information Technologies, 29(3), 3047–3073. https://doi.org/10.1007/s10639-024-12511-7
, , , & (2024). Sociodigital experiences and creativity in the metaverse: An integrative review. Heliyon, 10(7), 29–47. https://doi.org/10.1016/j.heliyon.2024.e29047
, , , & (2024). Constructing and validating authentic assessments: The case of a new technology-based assessment of economic literacy. Empirical Research in Vocational Education and Training, 16(1), 4–15. https://doi.org/10.1186/s40461-024-00158-0
& (1995). Planning and conducting needs assessment: A practical guide. Oh, Sage.
, , (2024). Virtual Reality to Teaching the Law of Conservation of Energy in Physics. In: J. Hung, N. Yen, J. Chang, (eds) Frontier Computing on Industrial Application (pp. 60–75). Springer, Singapore. https://doi.org/10.1007/978-981-99-9342-0_16
(2023). The Use of 3D Virtual Tours Technology in Physics Classes for Teaching Physical Quantities. Proceedings of the 2023 4th International Conference on Education Development and Studies, 0(0), 7–11. https://doi.org/10.1145/3591139.3591140
, , , , , , , & (2023). Scaling Relationship on Learning Mathematical Reasoning with Large Language Models (arXiv:2308.01825). arXiv. https://doi.org/10.48550/arXiv.2308.01825
, & (2024). Algorithm Optimization of Computer Simulation Vehicle Driving Simulation System Based on Virtual Reality Technology. International Journal of Computational Intelligence Systems, 17(1), 34–62. https://doi.org/10.1007/s44196-024-00426-7