Automation in Education with Digital Twins: Trends and Issues

Eric Hawkinson

Professor, Kyoto University of Foreign Studies, Japan, https://erichawkinson.com

Abstract

Keywords: digital twins, edutainment, augmented learning, virtual performers, virtual teachers

Introduction

Global economic growth rates have been impacted radically by the introduction of Industry 4.0 across many countries and sectors. Emerging technologies determine how competitive companies are in the technology market. There are many distributions and innovations around the creation, coping, and distribution of digital goods to that of physical ones. Books, music, videos, and games that are now predominantly digital were once physical products sold in brick- and-mortar stores. This trend is shifting with the emergence and availability of technologies like augmented and virtual reality, making the digitization of goods and services more possible. The concept of the digital twin is one of the breakthrough technologies resulting to a significant impact on growth in many more industries (Kamble et al., 2018; Xu et al., 2018).

In the early 2000s, when manufacturing machinery and production systems were being digitized, digital twins began to gain popularity (El Saddik, 2018). By using artificial intelligence, physical modeling, and data analytics, General Electric (GE) makes digital twins of its machines to improve machine management through sensor data. By serving as a virtual replica of what was created, digital twins can improve efficiency, reduce costs, stimulate innovation, and ensure quality by serving as a part of an enterprise-wide closed-loop product lifecycle (Tao et al., 2019).

Digital twin is a term used to refer to a variety of technologies that relate to the process of using digital copies of physical objects, places, people, environments to run virtual simulations. In manufacturing, digital twins have been used to analyze supply chains and other processes in factories (Jaensch et al., 2018). Digital twins of places have been used to study urban planning, simulate traffic, and analyze the harvesting of crops (Li et al., 2022). In learning, digital twins have been popular to help run simulations and experiments on a variety of things from the design of rockets to the layouts of classrooms (David et al., 2018).

In the future, digital twins will likely disrupt industries beyond manufacturing since they can be applied broadly to many technologies. Extending the definition of the concept is therefore critical. Data may be effortlessly transferred between the physical and virtual worlds, allowing all living and nonliving elements to be monitored, understood, and optimized.

An example of the jump from digital media to digital twins is the digitization of not only music but of the artists themselves. Swedish supergroup ABBA has enjoyed digital sales of their music over the world since originally putting out music to be sold in stores and broadcast on the radio from the 1970s. ABBA was a Swedish musical phenomenon, winning the Eurovision Song Contest in 1974 and breaking numerous worldwide sales records. From early on in their career until the band's final album in 1981, their music became known and loved around the world, being adopted into Broadway musicals, theatrical movies, and other forms of digital media. The next step for ABBA is to digitize themselves as humans to tour the world once again on stage as holographic superstars.

Digital twins of humans can be used to gather and analyze physical, physiological, and contextual information to get a better understanding of the human body and well-being. Digital twins can improve quality of life and well- being (El Saddik, 2018). The ability to forecast the onset of a stroke, for example, could allow for the implementation of prophylactic measures. Machine learning and deep learning could be effective for forecasting health concerns based on lifestyle habits (Ravì et al., 2017). Contextual data, such as information on the user's environment, age, emotional state, and preferences, could be utilized to characterize the user's holistic condition in addition to demographic data.

Features of Digital Twins

There are some common emerging features of digital twins across context and implementations. Others have implicated likely features of digital twins in the future (Rosen et al., 2015; Stark et al., 2019; Tao et al., 2019; Van der Valk et al., 2020). Many of these features and issues represented in virtual worlds now can be observed, such as the high level of data collected about user behavior and physical attributes leading to privacy and security concerns. The introduction of digital twins in educational contexts has the potential to bring several benefits and implications for the future. Some of these include:

• Real-time data synchronization: Digital twins continuously receive and reflect data from their physical counterpart, allowing for real-time monitoring and analysis.
• Advanced modeling and simulation: Digital twins utilize complex mathematical models and simulations to accurately represent the physical object or system.
• Interconnectivity: Digital twins can interact with other digital twins and systems, providing a unified view of multiple physical objects and their interrelationships.
• Predictive analytics: Digital twins use data and analytics to make predictions about the behavior of the physical object, system, or process they represent.
• Remote monitoring and control: Digital twins allow for remote monitoring and control of physical objects, systems, or processes, enabling organizations to respond quickly to issues and optimize performance.
• Dynamic updating: Digital twins can be updated in real-time as new data becomes available, providing organizations with up-to-date information about the physical object, system, or process they represent.

As the technologies around digital twins advance, such as 3D mapping, data visualization, 3D modeling, graphical processing and more, both the ability of digital twins to match more closely than that of its physical version and to simulate how a digital twin would behave in the physical world are obtained. These combine to increase the predictive power of digital twins which increase their value over many uses.

Digital Twins in e-Learning and Distance Education

Digital twins are already employed in a variety of ways in distance education. In the field of tourism and hospitality, virtual tours are produced to allow students to travel for study in digital copies of historical and culturally significant places. One example is My Hometown Project (Alizadeh & Hawkinson, 2021). Digital twins are used in the STEM fields to simulate engineering and biology concepts and facilitate a deeper understanding (David et al., 2018).

All of this is to say that digital twins are already present in various fields, and the more we use them, the more accurately we might build a better digital twin in the future. The trends in technology use are folding in on themselves to accelerate the blending of the digital and the physical, leading to more ethical and even existential questions (Hawkinson & Klaphake, 2020). The introduction of digital twins in educational contexts has the potential to bring several benefits and implications for the future. Some of these include:

• Personalized learning: Digital twins can provide a customized learning experience by analyzing students' strengths, weaknesses, and learning styles.
• Enhanced virtual reality experiences: Digital twins can enhance virtual reality experiences, enabling students to interact with realistic simulations in a variety of educational contexts.
• Improved assessment: Digital twins can provide real-time feedback and data analysis, allowing teachers to assess students' progress and identify areas where they need more support.
• Collaborative learning: Digital twins can facilitate collaboration and teamwork by enabling students to work together in virtual environments.
• Enhanced accessibility: Digital twins can make education more accessible, especially for students with disabilities who may have difficulty participating in traditional classroom settings.

Use Cases of Digital Twins in Education

A pilot study examined the design and implementation of a digital twin of a real location for leadership and diplomacy training in the form of a virtual learning environment (VLE) for Model United Nations (MUN) simulations. The VLE was designed as a highly customized WebVR experience and was created as a digital twin of the United Nations Security Council Chamber in New York. Seven students participated in a series of simulations in virtual reality to simulate a session for the UN Security Council. The study aimed to explore the affordances of WebVR as an online collaborative tool. The design of the VLE was documented, including the training and onboarding of participants, customizing simulation parameters, and observing and polling participation for acceptance and reactions. The study analyzed both the perspectives of a veteran MUN facilitator and an educational technologist to identify best practices and design principles for MUN simulations in VR. The findings highlight the need for iteration or re-design of activities from traditional MUN facilitation to align with the capabilities of VLEs (Mcgregor & Hawkinson, 2022).

Recreating digital twins of important historical artifacts is also a known popular use. VR Ban Dainagon Emaki, was developed to give access to Japanese Emaki (Panoramic Picture Scrolls) and address the challenge of students' low interest in Japanese literature and to expose students to the world of classic literature. The authors aimed to enhance students' learning experience by making use of VR technology to provide a more immersive and interactive environment. The VR app allows students to view the whole scroll, animate the characters, and view the Emaki together in a multiplayer experience. The authors used photos of the Emaki from the National Diet Library Digital Collections (Hawkinson et al., n.d.).

Methodology

To help gather signals to gain insights into possible trends of the use of digital twins, especially in educational contexts, three use cases from movies, music, and higher education were contrasted and compared.

An extensive literature review that compared three use cases of digital twins was conducted. Likely trends and issues in using digital twins in educational contexts moving to the future were highlighted.

ABBAtars

The Swedish group ABBA was formed by Agnetha Fältskog, Bjorn Ulvaeus, Benny Andersson, and Anni-Frid Lyngstad in Stockholm in 1972. Their album sales were in millions, and they toured the world but eventually split up in 1982. In 1989, Polygram purchased the ABBA catalog following ABBA’s break-up. A compilation album containing the group’s greatest hits was released in 1992 by the label. The album has sold over 30 million copies worldwide. They have used a variety of tools ever since to keep the band’s activity alive, such as musicals, movies, video games, or expositions, to keep people interested in the band and its brand. Over the years, they have become one of the best-selling bands of all time by maintaining their reputation and influence (Johansson, 2010).

The latest tool for ABBA to keep income flowing is the use of digital twins. The ABBA Voyage, a digital concert experience, will incorporate captured performances of ABBA while wearing a mocap suit. Virtual avatars will be accompanied by a 10-piece band performing in real-time remotely. Motion capture technology like that used in films like Star Wars and Lord of the Rings was used by the band to create their digital variants. ABBA used a picture of themselves in their motion capture suits to boost their press coverage. ABBA has created digital twins of themselves as they looked in their music prime and can now appear in concert in perpetuity and posthumously.

Carrie Fisher in ‘Star Wars

Star Wars, a series of very popular science fiction films starting in 1977, starred Carrie Fisher as Princess Leia. In December 2016, a prequel to the first 1977 movie was released, unfortunately Carrie Fisher had passed away prior to filming and could not reprise the role, so a digital twin was created to simulate the actress for a short appearance in the film. CGI has been used to animate dead celebrities in the past, but often to fill in a few scenes for films that had already begun shooting. As in the case of Carrie Fisher, the scenes were fabricated using existing footage. In the 2016 prequel called Rogue One, a digital twin was created using existing film of Carrie Fisher and motion capture of another actor was used to animate it. The result was a performance in film released after death, leaving questions to her estate about royalties and content rights (Naruniec et al., 2020). This has helped prompt legal questions about post-mortem publicity rights. Moreover, questions about the need for protection against digital resurrections that this technology invites, and its possible misappropriation have surfaced (Fontein, 2017).

Dead Professor

Students' shock was displayed when it was found that a professor that had passed away was 'teaching' an online course with previously recorded lectures (Elks, 2021). This was discovered when a student at a Canadian university tried to email this professor, only to find a link to a memorial web page. In this example, a university's online course materials, including video lectures, were used without informing students. This leads to questions about the ownership of lecture recordings or likeness rights for those in semi-public life (Brown, 2021). Some researchers have suggested that this may be the beginning of the end of the lecture and the lecturer as universities are seeking to leverage recorded lectures for online learning. As digital twins are introduced in lectures and classrooms, the exploitation of these digital versions may be more prevalent in the future, and ethical and legal issues around using these twins are yet to be worked out. Other perspectives include the eventual obsolescence of analog people in a digital university and a fear of teachers slowly being replaced with digital copies of themselves.

According to Hassan (2018), “We have unleashed a technological force - digitalization - that is inherent to what it means to be human because we have embraced digital technology so quickly and enabled it to pervade our university so fully.”

This trend may challenge the use of lectures by teaching staff and exacerbate the trend pushing teachers from delivering information to supporting or facilitating the understanding of it. Lectures are described as an anachronistic practice of a bygone age as a teacher-centered method of transferring knowledge from one human to another. This could imply that as these automated systems using digital twins as teachers become more prevalent, student-active engagement in collaboration, discussion, investigation, practice, and production will become more pronounced. The increased use of immersive technologies like augmented and virtual realities only points to making learning content and the teachers who deliver them immortalized with ever-so-detailed accuracy (Hawkinson et al., 2017).

Confidentiality Statement

No data from human participants were directly collected as a part of this research; therefore, no risk and liability is incurred in pursuing ethical human- based research. Instead, the methodology involved comparing a series of case studies with other prior research.

Discussion

To fully exploit the potential of digital twins, the above technologies will have to be converged. There will also be a need to address the rising issues of data collection, privacy, and digital rights poised to inform how tangential technologies are developed. To attain these objectives, more study into traditional data gathering and processing technologies, as well as the implementation of a digital twin communication interface, is required. To ensure user confidentiality and protect their private information, systems need to have security mechanisms that detect failures and incorrect data as well as provide recovery capabilities, all of which still need to be improved. Because there will be different realizations of a digital twin, interoperability of the technologies on a long-term basis will be imperative. Furthermore, digital twins must accommodate diverse cultures to facilitate worldwide collaboration. Legal issues must also be addressed, such as who is responsible for any wrongful activities attributed to digital twins and how much responsibility should be assigned to digital twins.

The digitization of the world could mean a digital twin of reality itself, allowing experimentation and simulations for learning like never before, but the ability to copy people, things, and places from the real world has educational merit. There are still many legal, ethical, safety, and privacy issues to be worked out. This study suggests that the benefits have great potential and should continue to be explored, but proceed with caution, especially with student use.

References

Alizadeh, M., & Hawkinson, E. (2021). Case study 10, Japan: Smartphone virtual reality for tourism education—A case study. In M. Alizadeh & E. Hawkinson (Eds.), Language Learning with Technology: Perspectives from Asia (pp. 211–222). Springer Singapore.

Brown, A. (2021). Who Owns Online Lecture Recordings? Higher Education Policy Institute. https://www.hepi.ac.uk/wp-content/uploads/2021/11/ Who-owns-online-lecture-recordings.pdf

David, J., Lobov, A., & Lanz, M. (2018, October). Learning experiences involving digital twins. In IECON 2018-44th Annual Conference of the IEEE Industrial Electronics Society (pp. 3681–3686). IEEE. https://doi.org/10.1109/IECON.2018.8591460

Elks, S. (2021, February 5). Analysis: Class Led by Dead Professor Spotlights COVID-Era Content Rights. Reuters. https://www.reuters.com/article/ us-global-tech-rights-analysis-trfn-idUSKBN2A521B

El Saddik, A. (2018). Digital twins: The convergence of multimedia technologies. IEEE multimedia, 25(2), 87–92. https://doi.org/10.1109/ MMUL.2018.023121167

Fontein, M. (2017). Digital Resurrections Necessitate Federal Post- Mortem Publicity Rights. J. Pat. & Trademark Off. Soc'y, 99, 481. https://heinonline.org/HOL/LandingPage?handle=hein.journals/ jpatos99&div=30&id=&page=

Hassan, R. (2018). Analogue people in a digital university. In R. Barnett, & M.A. Peters (Eds.), The idea of the university, 2.

Hawkinson, E., Alizadeh, M., Ijiri, A., Yano, K., McGregor, A., Klaphake, J., Yokoyama, E., & Noxon, C. (n.d.) Immersive Technology Uses in Interactive Media: A Collection of Case Studies. https://papers.iafor.org/ wp-content/uploads/papers/kamc2021/KAMC2021_54222.pdf

Hawkinson, E., & Klaphake, J. (2020, June). Work-in-Progress—Legal and Ethical Issues in Immersive Education. In 2020 6th International Conference of the Immersive Learning Research Network (iLRN) (pp. 305–307). IEEE. https://doi.org/10.23919/iLRN47897.2020.9155135

Hawkinson, E., Mehran, P., & Alizadeh, M. (2017). Using MAVR to bring new dimensions to the classroom. The Language Teacher, 41(3), 30–32.https://jalt-publications.org/files/pdf-article/41.3tlt-wired.pdf

Jaensch, F., Csiszar, A., Scheifele, C., & Verl, A. (2018, November). Digital twins of manufacturing systems as a base for machine learning. In 2018 25th International conference on mechatronics and machine vision in practice (M2VIP) (pp. 1–6). IEEE. https://doi.org/10.1109/M2VIP.2018.8600844

Johansson, O. (2010). Beyond ABBA: the globalization of Swedish popular music. FOCUS on Geography, 53(4), 134–141. https://doi.org/10.1111/ j.1949-8535.2010.00016.x

Kamble, S. S., Gunasekaran, A., & Gawankar, S. A. (2018). Sustainable Industry 4.0 framework: A systematic literature review identifying the current trends and future perspectives. Process safety and environmental protection, 117, 408–425. https://doi.org/10.1016/j.psep.2018.05.009

Li, X., Liu, H., Wang, W., Zheng, Y., Lv, H., & Lv, Z. (2022). Big data analysis of the internet of things in the digital twins of smart city based on deep learning. Future Generation Computer Systems, 128, 167–177. https:// doi.org/10.1016/j.future.2021.10.006

Mcgregor, A., & Hawkinson, E. (2022, May). WebVR in the Facilitation of Model United Nations Simulations. In 2022 8th International Conference of the Immersive Learning Research Network (iLRN) (pp. 1–6). IEEE. https:// doi.org/10.23919/iLRN55037.2022.9815998

Naruniec, J., Helminger, L., Schroers, C., & Weber, R. M. (2020, July). High- resolution neural face swapping for visual effects. Computer Graphics Forum, 39(4), 173–184. https://doi.org/10.1111/cgf.14062

Ravì, D., Wong, C., Deligianni, F., Berthelot, M., Andreu-Perez, J., Lo, B., & Yang, G. Z. (2016). Deep learning for health informatics. IEEE Journal of Biomedical and Health Informatics, 21(1), 4–21. https://doi.org/10.1109/ JBHI.2016.2636665

Rosen, R., Von Wichert, G., Lo, G., & Bettenhausen, K. D. (2015). About the importance of autonomy and digital twins for the future of manufacturing. Ifac-Papersonline, 48(3), 567–572. https://doi.org/10.1016/j. ifacol.2015.06.141

Stark, R., Fresemann, C., & Lindow, K. (2019). Development and operation of Digital Twins for technical systems and services. CIRP Annals, 68(1), 129–132. https://doi.org/10.1016/j.cirp.2019.04.024

Tao, F., Qi, Q., Wang, L., & Nee, A. Y. C. (2019). Digital twins and cyber–physical systems toward smart manufacturing and industry 4.0: Correlation and comparison. Engineering, 5(4), 653–661. https://doi.org/10.1016/j. eng.2019.01.014

Van der Valk, H., Haße, H., Möller, F., Arbter, M., Henning, J. L., & Otto, B. (2020, August).A Taxonomy of Digital Twins.In AMCIS.https://www.researchgate. net/profile/Hendrik-Van-Der-Valk/publication/341235159_A_ Taxonomy_of_Digital_Twins/links/5eb54f1892851cd50da1a442/A- Taxonomy-of-Digital-Twins.pdf

Xu, L. D., Xu, E. L., & Li, L. (2018). Industry 4.0: state of the art and future trends. International Journal of Production Research, 56(8), 2941–2962. https://doi.org/10.1080/00207543.2018.1444806

International Journal on Open and Distance e-Learning, Vol. 8 No. 2, December 2022 Issue