The Design and Evaluation of a Conference Exhibit in the Metaverse

Joshze Rica Esguerra¹, Hannah Gabriella Macaldo², Allan Nuñez³, Roberto Figueroa Jr.⁴, Lexter Mangubat⁵

¹Independent Researcher, Philippines, [email protected]
²Student, University of the Philippines Open University, Philippines, [email protected]
³Administrative Aide VI, University of the Philippines Open University, Philippines, [email protected]
⁴Associate Professor, University of the Philippines Open University, Philippines, [email protected]
⁵Information Systems Researcher II, University of the Philippines Open University, Philippines, [email protected]

Abstract

Keywords: metaverse, virtual exhibit, VR-user experience, situational interest

Introduction

Background

The University of the Philippines Open University (UPOU) is a pioneer in the field of online teaching and learning and has been instrumental in promoting open and distance education in the Philippines. Annually, the university hosts a conference that is dedicated to open and distance e-learning, a practice that dates to the inaugural National Conference on Open and Distance e-Learning (NCODeL) in 1998. Following this milestone, the first International Conference on Open and Distance e-Learning (ICODeL) was launched in 2012. These conferences provided a platform for policymakers, professionals in open and distance learning, and e-learning scholars to share their experiences, exchange ideas, and explore new possibilities for ODeL development.

In 2021, the UP Open University held ICODeL entirely online due to pandemic restrictions. VFairs (Figure 1) was used to allow participants to communicate, collaborate, and engage in the conference. However, while this solution offered notable advantages, it was not able to provide an immersive metaverse experience and came at a high cost to the university. Notably, visitors were confined to a static view where they could not move nor look left and right. To navigate, users can only click on the text where they will be redirected to the room of that presentation.

Figure 1

The lobby of the ICODeL 2021 VFairs platform

Recognizing these limitations, NCODeL 2022 observed a strategic shift in approach. The university opted for an in-house user portal coupled with Zoom as the conferencing software, and integrated FrameVR as the metaverse exhibit platform. Within this virtual world, conference sponsor materials and the university’s select projects and other learning materials were displayed. This study aims to understand how this metaverse exhibit was perceived by conference participants and the public, as it represents a new frontier in conference technology.

Virtual Exhibits

Despite the growing interest in virtual exhibits, there is a notable gap in understanding how specific factors influence user engagement and satisfaction in these digital environments. Previous studies have primarily focused on the technical aspects and general usability of virtual reality (VR) applications in cultural and educational settings. However, there is limited research exploring the relationship between situational interest and the sense of presence in virtual exhibits, and how these elements contribute to overall user engagement.

This study aims to fill this gap by examining the correlation between situational interest and the sense of presence and identifying key features that enhance the user experience in virtual exhibits. By addressing these aspects, this research provides valuable insights for the design and development of more engaging and effective virtual experiences.

Purpose of the Study

This research seeks insights into metaverse virtual experiences, emphasizing navigation, engagement, and collaborative design, focusing on exhibit attendees' situational interest and sense of presence in the absence of a VR Head-mounted Display (HMD). To achieve these objectives, the following research questions were formulated:

1. What is the relationship between the sense of presence experienced by participants and their situational interest in navigating the virtual exhibit without the use of VR HMDs?
2. Based on participant comments, what specific points, areas, or features of the virtual exhibit were considered interesting or noteworthy?

Review of Related Literature

The definition of the metaverse is evolving as new domains discover its utility and contextualize this emerging technology. However, this definition enumerates the various layers and elements that are usually present in the metaverse (Mystakidis, 2022):

The Metaverse is the post-reality universe, a perpetual and persistent multiuser environment merging physical reality with digital virtuality. It is based on the convergence of technologies that enable multisensory interactions with virtual environments, digital objects and people such as virtual reality and augmented reality (AR). Hence, the Metaverse is an interconnected web of social, networked immersive environments in persistent multiuser platforms. It enables seamless embodied user communication in real-time and dynamic interactions with digital artifacts. Its first iteration was a web of virtual worlds where avatars were able to teleport among them. The contemporary iteration of the Metaverse features social, immersive VR platforms compatible with massive multiplayer online video games, open game worlds, and AR collaborative spaces.

According to Kai Liao et al. (2021), the metaverse connects physical and virtual in a world where social, financial, and cultural sports end. It means that the metaverse, which is often associated with immersive digital environments and interactions, is shaping our world in ways that affect multiple aspects of life, especially in education.

Metaverse in Education

The metaverse is ideal for schools that focus on e-learning and offer all their training online, as it enhances the educational experience (Contreras et al., 2022). The learning process can be humanized by incorporating the metaverse into these educational systems, encouraging greater involvement and interaction between students and teachers. This transition toward a metaverse education could alter how we learn and develop our talents, providing a more engaging and dynamic approach to learning.

As the world advances, the presence of the metaverse in education offers a fresh perspective on educational technology. Through the metaverse, training programs and project initiatives that are not feasible in the real world can be conducted without the constraints of time, space, or unpredictable circumstances. Its features provide programs with effective learning support. However, the lack of relevant technological infrastructure can pose significant challenges throughout the learning process. With the increasing amount of research and development regarding the metaverse in today’s generation, these challenges can be brought to light (Hwang & Chien, 2022).

According to Azoury and Hajj (2023), while the concept of the metaverse is not new, its widespread adoption in education gained momentum with the onset of the COVID-19 pandemic. Institutions are now keen to enhance the educational system through the metaverse due to the numerous opportunities it presents. Technologies like AR and VR have the potential to help students improve their knowledge. These technologies can create a sense of presence in an unusual frame (a phenomenon known as digital frame possession) and foster a feeling of being present with others (known as co-presence).

Khalil et al. (2023) investigated the adoption of an educational metaverse in the context of higher education in Pakistan. The researchers used a mixed- methods approach, employing questionnaires and interviews to gather data from both students and teachers. Their findings indicated a positive reception among teachers and students toward integrating the educational metaverse into their teaching and learning practices. NVivo software was employed to code and extract thematic insights from the interview data.

Evaluating Virtual Exhibits

Recent studies have emphasized the importance of usability and user experience in VR applications for digital museums and cultural heritage exhibitions. Othman et al. (2021) conducted a usability evaluation of a VR and non-VR virtual tour application for a living museum, finding that the VR version had higher mean scores in usability. The study also highlighted the need for a nuanced understanding of how different user demographics interact with VR applications. Further exploring the user experience in VR settings, Chung et al. (2024) compared user experiences in reality-based (high representational fidelity level) and virtuality-based (surreal, low representational fidelity level) VR exhibition environments, finding that while the familiarity of the reality- based environment provided more comfort and concentration for participants, the newness and openness of the virtuality-based environment caused active movement.

Li et al. (2022) focused on online virtual museum tours and established a heuristic evaluation scale to assess aspects like authenticity, interaction, and navigation. Their findings suggested that while visual authenticity was high, behavioral authenticity and navigation needed improvement to enhance the overall user experience. Similarly, Pei et al. (2023) developed a comprehensive framework to evaluate user experience in VR-based digital museums, revealing that VR significantly enhances learning effectiveness compared to traditional desktop interfaces. This reinforces the notion that immersive technologies can significantly impact educational outcomes, providing a more engaging user experience. Complementing these insights, Gong et al. (2024) proposed a detailed user experience model for VR cultural heritage exhibitions, using grounded theory to combine qualitative and quantitative approaches. They emphasized the importance of designing user experiences that account for the unique interactions in VR environments. Together, these studies underscore the critical role of well-designed VR interfaces in enhancing user engagement and educational effectiveness in virtual exhibits.

Situational Interest and Presence

Literature in educational media cited relationships between spatial presence and various motivational variables. One motivational construct that has been extensively researched is situational interest. In the initial stages of research in motivation, interest was greatly overlooked. Recently, however, this focus has shifted. In 1999, Krapp categorized two primary types of interest: personal and situational. Figure 2 depicts the three approaches to interest research as outlined by Schunk et al (2014).

Figure 2

Approaches to interest research from Schunk et al. (2014)

Among these variables, situational interest was used as a motivational variable in a theoretical framework proposed by Figueroa in 2023.

Methodology

Selecting the Platform

There has recently been a rise in the development of virtual reality technologies, specifically in the realm of virtual collaborative spaces. Such technologies also gained significant traction during the pandemic.

One such platform considered for the virtual exhibit was Sansar, a social VR platform developed by Linden Lab. However, it was determined that Sansar presented certain limitations for our purposes. Firstly, it requires users to download and install the software on a Windows computer, thereby excluding potential participants using operating systems other than Windows. Additionally, Sansar's minimum system specifications require a dedicated GPU due to its higher graphical demands, potentially posing a barrier for individuals with less powerful setups. This would potentially restrict inclusivity compared to platforms that can be accessed directly through a web browser.

Another option was Mozilla Hubs (Figure 3), a versatile and browser-based virtual collaboration platform. Unlike Sansar, users can join directly through their web browsers, eliminating the need for software downloads or specific operating system requirements. This allowed participants to engage with the virtual exhibit on a wider range of devices. However, at the time of development, Mozilla Hubs had a limitation in that it could only accommodate up to 25 visitors at one time, potentially impacting its suitability for larger gatherings.

Figure 3

The user interface of Mozilla Hubs in desktop mode

Ultimately, the university opted to use FrameVR, a browser-based immersive multi-user virtual platform created by Virbela, and has been used by reputable institutions such as Microsoft, University of Massachusetts, and MIT Media Lab. Its cost-effective features and web browser compatibility provided several advantages. At the time of development, it could be accessed by up to 40 users, and the prebuilt worlds and allowed files were suited to the needs of the virtual exhibit.

Building the Exhibit in the Metaverse

The exhibit was created collaboratively in the Metaverse using FrameVR. Each team member was granted editor access to the space to work on. The team worked together to build the entire environment, starting with the selection of a space to build the exhibit. FrameVR offers different template spaces that users can choose from, as seen in Figure 3, which shows the initial environment of the exhibit.

Designing the Environment

The team assigned zones to each space in the Metaverse and began adding assets to the environment, including images, texts, video materials, and 3D models. However, as the team added more big assets like video materials and 3D models, they noticed that the site became laggy. To reduce the server load, the team decided to link the video materials directly to their thumbnail instead of embedding them in the environment. This was a valuable lesson learned during the exhibit-building process.

Integrating the Content

A dedicated area was allocated for sponsors and partner organizations to showcase their materials. Previous issues of the university's journal, the International Journal on Open and Distance e-Learning (IJODeL), were available to be read in the metaverse, serving as a valuable knowledge resource for visitors.

The exhibit also highlighted the creativity of UPOU students, featuring the videos and artworks they created in their multimedia courses. Additionally, visitors can watch educational video resources produced by the UPOU Multimedia Center (now Educational Media Production). VR tours created by the Immersive Open Pedagogies Program (IOP) are also featured, allowing visitors to visit various locations in the Philippines virtually.

To enhance the users' immersive experiences, interactive features and activities were added. Visitors could vote for their favorite student-submitted posters. They could also express their creativity on a collaborative drawing board. Conference sessions were streamed for virtual attendees to watch together (Figure 4), and they could even share their own screens. This thorough planning and inclusion of interactive elements made the educational experience enjoyable and enriching for everyone in the metaverse exhibit.

Figure 4

Streaming the conference presentations in the virtual exhibit

Participants

The study participants were NCODeL 2022 conference attendees and UP Open University students. After exploring the Metaverse virtual exhibit, they were requested to complete a questionnaire via Google Forms. The participants who consented to the survey are considered to be the participants of the study.

Data Collection and Analysis

The quantitative and qualitative data were collected using Google Forms. Participants were asked to answer the survey upon visiting the conference exhibit. The form aims to gather the situational interest, sense of presence, and the features of the exhibit that attract the participants the most. A total of 44 respondents participated in the study.

The quantitative data were then run through JASP, an open-source software that runs through R-code. To know about the relationship between sense of presence and situational interest, Pearson’s Correlation Test was used. To identify features and aspects from which the participants are attracted the most, the qualitative data went through reflexive thematic analysis following Braun and Clarke’s approach (Byrne, 2022).

Findings

Relationship between Sense of Presence and Situational Interest

Table 1

Items Used for Measuring the Situational Interest and Presence

Item	Mean	SD
Situational Interest
The exhibit was entertaining	4.591	0.583
I did not enjoy navigating the exhibit	2.023 (-) 3.997 (+)	1.338
I liked the exhibit	3.997	1.338
The topics in the exhibit were interesting	4.568	0.625
Presence
How much did you feel that you were in the exhibit and not just looking at a photo?	8.273	1.318

Table 1 shows the actual mean score and standard deviation of the variables relating to Situational Interest and Presence. Table 2 shows the results of the Pearson's correlation test between situational interest (SI) and presence. The test revealed a significant correlation between the two variables, with a p-value of less than 0.001. The r-value of 0.572 indicates a positive linear relationship between SI and presence.

Table 2

Pearson's Correlation Test Result for Situational Interest and Presence

	r	p-value	95% CI
Correlation (r)	0.572	<0.001	[0.332,0.743]

Note. CI = Confidence Interval.

Interesting Feature and Aspect of the Exhibit

Table 3

Themes that Emerged as Interesting Features/Points/Areas of the Exhibit

Feature/Point/Area	Frequency
Immersive experience	12
Interactive elements	9
Artistic quality	8
UPOU Virtual Tour	6
Educational content	5
Technical issues	5
Navigation challenges	4
Geographical accessibility	3
Comparison to VR/Gaming	3
Cultural references (e.g., BTS)	2
Navigational improvements	2
Balance of academic and fun	2
Individual exploration	2
References to other experiences (e.g., VR NFT, video games)	2
Desire for future revisions	2
Clear instructions/tutorials	2
Variety of formats/features	2

The analysis of participant feedback yielded several key themes, shedding light on the aspects that garnered attention and appreciation from the participants.

Immersive Experience

The predominant theme emerging from participant comments was the appreciation for the immersive experience offered by the virtual exhibit. Participants expressed a sense of presence and engagement within the virtual environment, using terms such as "excellent," "fascinating," and "entertaining" to describe their overall encounter.

Interactive Elements

The interactive nature of the exhibit emerged as a crucial point of interest. Participants valued features such as playable audio, links to works, and various interactive formats. This suggests that interactivity significantly contributed to the positive perception of the virtual exhibit, aligning with the growing emphasis on user engagement in digital environments.

Artistic Quality

The aesthetic appeal of the virtual exhibit was a noteworthy aspect, with participants lauding its "simple," "classic," and "well-decorated" attributes. The positive reception of the artistic quality underscores the importance of visual design in creating a favorable user experience.

UPOU Virtual Tour

Specific mention of the UPOU Virtual Tour indicated that participants found this feature particularly noteworthy. The ability to virtually explore distant locations, as exemplified by the UPOU Virtual Tour, resonated positively, offering an accessible and immersive experience for those unable to visit the physical locations.

Educational Content

Comments reflected a positive response to the educational content embedded in the virtual exhibit. Participants perceived value in the informative nature of the entries, emphasizing the balance between serious academic content and interactive, engaging elements.

Technical Issues and Navigation Challenges

While the overall feedback was positive, a subset of participants noted technical issues and navigation challenges. Addressing these concerns is pivotal for optimizing user experience, as smooth navigation and functionality are imperative for the success of virtual exhibits.

Geographical Accessibility

The appreciation for the virtual exhibit providing access to locations like Visayas underscores its potential to bridge geographical gaps and offer a unique experience to participants regardless of their physical location.

Comparison to VR/Gaming

References to other experiences, such as VR non-fungible tokens (NFT) and video games, suggest that participants drew comparisons, potentially influencing their expectations and experiences with the virtual exhibit. These external references provide context for understanding participants' perspectives.

Cultural References

The mention of cultural references, such as BTS, added an unexpected and exciting element to the exhibit. The incorporation of culturally relevant content contributes to the diversity and richness of the virtual space.

Desire for Future Revisions

Expressions of a desire for future revisions indicate a participant’s interest in continuous improvement and refinement of the virtual exhibit. These comments emphasize the evolving nature of digital experiences and the necessity of continuous development.

Discussion

Relationship between Sense of Presence and Situational Interest

Our study reveals key insights into how situational interest (SI) relates to the sense of presence within a virtual exhibit. The high mean score (4.591) for the statement "The exhibit was entertaining" suggests that participants generally found the virtual exhibit engaging and enjoyable. The positive correlation (r = 0.572) further supports this, indicating that those who enjoyed the exhibit more also found navigating it to be less of a hindrance.

The positive linear relationship between SI and presence (r = 0.572) is indicative of a strong connection between the perceived interest in the exhibit's content and the feeling of being present within the virtual environment. This aligns with existing literature suggesting that heightened situational interest can lead to an increased sense of presence in virtual environments (Slater & Wilbur, 1997). The statistically significant correlation (p < 0.001) further emphasizes the robustness of this relationship.

The high mean score (8.273) for the statement assessing the feeling of being in the exhibit rather than merely observing a photo supports the notion that participants experienced a strong sense of presence. This observation aligns with previous research that underscores the importance of presence in creating positive user experiences in virtual environments (Biocca et al., 2003).

The positive relationship between SI and presence has practical implications for the design of virtual exhibits. Focusing on creating content that elicits situational interest may enhance the overall sense of presence, thereby improving user engagement and satisfaction.

Interesting Feature and Aspect of the Exhibit

The analysis of participant feedback revealed several key themes that shed light on the aspects of the virtual exhibit that captured participants' attention and appreciation. The immersive experience emerged as a central theme, with participants reporting a strong sense of presence and engagement. This supports the positive correlation identified in the first research question, highlighting the crucial role of immersion in fostering both interest and presence.

The interactive elements, highlighted by the participants, indicate a growing emphasis on user engagement in digital environments. Incorporating interactive features, such as playable audio and links to works, can significantly contribute to the positive perception of virtual exhibits.

Artistic quality received positive feedback, emphasizing the importance of visual design in creating a favorable user experience. This finding underscores the need for virtual exhibits to prioritize aesthetics and design to enhance user satisfaction.

The UPOU Virtual Tour, mentioned as a noteworthy feature, indicates the potential of virtual exhibits to provide accessible and immersive experiences, particularly for those unable to visit physical locations. This finding supports the idea that virtual exhibits can bridge geographical gaps and offer unique experiences to a diverse audience.

While the overall feedback was positive, the identification of technical issues and navigation challenges highlight the importance of addressing such concerns to optimize user experience. Smooth navigation and functionality are crucial for the success of virtual exhibits.

References to other experiences, such as VR NFT and video games, suggest that participants drew comparisons, influencing their expectations and experiences with the virtual exhibit. Understanding these external references can provide valuable insights for designers and curators.

The inclusion of cultural references, like BTS, adds an exciting and unexpected element to the exhibit, contributing to the diversity and richness of the virtual space. This finding emphasizes the potential of virtual exhibits to incorporate culturally relevant content.

Expressions of a desire for future revisions indicate a participant's interest in continuous improvement. This feedback emphasizes the evolving nature of digital experiences and the necessity of continuous development to enhance virtual exhibits over time.

Conclusion

Our research offers valuable insights into the connection between situational interest and the sense of presence in virtual exhibits, highlighting its significant features. The positive correlation between these variables suggests that fostering situational interest can contribute to an enhanced sense of presence, thereby improving user engagement. The analysis of participant feedback further identifies key features and aspects that contribute to a positive user experience, including immersive experiences, interactive elements, artistic quality, virtual tours, educational content, and cultural references. However, attention must be given to addressing technical issues and navigation challenges to optimize user satisfaction. The findings of this study contribute to the expanding body of literature on virtual exhibits and provide practical implications for their design and development. Future research could investigate the long-term impact of these strategies on improving user engagement in virtual exhibits.

References

Azoury, N., & Hajj, C. (2023). Perspective Chapter: The Metaverse for Education. In IntechOpen eBooks. https://doi.org/10.5772/intechopen.109307

Biocca, F., Harms, C., & Burgoon, J. K. (2003). Toward a More Robust Theory and Measure of Social Presence: Review and Suggested Criteria. Presence: Teleoperators and Virtual Environments, 12(5), 456–480. https://doi.org/10.1162/105474603322761270

Byrne, D. (2022). A worked example of Braun and Clarke's approach to reflexive thematic analysis. Quality & quantity, 56(3),1391–1412. https:// doi.org/10.1007/s11135-021-01182-y

Chung, S. J., Kim, S. Y., & Kim, K. H. (2024). Comparison of visitor experiences of virtual reality exhibitions by spatial environment. International Journal of Human-computer Studies, 181, 103145. https://doi.org/10.1016/j. ijhcs.2023.103145

Figueroa, R. J. B. (2021). Spatial Presence, Situational Interest, and Ideal Future-Self in Virtual Reality among Adult Online Learners in the Philippines, Japan, and France. [Doctoral Dissertation) International Christian University Graduate School of Arts and Sciences]. https://doi. org/10.34577/0000005413

Figueroa, R. J. B. (2023). Immersive capability and spatial presence in virtual reality photo-based tours: implications for distance education. Asia Association of Open Universities Journal, 18(3). https://www.emerald. com/insight/2414-69994.htm

Figueroa, R. J. B., Gil, F. A. P., & Taniguchi, H. (2022). Piloting Virtual Reality Photo-Based Tours among Students of a Filipino Language Class: A Case of Emergency Remote Teaching in Japan. Avant, 13(2), 1-26. https://arxiv.org/pdf/2301.01904.pdf

Gong, Q., Zou, N., Yang, W., Zheng, Q., & Chen, P. (2024). User experience model and design strategies for virtual reality-based cultural heritage exhibition. Virtual Reality, 28(2). https://doi.org/10.1007/s10055-024- 00942-z

Hwang, G., & Chien, S. (2022). Definition, roles, and potential research issues of the metaverse in education: An artificial intelligence perspective. Computers & Education: Artificial Intelligence, 3, 100082. https://doi. org/10.1016/j.caeai.2022.100082

Khalil, A., Haqdad, A., & Sultana, N. (2023). View of Educational Metaverse For Teaching And Learning In Higher Education of Pakistan. Journal of Positive School Psychology. 7(2), 1183-1197. https://www.journalppw. com/index.php/jpsp/article/view/15888/10259

Li, J., Nie, J., & Ye, J. (2022). Evaluation of virtual tour in an online museum: Exhibition of Architecture of the Forbidden City. PloS One, 17(1), e0261607. https://doi.org/10.1371/journal.pone.0261607

Mystakidis, S. (2022). Metaverse. Encyclopedia, 2(1), 486–497. https://doi. org/10.3390/encyclopedia2010031

Othman, M. K., Nogoibaeva, A., Leong, L. S., & Barawi, M. H. (2021). Usability evaluation of a virtual reality smartphone app for a living museum. Universal Access in the Information Society, 21(4), 995–1012. https:// doi.org/10.1007/s10209-021-00820-4

Pei, X., Fu, S., & Jiang, T. (2023). An empirical study on user experience evaluation of VR interface in digital museums. Data and Information Management, 7(4), 100057. https://doi.org/10.1016/j.dim.2023.100057

Rahman, K. R., Shitol, S. K., Islam, M. S., Iftekhar, K. T., & Saha, P. (2023). Use of Metaverse Technology in Education Domain. Journal of Metaverse, 3(1), 79–86. https://doi.org/10.57019/jmv.1223704

Slater, M., & Wilbur, S. (1997). A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments, 6(6), 603–616. https://doi.org/10.1162/pres.1997.6.6.603

International Journal on Open and Distance e-Learning, Vol. 10 No. 1, June 2024 Issue