Google Analytics for User-Centered Design: A Case Study In Open and Distance e-Learning

Kamran Mir¹*, Ali Tahir², and Hajrah Amir³

¹PhD Scholar, National University of Sciences and Technology, Pakistan, [email protected]
²Associate Professor, National University of Sciences and Technology, Pakistan, [email protected]
³MS Scholar, National University of Sciences and Technology, Pakistan, [email protected]
* Corresponding author

Abstract

Keywords: Google Analytics, learner-centered design, open and distance learning

Introduction

In recent years, Open and Distance Learning (ODL) has become increasingly popular as an alternative mode of education for individuals who cannot attend traditional brick-and-mortar schools. This online and distance education mode uses technology to provide students with access to course materials, assignments, and assessments, regardless of their physical or geographical location. However, ensuring a high-quality online learning experience is a challenge for educators, as they must balance the need for flexibility and accessibility along with maintaining student engagement and motivation.

To address this challenge, Learning Analytics (LA) has emerged as a powerful instrument for tracking and analyzing student behavior, performance, and engagement in online learning environments. Similarly, Google Analytics (GA) is a widely used web analytics tool that can provide valuable insights into how students are using online resources, such as learning management systems (LMS) and course materials. GA can help educators in making data-driven decisions to improve the online learning experience, such as identifying which materials are most popular and effective and which areas need improvement, as well as in monitoring student progress.

Furthermore, learner-centered design (LCD) is an instructional design approach that focuses on the needs, motivations, and experiences of learners to create effective learning experiences. LCD emphasizes active learning, personalization, and technology use to support individualized learning experiences. Learner- centered is an approach or method of teaching from the teacher’s perspective and process of learning from the student’s perspective which encourages them to think critically and reflect on what they are learning and how they are learning (Students at Center, 2016). In practice, LCD involves key elements like assessment of learner needs which involves understanding the needs, motivations, and prior experiences of learners. These parameters are critical in the LCD process. Learner-centered design demands repetitive design and development of tasks and supporting students in their learning process and activities (Jackson et. al., 1995). The objectives of the learning experiences should be defined by keeping learner requirements in mind. LCD emphasizes active learning, where learners engage in hands-on, experiential activities that allow them to apply what they have learned. LCD approaches often involve personalizing the learning experience for each learner. Different methods can be used to achieve different learning goals. To enhance the LCD approach in online learning, Google learning analytics and spatial learning data can be used to collect and analyze location-based data about student behavior, activity, and performance within physical spaces, such as classrooms or libraries.

Learner-centered framework as illustrated in Figure 1 shows the overall context that covers the four primary components including learner personalization, competency, ownership, and location.

Figure 1

Student-centered Design Framework

Note. Copyright (2016) by Students at Center

Personalization is one of the important factors of a student-centered design framework. It involves the modification of the learning content and strategy as per the needs of the learner and the use of context and localization. This is generally called personalized learning teaching strategy. This also includes the personal interests of an individual student or a group of students. The example of personalization includes giving contextualized examples of generalized concepts according to the local sociopolitical environment. Personal interest also impacts while giving personal class assessments.

The second important factor is competency-based learning, which means making the learning and teaching process competency-based. The learning content, tasks, assessments, and activities should relate to the prevailing competencies of the learner. For instance, when a learner is part of a group that lacks physical strength, they might not be assigned tasks or activities demanding physical exertion or potentially impeding their learning progress. Similarly, the group assessments must be according to the individual or class’s overall competency.

Ownership is another important factor in which the purpose of learning and teaching depends on developing student ownership. The learning content usually in a classical learning environment comes from the teacher’s side only. However, having ownership encourages collaboration and teamwork. It is natural to have a sense of ownership when the content is also created by the learners. Wikipedia has a similar concept where a learner’s answer is not only evaluated by the teachers, but all the learners can read and give feedback to the answers given by a single student which will increase the motivation of students and their sense of ownership.

Lastly, conventional face-to-face education is dependent on space and time. However, in recent times, information and communication technologies can be effectively used as an alternative to traditional face-to-face teaching methods by making them independent of time and space. Online and distance learning is one of the prominent examples of this mode.

Likewise, in these theoretical frameworks, data analytics is one of the applied techniques which is being used in many industries to make better informed decisions by visualizing the data. In education, LA is a very popular term that is used to collect, measure, and analyze the learner’s data to make suitable developments in learning and teaching designs.

This study will focus on answering research questions regarding how GA can be helpful in making learner-centered application design and which factors are critical. Therefore, this research paper aims to explore the use of GA and spatial learning analytics in the context of learner-centered design for open and distance e-learning. The paper will provide a case study of an ODL institute that illustrates how these tools can be used to gather data on student behavior, performance, and engagement, and how this data can improve decision-making to enhance the online learning experience. Additionally, this research paper will contribute to the existing body of literature on the use of LA in education and provide insights into how LCD can be applied in the online learning context.

Objectives

The main objectives of this research study are:

1. To analyze user analytics data of an online student support system using Google Analytics to identify key factors important for learner- centered application design, including user system language, geo- location, browser and operating system, and user device category; and
2. To propose design attributes for future applications based on the analysis of user analytics data, to make them more personalized and user-centered, taking into consideration the identified factors such as user device, location, language, and operating system.

Literature Review

User personalization is one of the critical factors in several domains. For example, web links recommendation (Atahan & Sarkar, 2011), personalized blogs (Liu et al, 2014), personalized movies (Ying et al. 2006), personalized news (Shapira et al, 2009), personalization in maps (Tahir et al, 2010; Tahir et al, 2012) and personalized emails (Ansari & Mela, 2003) are some of the common application areas of user personalization. However, there are relatively fewer studies on learner-centered approaches using GA.

Personalization or user-centric approach is not a very new concept, historically, the collective work of different historic personalities of the 17th and 18th centuries like John Dewey, Jean Piaget, Lev Vygotsky, Al-Ghazali, and Shah Waliullah focused on student learning behavior and teaching methods which was not given the same term as student-centered at that time. However, all these concepts presented by these historic personalities had a similar type of domain which further led to teaching and learning approaches that involve student-centric techniques.

Developing a learner-centered design is not easy as it looks. This personalized design requires a lot of effort and planning. The major challenge is the arrangement of the content and converting them as per the requirements of the learner. The new idea must be incorporated with new activities. It may take a while to convert from traditional method to learner-centered approach. Nevertheless, this develops self-confidence and motivation in students to create their own approach to learning (Morphew, 2012).

There is a need to apply this approach in both offline and online education. Most teachers had a view that factors like problem-solving, enthusiasm, and peer learning had key roles in improving the learning process (Ertmer et al., 2012).

A study examines how computer and internet technology have made learning analytics a new discipline in online and distance education. To improve the quality of higher education, LA includes analyzing vast amounts of educational data to uncover trends and structures connected to students. The online environment enables thorough computer-generated logs of learner interactions, including content access, development, sources used, and sharing activities. Web analytics, academic analytics, educational data mining, and institutional intelligence are all included in the interdisciplinary discipline of LA. LA studies can make use of a variety of data generated by environments and tools, including LMS, social network apps, computer logs, games, simulations, video sharing platforms, and search engines. Data visualization, in-depth research, and statistical methods are also required for the examination of huge data (Firat & Yuzer, 2016).

Another study provides practical guidance for learning designers to create effective and engaging learning experiences. The study emphasizes the importance of considering the needs and perspectives of learners throughout the design process. It introduces a framework that outlines five stages of learner-centered design (LCD) and presents various LCD methods, such as user interviews and usability testing. The authors highlight the iterative nature of LCD, encouraging designers to continuously evaluate and refine their designs based on user feedback. They also highlight the importance of collaboration and interdisciplinary teamwork in implementing LCD in learning design. This research offers practical methods to incorporate LCD principles and enhance the learning experiences of the learners (Schmidt et al., 2020).

Another study by Vesin et al. (2018) focuses on the application of LCD and analytics in an adaptive learning system within smart environments. The study emphasizes the importance of understanding learners' needs and preferences to develop effective personalized learning systems. Through a case study, the authors showcase the iterative design process and implementation of an adaptive learning system that leverages data analytics to provide personalized learning experiences. The findings highlight the benefits of LCD and analytics in enhancing the learning experience, including personalized recommendations and adaptability to individual learner needs. The research contributes valuable insights into optimizing learning systems within smart environments through continuous evaluation and improvement based on user feedback and data analysis.

A study by Kilis and Gülbahar (2016) explores the application of learning analytics in distance education through a systematic literature review. This study highlights the importance of leveraging data from online learning platforms to improve educational practices and student outcomes. It identifies three key areas of focus for LA in distance education: student performance, engagement, and instructional design. The study emphasizes the potential benefits of LA, such as early identification of at-risk students and personalized learning experiences. However, it also highlights challenges related to ethics, data privacy, and data interpretation. The authors stress the need to address these challenges and provide support for educators to effectively utilize LA. The output of this research also offers valuable insights into the current landscape of LA in distance education and its implications for research and practice.

A study investigates user habits on academic education platforms using GA data. The study aims to gain insights into user behavior and preferences to improve the design and effectiveness of online education platforms. The study highlights the significance of analyzing user habits for optimizing educational platforms and enhancing user experience. By utilizing GA, the researchers gathered valuable data on user engagement, navigation patterns, and content consumption. The findings provide meaningful information about the preferences, interests, and learning patterns of users, which can be used to tailor educational content and enhance platform usability. The study contributes to the field of online education by offering practical insights that can inform the development of more user-centric and personalized learning experiences (Özen et al., 2014).

One example of the use of GA in education is a study conducted by Abeysekera and Dawson (2015), who used GA to analyze the usage patterns of an online learning system. They found that students who used the system more frequently and for longer periods of time tended to achieve higher grades, suggesting that personalized applications designed to encourage greater use of online learning systems could have a positive impact on student performance and grades.

Another study explores the landscape of distance education research during the specified time frame. The authors conducted a comprehensive review to identify major research areas, methodologies used, and patterns of authorship within the field. The purpose of this study was to provide an overview of the key trends and developments in distance education research. By analyzing research themes and methodologies, the paper offers insights into prevailing approaches and collaboration patterns. The paper provides valuable insights into distance education research during the specified period, shedding light on some key areas of study and highlighting the methods and authors contributing to the field (Zawacki-Richter et al., 2009).

Materials and Methods

The research methodology was quantitative in nature and the research design adopted was descriptive in general. To ensure the privacy and confidentiality of user data, all data collected during the study is treated with the utmost care. Participants' identities were kept strictly confidential, and all personal information was anonymized and stored separately from the research data except the public information. Any identifying information was removed or replaced with pseudonyms where required, and only the research team had access to the raw data.

The data analyzed was collected from GA configured with a helpdesk system for student support deployed at an ODL institute to provide immediate online support to student queries coming from different channels.

Figure 2

Help Desk Student Support System Workflow

Previously, there was no consolidated online help desk available for student support. For some queries, the record was maintained while not for others which caused administrative and processing delays in decision-making. Figure 3 below presents the data collection methodology which was used in this study to collect and analyze GA in identifying factors important for learner-centered application design.

Figure 3

Data Collection Methodology

Results and Discussion

To collect the GA data, one year of data starting from April 1, 2015, to April 1, 2016 was downloaded from GA which was configured with an online help desk system. Figure 4 below shows that during this period of one year 155,981 unique users accessed this Help Desk, and more than 715,052 times different pages were viewed. Each learner spent approximately three minutes during each session. On average three to four pages are visited by each learner in each session. The 47% bounce rate reflects that significant visitors exit the Help Desk after visiting the landing page.

Figure 4

Audience Overview April 2015-April 2016

Several design factors which include language, geo-location, browser, and user device were analyzed in this study.

Language Factor

Data analysis of system language shows that a majority of learners (80%) use US-English as a default language setting in their systems. Such kind of information is very important for designing a localized language application. In this case, the application design should be compatible with the US-English (en- us) language as the majority of learner’s device is compatible with the English language.

Figure 5

User System Language

Geo-location

The second factor which is analyzed is the geographical attribute. Using location data and co-relating it with other parameters, it was observed that most of the learners around 80% are from Pakistan. However, users from countries where the university is not providing services are questionable and need further analysis from an application security point of view (Figure 6).

Figure 6

Geo-location

Browser and Operating System

The browser is a client-side application software that renders a web page code in a presentable HTML form. Cross-browser compatibility is one of the prominent design issues in web application development.

Figure 7

Browser

Results show that most learners (more than 60%) use Chrome as an internet browser. Other browsers like Firefox, Opera Mini, UC, and Android are also seen to be significant. This information is very useful for application designers to consider while designing interfaces for any student-related application. Similarly, information regarding operating systems is also necessary for the developers and designers to develop or design such an application that is supported by all such operating systems which are used by potential or existing clientele. Figure 8 shows the operating systems used by most learners. From an application design viewpoint, it should be such that it is equally compatible with all operating systems used by users as shown priority-wise in the figure below.

Figure 8

Operating System

User Device

The device type or category is one of the key factors to consider while designing any application. Applications that are compatible with all kinds of devices are called responsive applications or responsive designs. In this case, Figure 9 below illustrates that learners use different types of devices such as desktop, mobile, and tablet. Application design must be responsive so that it may support all kinds of device types as listed below.

Figure 9

Device Types

There are prospects to further enhance the analysis of the design factors by incorporating additional tools or instruments to measure parameters beyond GA. This would provide a more comprehensive understanding of how these design parameters influence the development of learner-centered applications. For example, combining data from learner surveys or feedback with the analytics data collected from GA would provide a more in-depth analysis of user behavior and preferences.

In terms of the design parameters, it is important to consider the users' language, location, device category, and operating system to create personalized and user- centered applications. In this study, it was found that most users (around 8%) used US-English as their default language setting and were based in Pakistan. These insights are valuable for designing a localized language application that is compatible with the US-English language and Pakistani context. It can be further drilled down to province level to incorporate local cultural and language context in the design.

Furthermore, it was observed that many users (around 60%) used Chrome as their browser and used different types of devices such as desktop, mobile, and tablet. This highlights the importance of developing responsive applications that are compatible with different device categories and operating systems used by the target audience.

To enhance the analysis, it is recommended to incorporate additional tools or instruments to measure the parameters and gain a more comprehensive and holistic understanding of learner behavior and preferences. This could include surveys or feedback from learners, teachers, and administrators of the institute, spatial heat mapping tools using geographic data, or user testing to evaluate the effectiveness of the application design.

Conclusions

It is evident from the results that Google Analytics can play a significant role in understanding and developing learner-centered applications and user-centered interfaces. Factors including location, language, OS, and device were analyzed, and it was revealed that these factors are crucial in creating user-centered applications. By using GA, existing software and web applications for students can be improved by making them more personalized and learner-centered. Furthermore, integrating GA with online LMS, CMS, and related student support systems can help to relate different factors to students' performance and scores. In future work, learners' behavior on learning management systems can be studied by analyzing factors such as their geographical location, literacy rate, technology infrastructure, time, gender, age, and personal interests with their scores using spatial learning analytics. In general, the potential of learning analytics to enhance the effectiveness of personalized learning is vast, and this study provides valuable insights regarding the application of GA as learning analytics for researchers and developers in the field of education.

References

Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1),1-14. https://doi.org/10.1080/07294360.2014.934336

Anees, S. M. (2011). Muslim Thinkers of Education. In M. I. Yousuf, S. M. Anees, M. L. Sajjad, & M. N. Anwar. Islamic System of Education. Allama Iqbal Open University.

Ansari, A., & Mela, C. F. (2003). E-customization. Journal of Marketing Research, 40(2), 131-145. https://doi.org/10.1509/jmkr.40.2.131.19224

Atahan, P., & Sarkar, S. (2011). Accelerated learning of user profiles. Management Science, 57(2), 215-239. https://www.jstor.org/stable/41060714

Ertmer, P. A., Sadik, O., Sendurur, E., Sendurur, P., & Ottenbreit-Leftwich, A. T. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 423-435. https://doi.org/10.1016/j.compedu.2012.02.001

Firat, M., & Yuzer, T. V. (2016). Learning Analytics: Assessment of Mass Data in Distance Education. International Journal on New Trends in Education and Their Implications, 7(2), 1. http://www.ijonte.org/ FileUpload/ks63207/ File/01.mehmet_firat_.pdf

Hannafin, M. J., & Land, S. M. (1997). The foundations and assumptions of technology-enhanced student-centered learning environment. Instructional Science, 167-202. https://doi.org/10.1023/A:1002997414652

Jackson, S. L., Stratford, S. J., Krajcik, J. S., & Soloway, E. (1995). Model-it: A case study of Learner-Centered Design Software for Supporting Model Building. WCTASC.

Kilis, S., & Gülbahar, Y. (2016). Learning Analytics in Distance Education: A Systematic Literature Review. In T. Reiners & L. C. Wood (Eds.), Open Learning and Teaching in Educational Communities (pp. 319-326). Springer International Publishing.

Liu, D. R., Tsai, P. Y., & Chiu, P. H. (2011). Personalized recommendation of popular blog articles for mobile applications. Information Sciences, 181(9), 1552-1572.

Morphew, V. (2012). A constructivist approach to the national educational technology standards for teachers. International Society for Technology in Education.

O'reilly, T. (2007). What is Web 2.0: Design patterns and business models for the next generation of software. Communications & Strategies, (1), 17.

Özen, Z., Koçoğlu, F., & Beden, Ş. (2014). The Examination of User Habits through the Google Analytic Data of Academic Education Platforms. International Journal of E-Adoption, 6, 31-45. http://doi.org/10.4018/ijea.2014070103

Schmidt, M., Earnshaw, Y., Tawfik, A. A., & Jahnke, I. (2020). Methods of User Centered Design and Evaluation for Learning Designers. In D. Gibson, M. Ostashewski, & T. Nagata (Eds.), UX for Learning Design (pp. 11-26).

Shapira, B., Shoval, P., Tractinsky, N., & Meyer, J. (2009). ePaper: A personalized mobile newspaper. Journal of the American Society for Information Science and Technology, 60(11), 2333-2346. https://doi.org/10.1002/asi.21172

Students at Center. (2016). Jobs for the future. Students at the center Org. http://www.studentsatthecenter.org/about

Tahir, A., McArdle, G., & Bertolotto, M. (2012, June). Identifying specific spatial tasks through clustering and geovisual analysis. In 2012 20th International Conference on Geoinformatics (pp. 1-6). IEEE. https://doi.org/10.1109/Geoinformatics.2012.6270301

Tahir, A., McArdle, G., Ballatore, A., & Bertolotto, M. (2010). Collaborative filtering-a group profiling algorithm for personalisation in a spatial recommender system. Proceedings Geoinformatik, Kiel, Germany, 44-50.

Vesin, B., Mangaroska, K., & Giannakos, M. (2018). Learning in smart environments: user-centered design and analytics of an adaptive learning system. Smart Learning Environments, 5(1), 1-19. https://doi.org/10.1186/s40561-018-0071-0

Ying, Y., Feinberg, F., & Wedel, M. (2006). Leveraging missing ratings to improve online recommendation systems. Journal of Marketing Research, 43(3), 355-365. http://www.jstor.org/stable/30162410

Zawacki-Richter, O., Baecker, E. M., & Vogt, S. (2009). Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. The International Review of Research in Open and Distributed Learning, 10(6), 21–50. https://doi.org/10.19173/irrodl.v10i6.741

International Journal on Open and Distance e-Learning, Vol. 9 No. 1, June 2023 Issue