Evaluation of the UPOU MOOC "Principles of Graphic Design" using Product Variable of Biggs' 3P Model

Ma. Gian Rose Cerdeña^1*, Lexter Mangubat², Luisa Gelisan³, and Mari Anjeli Crisanto⁴

¹Project Staff, University of the Philippines Open University, [email protected]
²Information Systems Research, University of the Philippines Open University, [email protected]
³Assistant Professor, University of the Philippines Open University, [email protected]
⁴Assistant Professor, University of the Philippines Open University, [email protected]
* Corresponding author

Abstract

Keywords: MOOC Evaluation, Biggs’ 3P Model, Enjoyment and self-satisfaction, Completion Rates, Quality Assurance

Introduction

Background of the Study

Massive Open and Online Courses (MOOCs) have emerged as a transformative force in the field of education, breaking down traditional barriers to learning by providing access to quality and curated educational resources and opportunities. MOOCs have been characterized as open, participatory, and distributed (Baturay, 2015). It is open as it offers free enrollment for everyone who has access to the internet. The learning pace is also dictated by the students themselves. Further, Baturay (2015) argued that it is participatory and distributed as “learners may interact with their fellow learners and instructor and participate in the various learning activities prepared” and that “knowledge sharing is encouraged to foster creative thinking in its participants.”

The University of the Philippines Open University (UPOU) pioneered the development and offering of MOOCs in the country since its commencement in 2012 (Almodiel et al., 2020). It uses a Moodle-based platform called MODeL which means Massive and Open Distance e-Learning.

Among the array of MOOC offerings by UPOU, the MOOC titled "Principles of Graphic Design" has garnered substantial interest and active participation from learners. As multimedia production courses continue to attract attention, understanding their quality and effectiveness becomes paramount to optimize their impact and cater to the diverse needs of learners effectively. Therefore, the primary objective of this study is to evaluate the UPOU MOOC "Principles of Graphic Design" using the product variable of Biggs' 3P Model.

Statement of the Problem

In light of the dynamic and expanding landscape of MOOCs, this research sought to comprehensively assess the quality and efficacy of the UPOU MOOC titled "Principles of Graphic Design" through the lens of the product variable of Biggs' 3P Model. The product variable evaluation focused on critical dimensions such as completion/retention and certification rates alongside learners' subjective experiences of enjoyment and self-satisfaction with the course content and delivery.

Objectives

This study evaluated the UPOU MOOC “Principles of Graphic Design” based on completion/retention and certification rates as well as the enjoyment and self-satisfaction of the students.

1. Evaluate completion/retention and certification rates based on LMS analytics; and
2. Evaluate students’ enjoyment and self-satisfaction using a survey instrument based on the learner ratings for the product variable in the Guidelines for Quality Assurance and Accreditation of MOOCs.

Significance of the Study

With the increase in MOOC enrolment in UPOU, MOOC quality needs to be regularly evaluated and improved. While measures should be taken to make sure that a MOOC meets certain criteria before it is offered, the MOOC should also be evaluated post-delivery. This study contributes to studies on MOOC quality by presenting the results of the evaluation and by presenting an instrument that could be used for post-delivery evaluation.

Scope and Limitations

This study involved voluntary participants from the MOOC "Principles of Graphic Design" offered from February to March 2023. A total of 7,239 out of the 22,942 enrolled students (31.55%) consented to participate in the study.

Review of Literature

MOOCs

Massive Open Online Courses (MOOCs) are a form of online education that emerged in 2008 and transformed the landscape of learning worldwide. Through these online courses, anyone who has access to the Internet is provided with an affordable and flexible way of learning new knowledge and acquiring new skills as the courses are offered for free. MOOCs also provide learners with opportunities to access high-quality educational content from prestigious institutions without the constraints of geographical boundaries (Yousef et al., 2015).

MOOCs have been referred to as a type of disruptive technology, a major revolution in education, and have four key characteristics (Bates, 2014). These are as follows:

1. It has infinite scalability (massive) as it can accommodate as many learners as possible.
2. It has no prerequisites and does not require tuition fees for participants (open). Its only requirement is for the participant to have access to a computer/mobile device and the Internet.
3. Courses are available online although some institutions have started to implement MOOCs in blended format.
4. MOOCs are organized in one whole course.

The first MOOC offering is the course “Connectivism and Connectivity.” It was developed in 2008 by Stephen Downes and George Siemens (Bates, 2014). Since then, more than 200 million learners have enrolled in various MOOC offerings, reaching almost 20,000 courses offered by 950 universities all over the world (Shah, 2021).In the Philippines, the University of the Philippines Open University (UPOU) started to develop its own MOOC in 2012. By the following year, the university offered the MOOC “Introduction to Mobile Applications Using the Android Platform” through UPOU’s first MOOC platform called @ral. Bandalaria (2013), the main proponent of MOOCs at UPOU, said that the creation and offering of MOOCs is in line with the university’s public service programs and its advocacies for open learning. The university later developed the MOOC platform, UPOU MODeL, or Massive Open Distance eLearning .

The university’s MOOCs duration ranges from 4 to 6 weeks. The teachers in this MOOC, called course coordinators, evaluate the learners who are qualified to receive certificates for completion. The learner support system is the UPOU MOOC's distinguishing feature. Interactions between learners and course coordinators are also encouraged. These are held not only within the online learning management system but as well as in social media as social media platforms are more accessible to UPOU MODeL learners (Almodiel et al., 2020).

Evaluating the Quality of MOOCs

With MOOCs’ affordances – being offered for free, having no enrollment eligibility requirement, and aiding people to learn new knowledge and/or acquire new skills needed for the changing requirements in the workplace – its growth and development are projected to further propel by 34.54% from 2020 to 2027 (Business Wire, 2022).

Though there is a further increase in MOOC’s popularity, still there are high rates of dropout and non-completion (Aldowah et al., 2020; Badali et al., 2020; Rekha et al., 2023). According to a report on Forbes.com, MOOCs offered by Stanford, MIT, and Harvard have a course completion rate of around 20% (Newton, 2022).

Thus, it is necessary to evaluate MOOCs to gauge their effectiveness, understand their impact on learners, and identify areas for improvement.

Satisfaction in and Completion/Non-Completion of MOOCs

A learner's journey towards completion or non-completion of a MOOC has been the subject of several studies.

The primary factors that cause students to drop out from MOOCs are academic abilities, previous experience, course design, lack of feedback, social presence, and social support. On the other hand, the secondary factors identified were interaction, course and time difficulty, motivation, commitment, and family and work (Badali et al., 2020).

Motivation is also a MOOC completion factor identified in a study conducted in India (Aldowah et al., 2020). Motivation has six main factors that influence participants’ MOOC completion: academic, social, course, personal, professional, and technological. In a systematic literature review of 50 relevant publications conducted in 2022, the influence of motivation for completing MOOCs was further explored (Badali et al., 2020). Two categories of motivations were identified, need-based motivation and interest-based motivation with academic motivations as having the most influence on a participant's retention in a MOOC class.

In terms of continuance to participate in MOOCs, the learner's perceived reputation and perceived openness to a MOOC were shown to be the strongest predictors (Alraimi et al., 2015). In another study, MOOC performance, in terms of knowledge transmission quality, and the learner's preference for MOOCs as a learning mode significantly increases continuance intention for pursuing studies through MOOCs (Dai et al., 2020). Other factors identified to influence continuance and satisfaction are confirmation of learners' expectations of the MOOCs they are enrolled in and the usefulness of the courses, respectively (Rekha et al., 2023; Shah & Khanna, 2022).

Biggs 3P Model

The 3P Model developed by John Biggs, a scholar in the field of educational research and instructional design, has greatly influenced the teaching and learning assessment systems being used today (Barattucci, 2017).

The model was structured along three phases of learning (Biggs, 1993):

1. Presage, or the phase beginning the teaching-learning process or before learning is produced. There are two groups of variables at play in this stage: the characteristics of the students, i.e., prior knowledge/experiences, cognitive ability or skills, ways of learning and expectations and values and the characteristics of the teaching context, i.e., teaching objectives, teaching methods, assessment methods, and learning space conditions, among others.
2. Process, or the learning phase itself. It refers to what or how the student accomplishes a task in a specific context to learn. Biggs (1993) stated that how a student pursues a task is affected by one’s perception not only of the task to be done but also of oneself.
3. Product, or the learning outcomes which are of three types: – quantitative, qualitative, and affective. Quantitative refers to the amount of information and the skills acquired. Qualitative is the complexity of thoughts and knowledge that were developed while affective is the student’s commitment and satisfaction of the process.

Biggs' model assumes that learning outcomes are affected by several factors that interact with each other, thus requiring not only effectiveness and good quality but also compatibility of the components (Barattucci, 2017).

Due to its integrative character, the model is used in research as a framework to ensure that all aspects contribute to a student's learning process, allowing for a better understanding of how the factors influence each other (Allison, 2021; de la Fuente et. al., 2014; Kanashiro et al., 2020; Song, 2018). The 3P model has been used in various contexts such as MOOCs, psychological processes studies, and K-12 computing instruction, among others (Allison, 2021; Ganotice & Chan, 2019; Song, 2018).

Though studies have criticized the 3P Model as outdated, oversimplified, and not the only model that can be used to understand educational context, it appears that it is the “most prominent learning model in higher education” (Allison, 2021; Kanashiro et al., 2020).

Theoretical Framework

The Biggs’ 3P Model used in the Guidelines for Quality Assurance and Accreditation of MOOCs (Commonwealth of Learning, 2016) consists of three groups of variables: presage variables, process variables, and product variables. In Figure 1, it can be seen that these variables are interconnected with each other, as all factors contribute to a student’s learning process.

Figure 1

Biggs’ 3P Model of Student Learning

This research focused on the product variables in its post-delivery evaluation because also evaluating the presage and process variables will result in extensive research given the number of MOOC enrollees. For the product variables, the research considers enjoyment and self-satisfaction, with the completion/ retention and certification rates achieved by the students as variables identified by COL (2016).

Methodology

Research Design

The learning management system (LMS) analytics and data gathered from an online survey were used to evaluate the product variables in this study. Specifically, the LMS analytics used were course participation and activity completion. For the online survey, a Qualtrics form was created with questions about outcomes perception, continuing education, employment opportunities, and socializing variables to determine enjoyment and self-satisfaction.

Respondents of the Study

This study involved voluntary participants of the Principles of Graphic Design MOOC, offered from February to March 2023, who were 18 years old and above. An online questionnaire was given at the end of the class and the learners were able to choose whether they would participate or not. Participants were given informed consent upon accessing the survey instrument. Names and other personal information were not collected, and all data collected were treated with confidentiality.

The target sampling size for the study was 378/22,942 MOOC enrollees. This was calculated with a 95% confidence level and 10% margin of error. Since 7,239 learners participated in the study, the target sampling size was exceeded

Data Gathering Procedure

For this research, data was gathered in two ways:

1. The completion and certification rates were computed using the LMS analytics; and
2. The enjoyment and self-satisfaction rates were evaluated using a survey instrument based on the learner ratings for the product variable in the Guidelines for Quality Assurance and Accreditation of MOOCs using a five- point Likert scale: outcomes perception, continuing education, employment opportunities, and socializing.

Qualitative data was also collected through the survey instrument to gain further insight into the evaluation done through the quantitative scales.

Data Analysis

Data was collected for the completion and certification rates using the LMS analytics of course participation and activity completion which identified the number of enrollees in the course as well as course completers. Results for the enjoyment and self-satisfaction rates collected through the Qualtrics survey were analyzed quantitatively using mean and standard deviation and supported with the summary of their qualitative answers using a word cloud. Aside from the learner ratings for the enjoyment and self-satisfaction rates, data collected from their demographics were also analyzed in terms of their frequency distribution which included their age, gender, civil status, location, highest educational attainment, employment status, current program enrollment, and previous MOOC experience.

Results and Discussion

Table 1

Summary of demographic profile of online learners

Out of the 22,942 enrollees, a total of 7,874 were able to access the survey instrument. However, only 7,239 respondents were able to finish the Qualtrics survey on enjoyment and self-satisfaction as some of those who initially accessed the survey were minors (aged 17 and below) and some did not proceed with answering the survey.

For the demographics, learners were classified into eight categories: age, gender, civil status, location, highest educational attainment, employment status, enrollment in a program, and previous MOOC experience. Table 1 shows the summary of the demographic profile of the learners

Table 2

Summary of the highest demographic per learner rating category (+ for positive rating, 0 for neutral rating, - for negative rating)

These data were also analyzed in parallel with their enjoyment and self- satisfaction ratings, and it was found that there were no significant differences in the classification of the learners regarding the rating they were given. The ratings were categorized into positive, neutral, and negative based on their level of satisfaction. As shown in Table 2, the majority of the positive, neutral, and negative ratings were given by nearly the same demographic classification. The only difference was seen under the employment opportunities category, where the employment status for positive ratings was from employed learners and the neutral and negative ratings were from unemployed learners.

Using a five-point Likert scale, with 1 being the lowest and 5 being the highest, learners were asked to rate their enjoyment of the MOOCs course per item. The results were then calculated using mean and standard deviation.

Table 3

Summary of the outcomes perception rating of the learners using mean and standard deviation

Tables 3 to 6 show the mean and standard deviation results for the learner ratings. Table 3 reflects the outcomes perception rating. Each item had mean scores of four or higher, meaning a high enjoyment rating. The item with the highest mean score and low standard deviation was “I will be able to use what I learn in the future”.

Table 4

Summary of the continuing education rating of the learners using mean and standard deviation

Table 4 focuses on the continuing education rating with all items having mean scores of above four, meaning high enjoyment rating. The item with the highest mean score with a low standard deviation was “During learning I treat the resources I find as a starting point and try to develop my own ideas from them”.

Table 5

Summary of the employment opportunities rating of the learners using mean and standard deviation

Table 5 shows the employment opportunities rating results, also with all items having above four mean scores, again meaning high enjoyment rating. The item with the highest mean score under this learner rating with a relatively low standard deviation is “I often think about how my learning fits into the “bigger picture” of my work/practice”.

Table 6

Summary of the socializing rating of the learners using mean and standard deviation

Table 6 focuses on the socializing rating, where the mean scores ranged around three, meaning a neutral enjoyment rating from the learners. The highest mean score which had a relatively high standard deviation was the item on “I ask others for more information when I need it”.

Meanwhile, the satisfaction component of the survey had six parts: teachers/ coordinators, learner support, technical support, course materials, assessment, and overall satisfaction. Their qualitative answers per item were analyzed and presented using a word cloud.

Figure 2

Summary of satisfaction rating of learners

As seen in Figure 2 above, each item had a high satisfaction rate from the learners. Most of the learners were very satisfied, especially with the course materials provided in the MOOC, followed by their satisfaction with the teacher/ coordinator, assessment, and learner support. Technical support had a low satisfaction rating compared with the rest of the factors. The reason given by some of the learners for the said low rating is that they did not need much technical support from the course.

Overall, satisfaction with the MOOC course was very high. Other MOOC enjoyment and satisfaction survey results have also resulted in high satisfaction ratings; however, with the advantage of a more detailed survey, this paper has identified the nuances of what keeps a learner satisfied in a MOOC course. Hood et al. (2015) as well as Littlejohn et al. (2016), have identified that the satisfaction of a learner in a MOOC course is determined by their goals for learning and participation. This paper has determined the learners' goal for enrolling in the MOOC studied in this research. Specifically, the learners aimed for the knowledge they gained in the MOOCs course to be used for the future (outcomes perception learner rating, question number 5; continuing education learner rating, question number 3). In 2021, Alyoussef evaluated satisfaction based on its perceived ease of use and utility; the results of this paper have identified that informative course materials and knowledgeable teacher/ coordinator are some of the main components of a MOOC course that are important to keep learners satisfied.

Figure 3

Word cloud of the qualitative responses of learners

Figure 3 shows the compiled word cloud from the qualitative components of the survey answered by the learners to describe their learning satisfaction rating. The word cloud tracks the most used words in the comments. The bigger the font size of the word, the more frequently it is cited by the respondents. From this, the general sentiments of the learners can be inferred from the reasons/s as well for the satisfaction rating given.

The course material category with high satisfaction ratings is supported by its corresponding word cloud, where learners attribute their satisfaction with the “material” used in class. Surrounding the word “material” as the biggest font size, are words such as “helpful”, “useful”, and “easy” where it can be derived that the learners are highly satisfied due to the “helpful, useful, and easy material” provided in the learning course. Similarly, the overall satisfaction received high satisfaction ratings because, as gleaned from its word cloud, learners were able to “learn and experience a lot” and were “satisfied with the knowledge” they had gained from the course.

For the teacher/coordinator category, the word cloud has identified that learners described them with the following words: “easy”, “explain”, “understand”, “informative”, and “helpful” among many others. From here, it is understood that the high satisfaction rating from the learners was due to the “informative and helpful teacher/coordinator” that made the course “easy to understand with their explanations”. In the assessment category, the word cloud presents the word “assessment” with positive descriptions such as “good”, “satisfy”, “easy” and “learning” where it can be assumed that the assessments used in the course were “good, easy and satisfying assessments” that helped them “learn a lot”.

The learner support category also had the words “learner support” surrounded with positive descriptors like “great”, “helpful”, “good”, and “easy”. This provides context that the high satisfaction rating for the learner support category was due to the learners receiving “great and helpful learner support” that must have made their course “good and easy”. On the other hand, the technical support category with a lower satisfaction rating was surrounded with words such as “problem”, “encounter”, “good”, and “issue” which is clarified with the support- ing words “haven’t” and “didn’t” that were also seen in the word cloud. As it is, the reason for the lower satisfaction rating is for a more neutral stance as the learners “haven’t encountered issues or problems” while they are taking the course.

Conclusion and Recommendations

Overall, the MOOCs course “Principles of Graphic Design” evaluated using the product variable resulted in high enjoyment and self-satisfaction, and comple- tion/retention and certification rates. Of the 22,942 enrolled students, 7,620 managed to complete the course and received certificates of completion, which resulted in a high completion rate of 33.21%.

Under the enjoyment component of the survey, learners had high enjoyment for three out of the four learner ratings with a mean score of above four. Mean- while, the learner rating for socializing had a lower rating with a mean score around the range of three, with a more neutral tone of enjoyment. Based on the identified items per learner rating with the highest mean score, it can be concluded that most of the learners enjoy the MOOCs course for the additional knowledge and learning they can acquire, and do not place much focus on the socialization aspect.

For the self-satisfaction component, learners also had high ratings for each item, with their highest rating on the course materials, and the lowest at techni- cal support. The word cloud of qualitative answers from the learners also sup- ported the fact that the learners enjoyed the course and were generally satis- fied with their experience in the course.

For future studies, it is recommended that the survey instrument include ques- tions focusing on the learners’ intention upon enrolling in the course and study if the intentions have a direct effect on competition and certification rate. On the other hand, the low ratings given by learners regarding socializing as an enjoyment component can be used as a basis for developing various assess- ment activities for a MOOC. MOOC courses focus on independent learning to produce self-directed learners, but MOOC developers can also experiment with group requirements that can help develop collaborative skills or create a balance between independent learning activities and group activities to provide learners avenues to develop various life skills essential for 21st-century learn- ers and workers. A study on socializing as an enjoyment component can also be further studied in the future, i.e., What are the factors that promote social- izing as an enjoyment component? What are the learning and assessment ac- tivities that promote enjoyment in MOOCs? Is there a relationship between enjoyment factor, self-satisfaction, and topic of the MOOC?

Lastly, future MOOCs can also focus on providing more learner support as well, having been identified as another relatively low satisfaction category.

References

Aldowah, H., Al-Samarraie, H., Alzahrani, A.I., & Alalwan, N. (2020). Factors affecting student dropout in moocs: A cause and effect decision-making model. J Comput High Educ 32, 429–454. https://doi.org/10.1007/s12528- 019-09241-y

Allison, J. (2021, November). The Importance of Context: Assessing the Chal- lenges of K-12 Computing Education through the Lens of Biggs 3P Model. 21st Koli Calling International Conference on Computing Education Re- search (pp. 1-10).

Almodiel, M. C., Mampusti, K. G. A., & Tanay, S. (2020). Social Media as Com- munication and Learner Support Tool in Massive Open Online Courses (MOOCs). International Journal on Open and Distance e-Learning (IJODEL), 6(1).

Alraimi, K., Zo, H., & Ciganek, A.P. (2015). Understanding the MOOCs con- tinuance: The role of openness and reputation. Computers & Education, 80 (28-38). https://doi.org/10.1016/j.compedu.2014.08.006

Alyoussef, I. Y. (2021). Massive open online course (MOOCs) acceptance: The role of task-technology fit (TTF) for higher education sustainability. Sustain- ability, 13(13), 7374. https://doi.org/10.3390/su13137374

Badali, M., Hatami, J., Banihashem, S.K., Rahimi, E., Noroozi, O., & Eslami, Z. (2020). The role of motivation in MOOCs’ retention rates: a systematic lit- erature review. RPTEL, 17, 5. https://doi.org/10.1186/s41039-022-00181-3

Bandalaria, M.dP. (2013). MOOC- to MOOC+: Applying the ODeL Quality Framework to Fill the Gaps [Conference presentation]. 25th World Confer- ence of the International Council of Distance Education, China.

Barattucci, M. (2017). Approach to Study as an Indicator of the Quality of Teach- ing and of Learning Environment: the contribution of John Biggs. Journal of e-Learning and Knowledge Society, 13(2).

Bates, T. (2014). What is a MOOC? Online Learning and Distance Education Resources. https://www.tonybates.ca/2014/10/12/what-is-a-mooc/

Baturay, M. H. (2015). An overview of the world of MOOCs. Procedia- Social and Behavioral Sciences, 174, 427-433. https://doi.org/10.1016/j. sbspro.2015.01.685

Biggs, J. (1993). What do inventories of students’ learning processes really measure? A theoretical review and clarification. British Journal of Educational Psychology, 63(1), 3-19. https://doi.org/10.1111/j.2044-8279.1993. tb01038.x

Business Wire. (2022). The Worldwide Massive Open Online Courses Industry is Expected to Reach $7.2 Billion by 2027. https://www.businesswire. com/news/home/20220819005142/en/The-Worldwide-Massive-Open-Online-Courses-Industry-is-Expected-to-Reach-7.2-Billion-by-2027---ResearchAndMarkets.com.

Commonwealth of Learning (2016). Guidelines for Quality Assurance and Accreditation of MOOCs. https://oasis.col.org/handle/11599/2362

Dai, H.M., Teo, T., & Rappa, N.A. (2020). Understanding continuance intention among MOOC participants: The role of habit and MOOC performance. Computers in Human Behavior, 112. https://doi.org/10.1016/j. chb.2020.106455

de la Fuente, J., Zapata, L., Sander, P., & Cardelle-Elawar, M. (2014). The 3p And Dedepro Models as Research Heuristic. International Journal of Developmental and Educational Psychology, 4(1) 155-164. ISSN: 0214-9877.

Ganotice Jr, F. A., & Chan, L. K. (2019). How can students succeed in computer-supported interprofessional team-based learning? Understanding the underlying psychological pathways using Biggs' 3P model. Computers in Human Behavior, 91, 211-219. https://doi.org/10.1016/j.chb.2018.09.029

Hood, N., Littlejohn, A., & Milligan, C. (2015). Context counts: How learners’ contexts influence learning in a MOOC. Computers & Education, 91, 83–91. https://doi.org/10.1016/j.compedu.2015.10.019

Kanashiro, P., Iizuka, E. S., Sousa, C., & Dias, S. E. F. (2020). Sustainability in management education: A Biggs’ 3P model application. International Journal of Sustainability in Higher Education, 21(4), 671-684. https://doi. org/10.1108/IJSHE-05-2019-0176

Littlejohn, A., Hood, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs: Motivations and self-regulated learning in MOOCs. The internet and higher education, 29, 40-48. https://doi.org/10.1016/j.iheduc.2015.12.003

Newton, D. (2020). The “Depressing” And “Disheartening” News About MOOCs. Forbes. https://www.forbes.com/sites/dereknewton/2020/

Rekha, I.S., Shetty, J., & Basri, S. (2023). Students' continuance intention to use MOOCs: empirical evidence from India. Educ Inf Technol, 28(4):4265- 4286. https://doi.org/10.1007/s10639-022-11308-w

Shah, D. (2021). By the Numbers: MOOCs in 2021. https://www.classcentral.com/report/mooc-stats-2021/

Shah, J., & Khanna, M. (2022). What Determines MOOC Success? Validation of MOOC Satisfaction Continuance Model. Vision, https://doi. org/10.1177/09722629221131386

Song, J. (2018). Elements in Mol-based College English learning environment- based on Biggs' 3P Model. Advances in social science, education and information research, 89, 5-14.

Yousef, A. M. F., Chatti, M. A., Schroeder, U., & Wosnitza, M. (2015). A Usability Evaluation of a Blended MOOC Environment: An Experimental Case Study. The International Review of Research in Open and Distributed Learning, 16. 69-93. https://doi.org/10.19173/irrodl.v16i2.2032

International Journal on Open and Distance e-Learning, Vol. 9 No. 2, December 2023 Issue