Abstract Introduction Objectives Conceptual Framework Methodology Results and Discussion Conclusions and Recommendations References

Evaluation of Supplemental Interactive Learning Material for Teaching Physics in University of the Philippines Los Baños

Beverly R. Pabro1 and Maribel Dionisio-Sese2

1University Extension Specialist, Interactive Learning Center - University of the Philippines Los Baños, Philippines, [email protected]
2Director, Interactive Learning Center - University of the Philippines Los Baños, Philippines, [email protected]

Abstract

With the advent of new technologies being infused in school curricula, educators and school leaders of higher education in the Philippines have embraced e-learning, which involves technology-mediated teaching strategies and a variety of tools to facilitate learning. In this regard, the University of the Philippines Los Baños (UPLB) through the Interactive Learning Center (ILC) started developing and producing online interactive educational materials known as Learning Objects (LOs), which are now being used by UPLB students as supplemental interactive learning materials in various courses in the university.

This study focused on the evaluation of the LO on Rotation of Rigid Bodies in terms of knowledge gain of the students and determine its effectiveness in terms of attractiveness, comprehensibility, applicability, interactivity, and assessment function. Students enrolled in PHYS 81 (Fundamental Physics 1) during the First Semester AY 2017-2018 were divided into control and treatment groups and subjected to pre-test-post-test for the LO evaluation. Using the t-test of significance, results showed that the treatment group had a significant increase in the mean scores after being exposed to the LO compared to the control group who did not view the LO, indicating that students aided with LO had greater knowledge gain than those without LO supplementation.

Additionally, the Likert scale of scores 1 (strongly disagree) to 5 (strongly agree) was used to evaluate the treatment group’s responses on the different components of the LO. Computing for the weighted mean in each component, results showed that most respondents agreed that the LO was attractive (3.86) and that it was both comprehensible (4.38) and applicable (4.16). Similarly, most respondents in the treatment group agreed that the LO was interactive (4.14) and its assessment items were appropriate (4.28) to the program. Overall, the respondents agreed (4.27) and assessed the evaluated LO as an effective supplementary learning material for the physics topic on the rotation of rigid bodies.

Keywords: interactive learning, learning object, rotation of rigid bodies, UP Los Baños

Introduction

The field of e-learning is changing so rapidly that there is a growing need for the development of quality of materials as well as to provide excellent and effective pedagogical models and assessment programs (Frydenbeg, 2002). As more institutions of higher education have begun to appreciate and embrace e-learning, which involves technology-mediated teaching strategies and a variety of tools to facilitate learning, it becomes imperative that standards be developed which will ensure quality and consistency in its creation and use (Ortiz & Green, 2019, as cited in Reid, 2019).

One of the online educational materials available that is used as an innovative approach in offering curricular programs is the Learning Objects (LOs). These are short, self-contained, reusable teaching materials that can be aggregated for larger collection of contents and tagged with metadata (Beck, 2010). Each LO is a collection of content items, practice items, and assessment items that are combined based on a single learning objective (Cisco Systems, 1999). They are small in size and can take on a variety of different shapes, formats, and purposes. According to Griffith et al. (2003), most institutions reported in consensus that LOs can be used in all instructional environments, including campus-based (face-to-face and/or traditional) as well as all types of online instruction, either instructor-led or self-paced. They can also be used to illustrate, support, supplement, or assess student learning.

To support the interactive learning process, the Interactive Learning Center (ILC), which was inaugurated in 2005 at the University of the Philippines Los Baños (UPLB), started the development and production of multimedia materials, specifically LOs, primarily for undergraduate program courses. Among the twelve LOs initially developed was the LO on Rotation of Rigid Bodies for Physics which was chosen as the focus of study.

Objectives

The study specifically aims to determine the effect of LO exposure on the knowledge gain of the students and evaluate its effectiveness as supplemental learning material based on its various components.

Conceptual Framework

The conceptual framework for this study illustrates the factors that will affect the knowledge gain of the students (Figure 1). The dependent variable of the study was the knowledge gain of students. Effectiveness was determined if there is a significant increase in scores from pre-test to post-test. On the other hand, the independent variable pertains to exposure to the learning objects. Using the quality standards (attractiveness, clarity, comprehensibility, applicability, interactivity, and assessment) as variables in the learning object, the effectiveness of the learning object was determined.

The respondents’ socio-demographic characteristics as an intervening variable were observed to determine whether it may affect the relationship between the independent variable and the dependent variable.

Figure 1. Conceptual framework of the study

Methodology

For this study, the learning object on Rotation of Rigid Bodies in Physics produced by UPLB-ILC in 2007 was evaluated (Figure 2). The study was conducted during the First Semester Academic Year 2017-2018 with the LO evaluation done by students enrolled in PHYS 81 (Fundamental Physics 1). A total of 150 students from two lecture sections were chosen as respondents for this study. Stratified random sampling based on recitation sections was implemented in identifying the 72 students who comprised the control group while 78 students formed the treatment group.

The pre-test was administered to all the students using a nine (9)-item questionnaire. After the pre-test, the students were randomly separated into the control group, who were only exposed to their regular classroom discussion, and the treatment group, who were exposed to regular classroom discussion plus the LO viewing. Both groups were given afterward the post-test to evaluate their knowledge gain. The mean scores, standard deviation, t-test, and z-test were then computed for the analysis of the obtained responses. To determine if socio-demographic characteristics affect the knowledge gain of respondents, Mann-Whitney Test was employed.

In addition, following the Learning Object Peer Review Rubric Adapted from Wisconsin Online Resource Center Interactive Learning Objects Quality Standards (2013), the treatment group was requested to evaluate the LO based on the following components: a) attractiveness; b) clarity and comprehensibility; c) applicability; d) interactivity; and, e) assessment.

Figure 2. Learning Object on Rotation of Rigid Bodies

Likert scale was adopted to evaluate the students’ responses with the following notations: 1-strongly disagree (SD), 2-disagree (D), 3-neither agree or disagree (NAD), 4-agree (A), and 5-strongly agree (SA). Recorded frequencies in each category were used in computing for the weighted mean values to aid in the analysis of the obtained responses.

Results and Discussion

A.) Respondents’ Profile

Table 1 shows that more than half of the respondents in the control group were male, comprising 41 out of 72 students (56.94%), while only 43.06% were female. For the treatment group, female respondents dominated the group with 53.85% while males comprised 46.15% of the total respondents.

Table 1. Gender of the respondents

Gender Control Treatment
N % N %
Female 31 43.06 42 53.85
Male 41 56.94 36 46.15
Total 72 100 78 100

As shown in Table 2, the majority of the respondents both for the control (73.61%) and treatment (80.77%) groups were sophomore students. On the other hand, the least respondents for both groups were either senior or old freshman students.

Table 2. Classification of the respondents

Classification Control Treatment
N % N %
Old Freshman 3 4.17 1 1.28
Sophomore 53 73.61 63 80.77
Junior 14 19.44 14 17.95
Senior 2 2.78 0 0
Total 72 100 78 100

Table 3 shows that almost 38% of the respondents in the control group were taking BS Civil Engineering, followed by BS Electrical Engineering (27.78%), and BS Industrial Engineering (22.20%). The least degree programs of the students were BS Chemistry and BS Applied Physics, both with 5.56%, as well as BS Mathematics and Science Teaching having only 1.39% distribution.

Table 3. Degree courses of the respondents

Degree Course Control Treatment
N % N %
BS Civil Engineering 27 37.50 26 33.33
BS Chemistry 4 5.56 21 26.90
BS Mathematics and Science Teaching 1 1.39 2 2.56
BS Industrial Engineering 16 22.20 15 19.23
BS Electrical Engineering 20 27.78 13 16.70
BS Applied Physics 4 5.56 1 1.28
Total 72 100 78 100

On the other hand, the top two degree programs among the respondents of the treatment group were BS Civil Engineering (33.33%) and BS Chemistry (26.90%). Only 2.56% of the total respondents were pursuing BS Mathematics and Science Teaching and 1.28% were taking BS Applied Physics. Since most of the respondents were pursuing engineering programs, students from the College of Engineering and Agro-Industrial Technology dominated both groups, with the rest of the students coming from the College of Arts and Sciences (Table 4).

Table 4. College affiliation of the respondents

College Control Treatment
N % N %
College of Arts and Sciences (CAS) 9 12.5 24 30.77
College of Engineering and Agro-Industrial Technology (CEAT) 63 87.5 54 69.23
TOTAL 72 100 78 100

B.) Knowledge Gain of the Respondents

Knowledge gain pertains to the performance of students based on the difference of their mean scores in the pre-test and post-test questionnaire.

Results showed in Table 5 that the mean pre-test score of the control group was 2.00 while the post-test score has a mean of 4.23. Based on the t-computed value of 7.87 at 5% level of significance, there was a significant difference between the pre-test and post-test scores of the respondents. It can be concluded then that regular classroom discussion helped increase the knowledge gain of students.

Similarly, the result showed that there was also an increase in the mean score of the treatment group of 5.95 from the pre-test score (1.85) to the post-test score (7.80). Based on the t-computed value of 24.88 at 5% level of significance, there was also a significant difference between the pre- test and post-test scores of the respondents. It can be concluded that regular classroom discussion plus the use of the LO greatly helped increase the knowledge gain of students.

Table 5. Mean scores of the respondents in the pre-test and post-test

Rotation of Rigid Bodies Control Treatment
Pre-test Post-test Pre-test Post-test
2.00 ± 1.56 4.23 ± 2.14 1.85 ± 1.34 7.80 ± 1.46

Although the control group has a higher mean pre-test score than the treatment group, with a z-test result of 0.58 at 5% level of significance, there was no significant difference between the pre-test scores of both groups. Therefore, it can be assumed that both groups of students have the same level of knowledge on Rotation of Rigid Bodies prior to the class discussion and use of the LO.

On the other hand, the mean post-test scores of the treatment group (7.80) were higher by 3.57 compared to the mean post-test scores of the control group (4.23). With a z-test result of 11.45 at 5% level of significance, there was a significant difference in the post-test scores of the treatment and control groups with the former gaining more knowledge than the latter. The significant improvement in the post-test scores of the treatment group can be attributed to the use of LO as a supplemental learning material to improve the knowledge gain of the students. From the results, it is advisable then to supplement regular classroom discussions with LOs to enhance the information that will be assimilated by students on a particular subject matter.

In addition, it was determined if socio-demographic characteristics affect the knowledge gain of respondents. Table 6 shows that each of the socio-demographic characteristics exerts significant differences in the knowledge gain between pre-test and post-test both for the control and treatment groups. However, based on Mann-Whitney test, results showed that there were no significant differences between the differences between pre-test and post-test scores across gender, classification, and college affiliation both for the control and treatment groups.

Table 6. Effect of socio-demographic characteristics to the respondents’ knowledge gain between pre-test and post-test

Socio-Demographic Characteristic Control Treatment
Pre-test versus Post-test Pre-test versus Post- test
p-value Mann- Whitney Test p-value Mann- Whitney Test
Gender Male * .058ns * .441ns
Female * *
Classification Sophomore * .066ns * .852ns
Junior * *
College CAS * .844ns * .369 ns
CEAT * *
* = significant ns = not significant

On the other hand, Table 7 shows that each of the socio-demographic characteristic exerts no significant difference in the pre-test scores of the respondents both in the control and treatment groups. However, the results showed that there were significant differences in the post-test scores of respondents between the control and treatment groups. Based on Mann-Whitney tests, there were significant differences in the scores on gender, classification, and college affiliation between the two groups, with the treatment group gaining more knowledge than the control group. The significant difference of scores based on the socio-demographic characteristic also supports the earlier result that the treatment group gained more knowledge than the control group.

Table 7. Effect of socio-demographic characteristics to the respondents’ knowledge gain between the control group and treatment group

Socio-Demographic Characteristic Pre-test Post-test Mann- Whitney Test
Control versus Treatment Control versus Treatment
p-value
Gender Male .902ns * *
Female .349ns * *
Classification Sophomore .704ns * *
Junior .609ns * *
College CAS .316ns * *
CEAT .924ns * *
* = significant ns = not significant

C.) Evaluation of the Learning Object

Based on the criteria of the Learning Object Peer Review Rubric adapted from the Wisconsin Online Resource Center Interactive Learning Objects Quality Standards and using the Likert scale for evaluation, the effectiveness of the LO was determined in terms of attractiveness, clarity, and comprehensibility, applicability, interactivity, and assessment function.

Attractiveness

Table 8 shows that the majority of the respondents (75.64%) exhibited a positive response indicating that the LO is appealing both in font styles and sizes. Of the 78 students, 18 (23.08%) strongly agreed, 41 (52.56%) agreed, while 6 respondents (7.69%) strongly disagreed.

Similarly, the majority of the respondents (91.03%) indicated that the text used was legible with almost 44% of them strongly agreeing. As to the embedded visual used, more than half (56.41%) agreed that it was not distracting; however, 21.79% indicated otherwise. Some of the respondents suggested to reduce the number of text lines per slide rather use fewer text animations and transitions should be simple to enhance its readability.

On the other hand, 92% of the respondents indicated that the graphs and charts were labeled properly and free from clutter with 55% of them agreeing.

Less than half of the respondents (47.43%) indicated that the use of color and other features such as pictures and clip arts in the LO are aesthetically pleasing. However, 23% disagreed while almost 30% neither agree nor disagree. Some of the respondents commented that the colors used were not complementing each other, too much text animations were used which made the text difficult to read, and the pop-up texts were distracting.

In their overall comments to the LO, most of the students suggested the use of other appropriate color schemes, shapes, pictures and graphics. Others commented to reduce the use of rotating text animations and transitions must be simple to make it more appealing and to further enhance its attractiveness. Other respondents also suggested to include videos relevant to the topic and to add voice narration for more information and interactivity. Nevertheless, the results showed that almost 72% of the respondents indicated that the overall layout of the LO was presented in an interesting manner.

In general, the respondents agreed and had a generally positive view towards the attractiveness of the LO with a computed weighted mean of 3.86.

Table 8. Frequencies, percentage and weighted mean values of the respondents on the attractiveness of the LO

Attractiveness SA A NAD D SD Weighted Mean
1. The use of font styles and font sizes was appealing. 18
(23.08%)
41
(52.56%)
11
(14.10%)
6
(7.69%)
2
(2.56%)
3.86
2. The text used was legible. 34
(43.59%)
37
(47.44%)
3
(3.85%)
4
(5.13%)
0
(0.00%)
3. The embedded visuals (text, pictures, graphs) used were not distracting. 17
(21.79%)
27
(34.62%)
17
(21.79%)
15
(19.23%)
2
(2.56%)
4. The graphs and charts were labeled properly and free from clutter. 29
(37.18%)
43
(55.13%)
3
(3.85%)
2
(2.56%)
1
(1.28%)
5. The use of color and other features (pictures, clip arts, etc) is aesthetically pleasing. 11
(14.10%)
26
(33.33%)
23
(29.49%)
14
(17.95%)
4
(5.13%)
6. The overall layout of the LO was presented in an interesting manner. 22
(28.21%)
34
(43.59%)
16
(20.51%)
5
(6.41%)
1
(1.28%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Clarity and Comprehensibility

Table 9 indicates that almost 95% of the respondents displayed a positive response showing that the LO has a clear purpose that is relevant to the learner.

Forty-five percent (45%) of the respondents strongly agreed that the LO reflected a measurable learning outcome while 48% agreed that it addressed content mastery as well as critical thinking ability. However, almost 12% neither agree nor disagree.

In addition, almost all (94%) of the respondents favorably indicated that the LO helped learners to understand the concept being presented.

Results showed that the respondents agreed that the LO is effective in showing clarity of purpose, learning outcomes, content mastery as well as addressing the critical thinking ability of the respondents, with a computed weighted mean of 4.38.

Table 9. Frequencies, percentage and weighted mean values of the respondents on the clarity and comprehensibility of the LO

Clarity and Comprehensibility SA A NAD D SD Weighted Mean
1. The LO shows a clear purpose (ex. it is immediately relevant to the learner. 42
(53.85%)
32
(41.03%)
3
(3.85%)
0
(0.00%)
1
(1.28%)
4.38
2. It reflects a measurable learning outcome. 35
(44.87%)
38
(48.72%)
4
(5.13%)
0
(0.00%)
1
(1.28%)
3. It addresses content mastery as well as critical thinking ability. 31
(39.74%)
37
(47.44%)
9
(11.54%)
0
(0.00%)
1
(1.28%)
4. It helps learners understand the concept that is being presented. 42
(53.85%)
31
(39.74%)
4
(5.13%)
0
(0.00%)
1
(1.28%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Applicability

As shown in Table 10, the majority of the respondents (86%) favorably indicated that the LO can be applied to courses in different subject areas with half (50%) of the respondents agreeing with the statement. However, almost 8% of the respondents neither agree nor disagreed.

Majority (54%) of the respondents agreed that the LO can be applied to different programs of study while almost all of the respondents (91%) indicated positive views that it can also be grouped into larger collections of content including traditional course structures. However, some respondents suggested including real-life applications so that students from various majors can easily relate to the topic.

Based on the rating given by the respondents, results showed that the respondents agreed that the LO was effective in terms of its perceived applicability with a computed weighted mean of 4.16.

Table 10. Frequencies, percentage, and weighted mean values of the respondents on the applicability of the LO

Applicability SA A NAD D SD Weighted Mean
1. It can be applied to courses in different subject areas. 28
(35.90%)
39
(50.00%)
6
(7.69%)
4
(5.13%)
1
(1.28%)
4.16
2. It can be applied to different programs of study 26
(33.33%)
42
(53.85%)
6
(7.69%)
3
(3.85%)
1
(1.28%)
3. Can be grouped into larger collections of content, including traditional course structures. 26
(33.33%)
45
(57.69%)
4
(5.13%)
2
(2.56%)
1
(1.28%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Interactivity

As shown in Table 11, almost all (90%) of the respondents indicated positive views that the LO necessitated interaction between the learner and the learning material, which suggests responding and acting to apply higher-order thinking skills. Thirty-eight percent (38%) strongly agreed on that statement.

Additionally, two-thirds (68%) of the respondents favorably indicated that the LO can stand alone, that it is not dependent on other sources such as textbook chapters and videos.

Meanwhile, 33% of the respondents strongly agreed that the LO contains all the information and materials needed to complete the activity. However, 18% of the respondents neither agree nor disagree.Results also showed that great majority (88%) of the respondents believed that the LO is easy to use with almost half (49%) of the respondents strongly agreeing on the statement. However, some students commented that controls were difficult to use with a mouse, and using arrow keys on a keyboard would be relatively easier.

In summary, respondents agreed that the LO was able to support usability and navigation to ensure the independence of its use with a computed weighted average of 4.14.

Table 11. Frequencies, percentage and weighted mean values of the respondents on the interactivity of the LO

Interactivity SA A NAD D SD Weighted Mean
1. Requires interaction on the part of the learner with the learning materials, i.e. responding and acting to apply higher- order thinking skills. 30
(38.46%)
40
(51.28%)
4
(5.13%)
3
(3.85%)
1
(1.28%)
4.14
2. It can stand alone (it is not dependent on external sources (textbook chapters, videos). 29
(37.18%)
24
(30.77%)
17
(21.79%)
8
(10.26%)
0
(0.00%)
3. Contains all information and materials needed to complete the activity (ex. introduction, summary, learning content). 26
(33.33%)
35
(44.87%)
14
(17.95%)
2
(2.56%)
1
(1.28%)
4. It is easy to use for the learner. 38
(48.72%)
31
(39.74%)
8
(10.26%)
0
(0.00%)
1
(1.28%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Assessment

Results in Table 12 shows that almost all (94%) of the respondents gave a positive response that the LO has an assessment that measures the achievement of the stated objective. This is supported by more than half of the respondents agreeing (53.85%) on the said statement. With regard to the responses on whether the LO has an assessment that provides feedback, 52.56% of the respondents agreed while two (2.56%) respondents disagreed.

On the other hand, half (50%) of the respondents strongly agreed that the assessment type was appropriate while more than half (55.13%) agreed that the “Self-Check” or practice assignments provided for quick learner feedback. However, some students suggested having more questions on “Test Yourself” ranging from easy to difficult items. They also prefer to have more examples, word problems, definitions and explanations with regard to the correct answer to the questions.

Overall, the respondents agreed that the LO was effective in its assessment with a computed weighted mean of 4.28.

Table 12. Frequencies, percentage, and weighted mean values of the respondents on the assessment of the LO

Assessment SA A NAD D SD Weighted Mean
1. It has an assessment that measures the achievement of the stated objective. 31
(39.74%)
42
(53.85%)
4
(5.13%)
0
(0.00%)
1
(1.28%)
4.28
2. It has an assessment that provides feedback. 29
(37.18%)
41
(52.56%)
6
(7.69%)
2
(2.56%)
0
(0.00%)
3. It has an assessment type that is appropriate. 33
(42.31%)
39
(50.00%)
4
(5.13%)
2
(2.56%)
0
(0.00%)
4. It has “Self- Check” or practice assignments are provided for quick learner feedback. 29
(37.18%)
43
(55.13%)
4
(5.13%)
1
(1.28%)
1
(1.28%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

D.) Enhancement of Learning of the Respondents

The respondents were also asked if the LO enhanced their learning on the topic. Table 13 shows that almost all (92%) of the respondents gave a very positive rating. With a computed weighted mean of 4.27, the respondents agreed that the LO enhanced their learning and they deemed it an effective supplemental interactive learning material on Rotation of Rigid Bodies. This perceived enhancement in learning by the respondents supports the earlier conclusion that as an addition to regular classroom discussion, the use of the LO greatly helped increase the knowledge gain of students.

Table 13. Frequencies, percentage and weighted mean values of the respondents on the enhancement of learning on the use of the LO

The Learning Object enhance my learning on the topic SA A NAD D SD Weighted Mean
28
(35.90%)
44
(56.41%)
5
(6.41%)
1
(1.28%)
0
(0.00%)
4.27
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Conclusions and Recommendations

The evaluation of the LO on Rotation of Rigid Bodies by selected UPLB students revealed that it is an effective supplemental interactive learning material that can enhance the knowledge gain of the students. Most respondents also agreed that the LO is aesthetically pleasing although improvements can still be made with regard to its fonts, color scheme, graphics, and animation. In addition, the LO is successful in showing clarity of purpose and learning outcomes as well as on its perceived applicability and interactivity. Overall, based on the weighted mean of each criterion, all values indicate that the LO on Rotation of Rigid Bodies is an effective tool for supplementary teaching and learning of students.

With these results, it is highly recommended that LO viewing be encouraged in supplementing various topics in a classroom discussion since it is found to be beneficial in enhancing student learning. The development of more LOs with readable fonts, simplified visuals/graphics, and more test item questionnaires is also recommended.

References

Beck, Robert J. (2001, May 17).What Are Learning Objects? Retrieved from http://www4.uwm.edu/cie/learning_objects.cfm?gid=56

Cisco Systems, Inc.(1999). Cisco Systems Reusable Information Object Strategy Definition, Creation Overview, and Guidelines. Retrieved from https://web.archive.org/ web/20130530223125/http://www.cisco.com/warp/public/779/ibs/solutions/learning/ whitepapers/el_cisco_rio.pdf

Frydenberg, J. (2002, October). Quality standards in e-learning: A matrix of analysis. Retrieved from The International Review of Research in Open and Distributed Learning website: http://www.irrodl.org/index.php/irrodl/article/view/109/189

Griffith, R., & Academic ADL Co-Lab, Staff. (2003). Learning Objects in Higher Education. Retrieved from http://www.academiccolab/org/resources/webct_learning objects.pdf

Iskwiki. (2013). Learning Object Peer Review Rubric Adapted from Wisconsin Online Resource Center Interactive Learning Objects Quality Standards. Retrieved from http://iskwiki.upd.edu.ph/index.php/Quality_Standards_of_Online_Instructional_ Materials#.W1pnXmQza2w

Reid, L. (2019). Learning management systems: The game changer for traditional teaching and learning at adult and higher education institutions. Global Journal of Human Social Science: G Linguistics & Education, 19(6). Retrieved from https://globaljournals.org/ GJHSS_Volume19/1-Learning-Management-Systems.pdf>





International Journal on Open and Distance e-Learning, Vol. 5 No. 1, June 2019 Issue