Assessment of Supplementary Learning Resource Material on Selected Topic in Economics

Beverly R. Pabro¹ and Maribel L. Dionisio-Sese²

¹University Extension Specialist, Interactive Learning Center, University of the Philippines Los Baños, Philippines, [email protected]
²Professor & Director, Institute of Biological Sciences, Interactive Learning Center, University of the Philippines Los Baños, Philippines, [email protected]

Abstract

Keywords: interactive learning, Learning Object, learning resource material

Introduction

The field of e-learning is changing so rapidly that there is a growing need to provide excellent and effective pedagogical models and assessment programs, which involves the development of quality materials. As more higher education institutions begin to appreciate and embrace e-learning that involves technology-mediated teaching strategies and a variety of tools to facilitate learning, it has also become imperative to have high standards in developing these learning materials. Doing so will ensure the quality and consistency in its creation and use.

One of the online educational materials available that is used as an innovative approach in offering curricular programs are the Learning Objects (LOs), which are short, self-contained, reusable teaching materials that can be aggregated for a larger collection of contents, and tagged with metadata (Beck, 2010). Each LO is a collection of content items, practice items, and assessment items that are combined based on a single learning objective (Cisco Systems 1999). They are small in size and can take on a variety of different shapes, formats, and purposes. According to Griffith et al. (2003), most institutions reported at least a consensus that LOs can be used in all instructional environments, including campus-based (face-to-face and/or traditional) as well as all types of online instruction which are instructor-led and self-paced. They can also be used to illustrate, support, supplement, or assess student learning.

To support the interactive learning process, the Interactive Learning Center (ILC) at the University of the Philippines Los Baños (UPLB) initiated the development and production of multimedia materials, specifically LOs primarily for undergraduate program courses. Among the LOs developed is the Consumption Function for Economics, which is the focus of this study.

Objectives

The general objective of the study is to assess the effectiveness of the LO on Consumption Function as supplementary learning resource material in teaching Economics. Specifically, it aims to determine the effect of LO exposure on the knowledge gain of the students, assess its effectiveness based on its various quality components and propose recommendations to improve the LO as an interactive learning tool.

The conceptual framework for this study illustrates the factors that will affect the knowledge gain of the students, which is considered as the dependent variable in the study (Figure 1). Effectiveness of the LO was determined if there is a significant increase in scores from pre-test to post-test. On the other hand, the independent variable pertains to the exposure to LO. Using the quality standards (attractiveness, clarity, and comprehensibility, applicability, interactivity, and assessment) as variables, the effectiveness of the LO was determined.

In addition, the respondents’ socio-demographic characteristics as an intervening variable were considered in order to determine whether it may affect the relationship between the independent variable and the dependent variable.

Conceptual Framework

Figure 1. Conceptual Framework of the study

Methodology

The study was conducted during the First Semester Academic Year 2017-2018. The LO on Consumption Function (Figure 2) was evaluated by 134 students in ECO 11 (General Economics).

The pre-test was administered to all the students using a five-item questionnaire about the evaluation of LO. After the pre-test, the students were separated randomly into the control group, who were exposed only to their regular classroom discussion, and the treatment group exposed to a regular classroom discussion plus the LO viewing. Both groups were afterward given the post-test to evaluate their knowledge gain. The mean scores, t-test, and z-test were then computed for the analysis of the obtained responses. On the other hand, the Mann-Whitney test and Kruskal-Wallis test were employed to determine if socio-demographic characteristics affect the knowledge gain of respondents.

In addition, following the Learning Object Peer Review Rubric Adapted from Wisconsin Online Resource Center Interactive Learning Objects Quality Standards (2013), the Treatment group was requested to assess the LO based on the following quality components: a) Attractiveness, b) Clarity and Comprehensibility, c) Applicability, d) Interactivity and e) Assessment. Likert scale was adopted to evaluate the students’ responses with the following notations: 1-strongly disagree (SD), 2-disagree (D), 3-neither agree nor disagree (NAD), 4-agree (A), and 5-strongly agree (SA). Recorded frequencies in each category were used in computing for the weighted mean values to aid in the analysis of the obtained responses.

Figure 2. The LO on Consumption Function.

Results and Discussion

A. Respondents’ Profile

As shown in Table 1, the majority of the respondents in the control and treatment groups were female. For the control group, females comprise 41 out of 75 students (54.67%) and 45.33% were male. For the treatment group, 67.80% were female while only 32.20% of the respondents were male.

Table 1. Gender of the respondents

Gender	Control		Treatment
	N	%	N	%
Female	41	54.67	40	67.80
Male	34	45.33	19	32.20
Total	75	100	59	100

Table 2 shows that more than half (56%) of the respondents for the control group were junior students while 54% of the respondents from the treatment group were senior students. Least among the respondents for the control group was senior students (16.67%) while sophomore students (8.47%) numbered least for the treatment group.

Table 2. Classification of the respondents

Classification	Control		Treatment
	N	%	N	%
Sophomore	22	29.33	5	8.47
Junior	42	56.00	22	37.29
Senior	11	16.67	32	54.29
Total	75	100	59	100

Regarding the college affiliations of the respondents, students from the CAFS (44.00%) dominated the control group. While students from CAFS (28.81%), together with CHE students (23.73%) dominated the respondents of the treatment group (Table 3).

Table 3. College affiliation of the respondents

College	Control		Treatment
	N	%	N	%
College of Agriculture and Food Science (CAFS)	33	44.00	17	28.81
College of Arts and Sciences (CAS)	14	18.67	9	15.25
College of Development Communication (CDC)	1	1.33	3	5.08
College of Engineering and Agro-Industrial Technology (CEAT)	2	2.67	6	10.17
College of Economics and Management (CEM)	6	8.00	5	8.48
College of Forestry and Natural Resources (CFNR)	11	14.67	4	6.78
College of Human Ecology (CHE)	8	10.67	14	23.73
College of Veterinary Medicine (CVM)	0	0	1	1.70
TOT	75	100	59	100

B. Knowledge Gain of Respondents

Knowledge gain pertains to the performance of the students based on the significant difference in their mean scores in the pre-test and post-test. The results in Table 4 showed that based on the t-test at 5% level of significance, there was a significant difference between the mean pre-test and post-test scores of the respondents in the control group, indicating that regular classroom discussion helped increase the knowledge gain of students. Similarly, there was a significant difference between the mean pre-test and post-test scores of the respondents in the treatment group. It indicates that regular classroom discussion plus the use of the LO helped increase the knowledge gain of students.

Table 4. Mean scores of the respondents in the pre-test and post-test and their computed t-test and z-test values

Respondents	Mean score		T-test(P=0.05)
	Pre-test	Post-test
Control	3.07	3.71	4.33*
Treatment	2.92	4.83	12.64*
Z-test (P=0.05)	0.07ⁿˢ	7.83*

* = significant
ns = not significant

Although the control group has a higher mean pre-test score than the treatment group, based on the z-test at 5% level of significance, there was no significant difference in their mean scores. It indicated that both groups of students have the same level of knowledge on Consumption Function before class discussion or LO viewing. On the other hand, when their mean post-test scores were put to a comparison, a significant difference was observed, with the treatment group gaining more knowledge than the control group. This significant improvement in the mean post- test score of the former is attributable to the use of LO as a supplementary learning resource material. From the results, it is advisable then to supplement regular classroom discussions with LO to enhance the information that will be assimilated by students on a particular subject matter.

To determine if the socio-demographic characteristics like gender, classification, and college affiliation of the respondents affected the observed significant differences in their mean pre- test scores and post-test scores, the Mann-Whitney and Kruskal-Wallis tests were used. The results revealed no significant differences in all (data not shown), an indication that the observed increases in knowledge gain of the control or treatment group are not influenced by their gender, classification, or college affiliation.

C. Evaluation of the Learning Object

Based on the criteria of the Learning Object Peer Review Rubric that was adapted from the Wisconsin Online Resource Center Interactive Learning Objects Quality Standards and also from using the Likert scale for evaluation, the effectiveness of the LO was determined in terms of attractiveness, clarity and comprehensibility, applicability, interactivity, and assessment function. Recorded frequencies in each category were used in computing for the weighted mean values to aid in the analysis of the obtained response.

Attractiveness

Table 5 indicated that the majority of the respondents (86.44%) find the LO appealing both in terms of font styles and sizes. Of the 59 students, only eight (13.56%) neither agreed nor disagreed. Likewise, most of the respondents (96.81%) indicated that the text was legible, with 40 out of the 59 students (67.80%) strongly agreeing.

With regard to the embedded visuals of the LO, 49.15% of the students strongly agreed that they were not distracting while only ten students (10.16%) neither agreed nor disagreed. On the other hand, fifty-five (55) out of the 59 students (93.22%) indicated that the graphs and charts were labeled properly and free from clutter, with only four (6.77%) of them neither agreeing nor disagreeing.

Almost 80% of the respondents indicated that the use of color, pictures, and clip arts in the LO are aesthetically pleasing. Six respondents (10.17%), however, neither agreed nor disagreed and one student (1.69%) strongly disagreed. Some students suggested the following to improve the LO’s attractiveness: (a) use other fonts like sans serif, (b) adjust the brightness of colors, and (c) add more pictures and graphics. In terms of the overall layout, 88.13% of the respondents indicated that the LO was presented in an interesting manner although some students commented that it can still be improved and the topics can be presented better.

In general, however, the respondents gave a positive view of the attractiveness of the LO with a computed weighted average of 4.39.

Table 5. Frequencies, percentage and weighted mean values of respondents on the attractiveness of the LO

Attractiveness Criteria	SA	A	NAD	D	SD	WEIGHTED MEAN
1. The use of font styles and font sizes was appealing.	2 6 (44.07%)	25 (42.37%)	4 (6.78%)	4 (6.78%)	0 (0.00%)	4.39
2. The text used was legible.	4 0 (67.80%)	17 (28.81%)	2 (3.39%)	0(0.00%)	0 (0.00%)
3. The embedded visuals (text, pictures, graphs) used were not distracting.	2 9 (49.15%)	2 4 (40.68%)	3 (5.08%)	3 (5.08%)	0(0.00%)
4. The graphs and charts were labeled properly and free from clutter.	3 8 (64.41%)	17 (28.81%)	3 (5.08%)	1 (1.69%)	0 (0.00%)
5. The use of color and other features (pictures, clip arts, etc.) is aesthetically pleasing.	2 9 (49.15%)	18 (30.51%)	6 (10.17%)	5 (8.47%)	1 (1.69%)
6. The overall layout of the LO was presented in an interesting manner.	3 1 (52.54%)	21 (35.59%)	5 (8.47%)	2 (3.39%)	0 (0.00%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Clarity and Comprehensibility

Table 6 shows that 98.31% of the respondents indicated that the LO has a clear purpose which is relevant to the learner. More than half of the respondents (54.24%) strongly agreed that the LO reflected a measurable learning outcome while 44.07% strongly agreed that it addressed content mastery as well as critical thinking ability. Nevertheless, four (6.78%) of the respondents, neither agreed nor disagreed on this statement, and two (3.39%) specified disagreements. In addition, 94.92% of the respondents favorably signified that the LO helped learners to understand the concept being presented.

Overall, the respondents agreed that the LO is effective in showing clarity of purpose, comprehensibility of learning outcomes, content mastery, and is able to address the critical thinking ability of the respondents with a computed weighted average of 4.48.

Table 6. Frequencies, percentage and weighted mean values of respondents on the clarity and comprehensibility of the LO

Clarity and Comprehensibility Criteria	SA	A	NAD	D	SD	WEIGHTED MEAN
1. The LO shows a clear purpose, i.e., it is immediately relevant to the learner.	38 (64.41%)	20 (33.90%)	1 (1.69%)	0 (0.00%)	0 (0.00%)	4.48
2. It reflects a measurable learning outcome.	32 (54.24%)	23 (38.98%)	3 (5.08%)	1 (1.69%)	0 (0.00%)
3. It addresses content mastery as well as critical thinking ability.	26 (44.07%)	27 (45.76%)	4 (6.78%)	2 (3.39%)	0 (0.00%)
4. It helps learners understand the concept that is being presented.	34 (57.63%)	22 (37.29%)	3 (5.08%)	0 (0.00%)	0 (0.00%)

Applicability

Majority of the respondents (89.83%) favorably indicated that the LO can be applied to courses in different subject areas with 50.85% of the respondents strongly agreeing on this statement (Table 7). However, four (5.08%) students disagreed or strongly disagreed on this statement. More than half of the respondents (54.24%) strongly agreed that the LO can be to different programs of study while almost 92% signified that it can also be grouped into larger collections of content, including traditional course structures.

Based on the rating given by the respondents, results showed that the respondents agreed that the LO was effective in terms of its perceived applicability with a computed weighted mean of 4.40.

Table 7. Frequencies, percentage and weighted mean values of respondents on the applicability of the LO

Applicability Criteria	SA	A	NAD	D	SD	WEIGHTED MEAN
1. It can be applied to courses in different subject areas.	30 (50.85%)	23 (38.98%)	3 (5.08%)	2 (3.39%)	1 (1.69%)	4.40
2. It can be applied to different programs of study.	31 (54.24%)	23 (38.98%)	4 (6.78%)	0 (0.00%)	1 (1.69%)
3. It can be grouped into larger collections of content, including traditional course structures.	32 (52.54%)	23 (38.98%)	3 (5.08%)	0 (0.00%)	1 (1.69%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Interactivity

As shown in Table 8, 42.37% of the respondents strongly agreed that the LO offered interaction on part of the learner with the learning materials, which suggests responding and acting to apply higher-order thinking skills. Almost half of the respondents (46%) also agreed on this statement. Additionally, 81.99% of the respondents indicated that the LO can stand alone or it is not dependent on other sources such as textbook chapters and videos.

Meanwhile, half of the respondents (50.85%) strongly agreed that the LO contains all the information and materials needed to complete the activity. However, 8.47% of the respondents neither agreed nor disagreed, and with three (5.08%) students disagreeing or strongly disagreeing. Results also showed that almost 95% of the respondents believed that the LO is easy to use with more than half (54.24%) of the respondents strongly agreeing on this statement.

Still, some students commented there is a need to improve the flow and transition of the LO by having a replay or back button on each part so that there is no need to go back to the main menu. Some of the students also suggested to include an option that will let the user adjust the volume of audio and to regulate the speed of the video, based on the user’s preferences.

In general, the respondents agreed that the LO was able to support usability and navigation to ensure independence of its use, having a computed weighted average of 4.30.

Table 8. Frequencies, percentage and weighted mean values of respondents on the interactivity of the LO

Interactivity Criteria	SA	A	NAD	D	SD	WEIGHTED MEAN
1. It requires interaction on the part of the learner with the learning materials, i.e. responding and acting to apply higher-order thinking skills.	25 (42.37%)	27 (45.76%)	7 (11.86%)	0 (0.00%)	0 (0.00%)	4.30
2. It can stand alone, i.e., it is not dependent on external sources (textbook chapters, videos).	22 (37.29%)	26 (44.07%)	7 (11.86%)	4 (6.78%)	0 (0.00%)
3. It contains all information and materials needed to complete the activity, e.g., introduction, summary, learning content.	30 (50.85%)	21 (35.59%)	5 (8.47%)	2 (3.39%)	1 (1.69%)
4. It is easy to use for the learner.	32 (54.24%)	24 (40.68%)	1 (1.69%)	2 (3.39%)	0 (0.00%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Assessment Function

The results in Table 9 shows that almost 97% of the respondents signified that the LO has assessment items that measure the achievement of the stated objectives. This is supported by 44.07% of the respondents strongly agreeing and 52.54% agreeing on the said statement. With regard to the responses on whether the LO has assessment items that provide feedback, 84.74% of the respondents strongly agreed or agreed while four (6.78%) respondents disagreed.

Almost 46% of the respondents strongly agreed that the assessment type is appropriate while about 48% strongly agreed that the “Self-Check” or practice assignments provided for quick learner feedback. However, some students suggested having more questions ranging from easy to difficult items. They also prefer to have more examples and explanations concerning the correct answer to the questions.

Overall, the respondents agreed that the LO was effective in its assessment function with a computed weighted mean of 4.30.

Table 9. Frequencies, percentage and weighted mean values of respondents on the assessment function of the LO

Assessment Criteria	SA	A	NAD	D	SD	WEIGHTED MEAN
1. It has an assessment that measures the achievement of stated objective.	26 (44.07%)	31 (52.54%)	2 (3.39%)	0 (0.00%)	0 (0.00%)	4.30
2. It has an assessment that provides feedback.	19 (32.20%)	31 (52.54%)	5 (8.47%)	4 (6.78%)	0 (0.00%)
3. It has an assessment type that is appropriate.	27 (45.76%)	29 (49.15%)	2 (3.39%)	0 (0.00%)	1 (1.69%)
4. It has “Self-Check” or practice assignments are provided for quick learner feedback.	28 (47.46%)	24 (40.68%)	5 (8.47%)	2 (3.39%)	0 (0.00%)
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Enhanced Learning

Lastly, it was determined if the students perceived that the LO enhanced their learning on the topic. Table 10 shows that 94.92% of the respondents gave a very positive rating confirming that the LO has enhanced their knowledge of the subject matter. With a computed weighted mean of 4.36, the respondents agreed that the LO is an effective supplemental interactive learning material on Consumption Function.

Table 10. Frequencies, percentage and weighted mean values of respondents on the enhancement of learning due to the use of LO

The Learning object enhanced learning on the topic.	SA	A	NAD	D	SD	WEIGHTED MEAN
	24 (40.68%)	32 (54.24%)	3 (5.08%)	0 (0.00%)	0 (0.00%)	4.36
SA=Strongly agree; A=Agree; NAD= Neither agree or Disagree; D=Disagree; SD= Strongly Disagree

Conclusion and Recommendations

Assessment of the LO on Consumption Function by selected UPLB students revealed that it is an effective supplementary learning resource material that can enhance the knowledge gain of the students. The results also showed that socio-demographic characteristics exerted no significant differences in the respondents’ knowledge gain between pre-test and post-test scores both for the control and treatment groups across gender, classification, and college affiliations of the respondents. Most respondents also agreed that the LO is aesthetically pleasing although improvements can still be made with regard to its fonts, color brightness, and inclusion of additional pictures and graphics. In addition, the LO is successful in showing clarity of purpose and learning outcomes as well as on its perceived applicability, interactivity, and assessment function. Overall, based on the weighted mean of each criterion, all values indicate that the LO on Consumption Function is an effective tool for supplementary teaching and learning of students.

Acknowledgments

The researchers would like to extend their utmost gratitude to Profs. Ma. Luisa G. Valera and Maria Angeles O. Catelo of Department of Economics, College of Economics and Management (DE-CEM) and students enrolled in ECON 11 during the First Semester, 2017-2018 who cooperated in the conduct of this research study.

The ILC staff, namely Marilou L. Mercado, Rommer S. Mente, Leonilo H. Torio and Marissa M. Lanao, also deserves special mention, as well as Prof. Lynie B. Dimasuay for the statistical analysis.

References

Beck, Robert J. (2001). What are learning objects? Learning Objects Center for International Education, University of Wisconsin,Milwaukee. http://www4.uwm.edu/cie/learning_ objects.cfm?gid=56

Cisco Systems Reusable Information Object Strategy Definition, Creation Overview and Guidelines. (1999) Cisco Systems, Inc. https://web.archive.org/web/20130530223125/http://www.cisco. co m/warp/public/779/ibs/solutions/learning/whitepapers/el_cisco_rio. pdf

Griffith, Rosemary and Academic ADL Co-Lab, Staff. (2003). Learning Objects in Higher Education. http://www.academiccolab/org/resources/webct_learning objects. pdf

Wisconsin Online Resource Center Interactive Learning Objects Quality Standards.(2013).Learning Object Peer Review Rubric. http://iskwiki.upd.edu.ph/index.php/Quality_ Standards_of_ Online_Instructional_Materials#.W1pnXmQza2w

International Journal on Open and Distance e-Learning, Vol. 6 No. 1, June 2020 Issue