Резюме. This research investigates the impacts of the implementation of Science, Technology, Engineering, and Mathematics (STEM) education in Environmental Chemistry course to enhance environmental literacy of prospective chemistry teacher. The research method used was quasi-experiment. Problem based learning (PBL) approach was implemented at the control group, while PBL-STEM was implemented at the experimental group. The dimensions of Environmental literacy to be concerned were environmental competence, environmental knowledge, and attitudes towards environment. The results of research reveal that PBL-STEM education can be used as an effective approach to improve the environmental literacy of prospective chemistry teachers. The average improvement of environmental literacy and its dimensions in the experimental group was significantly higher than the control group.

Ключови думи: environmental literacy; PBL; prospective chemistry teachers; STEM education

Introduction

Some environmental problems increasingly improve in the last decade (Kalaboikidis et al., 2015). Interrelated environmental issues make the earth an uncomfortable place to live. These environmental issues include climate change, global warming, and forest fires. An et al. (2015) revealed that forest fires is one of causes of climate change and global warming.

The most severe forest fires once occurred in Indonesia in 2015. It caused 24 deaths, over 600,000 people suffering from upper respiratory infections, over 60 million people exposed to smoke, 2.61 million hectares of burnt land, and Economic losses estimated at Rp 221 trillion (Koesmawardhani, 2015). However, the threat to environmental degradation is more worrying because the impact of this problem will be felt for the next hundreds of years.

Ninety percent of forest fires are caused by intentional factors (Prabowo, 2015). It is also caused by the lack of education about the environment obtained by citizen. In addition, the stigma of many people who argue that humans cannot control the state of the environment is also another factor that cause forest fires (Pavalache-Ilie & Unianu, 2012). Therefore, it is very important to provide environmental education to society to keep the environment well maintained.

Environmental education programs in Indonesia have been conducted since 1975. 1) The government has regulated and supervised the implementation of environmental education programs for primary and secondary levels, however, it has not been implemented to university level. Therefore, the program is considered necessary to be integrated in courses in university to help students to get environmental education.

Environmental education programs can be integrated into courses, such as Environmental Chemistry. Based on the environmental education program and oriented in the Indonesia curriculum of national higher education, Indonesian Chemical Society2) has formulated the objectives of Environmental Chemistry lecture, such as to provide the ability of implementing science and technology, to help students to make decisions, to guide students to be responsible, to improve the quality of life through the mastery of concepts related to Environmental Chemistry.

The results of survey of Environmental Chemistry course in several Chemical Education study program in Indonesia revealed that this course is presented by lectures, discussions, giving assignments, and presentations (Farwati et al., 2015). These ways only fulfill some demands of Indonesia curriculum of national higher education. Therefore, another innovative way is required, thus the Environmental Chemistry course can be a bridge to fulfill the demands.

The Environmental Chemistry course learns environmental issues from chemical standpoint. Environmental problems are appropriate to be analyzed by using multidisciplinary approach (Norman et al., 2006). Thus, an innovative way of presenting Environmental Chemistry course is integrating other relevant fields of science and oriented to the demands of Indonesia curriculum of national higher education.

STEM education can be applied as an alternative learning approach. This approach is one of the multidisciplinary approaches (Doerschuk et al., 2016). STEM stands for science, technology, engineering, and mathematics. Implementation of STEM education in lectures aims to increase STEM literacy (Zollman, 2012). Each element of the S-T-E-M plays an important role in the course, whether as the content being studied, the tools used, the product produced, and the process in the course. Naturally, STEM education can be used as an approach to improve the quality of students related to the purpose of lectures.

Some research has been done regarding the implementation of STEM education. The implementation of STEM education has been proved as an approach to increase students’ concept (Han et al., 2015; Jones et al., 2013). STEM education is also considered to be able to improve students’ problem-solving skills (Huang et al., 2016; Traphagen & Traill, 2014), to enhance critical thinking (NRC, 2012), to increase reasoning (Alfieri et al., 2015; Ding et al., 2014), and to develop science literacy skill of students of elementary level (Afriana et al., 2016) as well as secondary level (Ismail et al., 2016; Khaeroningtyas et al., 2016).

The development of literacy capabilities is one of the goals of STEM education (Benjamin et al., 2017). One important part of the ability is about environmental quality (Bybee, 2008). Maintaining environmental quality is an obligation to everyone. Maintaining environmental quality can be done through small units such as STEM-based learning (Wright et al., 2015). This research focused on an investigation of the impact of the implementation of

STEM education on environmental literacy through Environmental Chemistry lecture.

Environmental Chemistry course is a medium for students to implement their knowledge, to develop their ideas and to reconstruct the environment. These abilities are a part of environmental literacy (Paden, 2012). Environmental literacy indicates by three dimensions, namely environmental competence, environmental knowledge, and attitudes towards the environment. 3) These three dimensions are needed by prospective chemistry teachers to make decisions in solving environmental problems. 4) In other words, the improvement of these three dimensions is very important for prospective chemistry teachers, so that the prospective chemistry teachers can provide appropriate and relevant solutions. Therefore, the description of the capabilities accommodated by the Environmental Chemistry course based on Indonesia curriculum of national higher education is related to the concept of environmental literacy.

The design of course was developed based on environmental literacy. Several researches indicated that the improvement in environmental literacy is in line with the implementation of PBL (Ardianto & Rubini, 2016; Shume, 2016). In addition, environmental literacy can be also enhanced through inquiry-based lectures (Adler et al., 2016), technology-based (Storksdieck, 2016), and fieldtrip study (Mazanal et al., 1999).

Based on literature studies, the improvement of environmental literacy by using STEM education has not ever been investigated (Henderson et al., 2011). This research was focused on the implementation of STEM education integrated to PBL model to enhance environmental literacy. This research employed the prospective chemistry teachers as research subjects. The research question of this study is “How is the impact of PBL-STEM to the environmental literacy of the prospective chemistry teachers in environmental chemistry course?”

Research methodology

The impact of the implementation of STEM education on environmental literacy of the prospective chemistry teachers can be investigated from the different average improvement of their abilities. Therefore, this research used quasi experimental method with pre-posttest control group design. One research class was used as a control group. This group implemented PBL. Another research class was used as an experimental group. This group implemented PBL-STEM. Thus, the improvement in the experimental group became indication of the impact of STEM education.

The two research classes involved were prospective chemistry teachers’ classes with Environmental Chemistry courses in academic year 2014/2015 at one university in Palembang city, South Sumatra, Indonesia. Both classes have similar characteristics, such as, homogeneous GPA of prospective chemistry teachers, female sex, and age between 19-21 years. In addition, the facilities owned by the two classes were relatively similar among others, such as similar classroom size, air conditioning, and similar class organization.

The topic studied by the two classes was about air pollution. This topic was discussed for 6 meetings x 100 minutes. The pre-test and post-test were done in the first and sixth meetings. The learning process was conducted along the second to the fifth meetings by using PBL (control class) and PBL-STEM (experimental class). Table 1 presents a summary of the lecture activities in the two classes.

The pre-test and post-test was done on using the same instrument. The research instrument consisted of 12 items of essay to measure environmental competence, 16 items of multiple choice to measure environmental knowledge, and 15 items to measure attitudes towards the environment measurement. Each item of the essay question had different values, so the answer for each item was written in the rubric. Each item of multiple choice questions had 1 score for the correct answer. Meanwhile, every item of attitude scale has scale between 1-5 (Likert scale). The instrument of this study had been validated by 7 experts and the reliability was measured on using Cronbach’s alpha coefficient of 0.73.

The score of environmental literacy was the total scores of the assessment of the three dimensions of literacy. The pre-test and post-test scores of environmental literacy and its dimensions became the basis for calculating the average improvement in the experimental and control groups. Then, the mean score was used to tested the significance of the different between two groups on using SPSS program. The effect size was also measured by using an online calculator. 5)

The results of statistical calculation were then used to describe the findings about the impact of STEM education on the prospective chemistry teachers’ environmental literacy and its dimensions. These findings were then triangulated with observational results during the implementation of PBL-STEM in the experimental group. In addition, relevant national or international journals were also used to support these findings.

Findings

The impact of PBL-STEM education to the environmental literacy

Descriptive data and statistical test results from the environmental literacy evaluation are summarized in Table 2. The results revealed that the experimental group has significantly higher mean score improvement in environmental literacy than the control group. Thus, PBL-STEM education is more effective than PBL in increasing the environmental literacy of prospective chemistry teacher. These findings indicate that STEM education has positive impact on the improvement of prospective chemistry teachers’ environmental literacy.

STEM education worked well for most prospective chemistry teachers in improving their environmental literacy. This statement is based on the ES scores from the experimental group and the control group. The ES score in Table 2 of both groups indicates that almost 96% of the prospective chemistry teachers in the experimental group had higher mean scores of environmental literacy than the control group.

The impact of PBL-STEM education to the environmental competence

STEM education has been proved able to improve prospective chemistry teachers’ environmental competence. From the processing test scores, the comparison result of the mean scores of improvement in environmental competence in the experimental group is higher than in the control group. The descriptive data and statistical test results for prospective chemistry teachers’ environmental competence are presented in Table 3.

The statistical test results in Table 3 show that the prospective chemistry teachers’ environmental competence in the experimental group increased significantly with ES in the strong category. Interpretation of ES scores indicates that 96% prospective chemistry teachers in the experimental group had higher mean scores of environmental competence than the control group. These findings prove that the STEM education has worked well with almost all prospective chemistry teachers of prospective chemistry teacher.

The implementation of STEM education leads prospective chemistry teachers to think and to solve environmental problems from all areas. Thus, prospective chemistry teachers are trained to be able to identify, to use data, to analyze, and to design ways to address environmental problems. These capabilities are specific features of environmental competence.

Contextually, using the data to analyze environmental problems can be interpreted as prospective chemistry teachers’ ability to use facts to explain the phenomenon of air pollution. This ability is predicted can be mastered by the prospective chemistry teachers when they make observations on environmental issues. This activity was conducted at the first meeting of PBL-STEM-based lecturing. The prospective chemistry teachers were stimulated to recognize environmental problems in three ways. They were asked to directly observe the environment and analyze environmental problems in the air pollution videos prepared by researcher. In addition, the prospective chemistry teachers were also asked to remember the environmental problems that have occurred in South Sumatra.

Regarding the results of observation, the prospective chemistry teachers could mention the environmental problems that have been occurred. Furthermore, they were asked to explain what they felt towards the impacts of environmental problems. For example, in the case of forest fires, the majority of prospective chemistry teachers in the experimental class stated that in South Sumatra became smoke, visibility was limited, and people became difficult to breathe.

The prospective chemistry teachers’ explanations of what they felt as the impact of environmental problems were facts. Thus, PBL-STEM lecturing for the first meeting was appropriate. The design of this course has been able to improve prospective chemistry teachers’ ability in using data to explain the phenomenon of environmental problems.

The first meeting in PBL-STEM lecturing program was also able to improve the prospective chemistry teachers’ ability to identify environmental problems. In the context of this research, the ability is defined as prospective chemistry teachers’ capability in determining or establishing evidence of environmental problems that occur around them. This ability is highly likely to be mastered through STEMbased lecturing. When STEM education was implemented in the classroom, prospective chemistry teachers were guided to put forward the ideas in overcoming the environmental problems, both local and global problems.

Each group learned how to convey different ideas depending on the point of view of each group. The different ideas shown when the prospective chemistry teachers tried to solve forest fire problems. The ideas stated by each group when they tried to overcome forest fires are presented in Fig. 1.

Based on the ideas of overcoming forest fires that have been expressed by each group, there are at least three points of view for one environmental problem. These three viewpoints are viewed from materials that can be used to extinguish fires, simple tools to extinguish fires, and simple tools to clean smoke from forest fires. This discussion activity has opened the prospective chemistry teachers’ mind in reviewing one problem. The design of this lecture made prospective chemistry teachers accustomed to thinking comprehensively about environmental issues. Therefore, prospective chemistry teachers could easily show the evidence of environmental problems in one area.

The implementation of PBL-STEM education has led the prospective chemistry teachers to work cooperatively in figuring out the way to solve environmental problems through literature review. Furthermore, the prospective chemistry teachers tested the results of these thoughts in many times to obtain the best solution. Thus, the prospective chemistry teachers’ ability to analyze environmental problems and to figure out best solution were taught.

The impact of PBL-STEM education to the environmental knowledge

The results of statistical tests indicate that environmental knowledge in the experimental and control groups show differences. The results are summarized in Table 4. STEM education has had a positive impact to prospective chemistry teachers, with ES on strong criteria. The interpretation of ES score indicated that almost 84% of the prospective chemistry teachers in the experimental group had higher mean than the control group.

During instruction, the experimental group was given an assignment as a guide to review information about air pollution problem. This task contains some questions to guide the prospective chemistry teachers to dig up the information. The control group was assigned to make a paper on air pollution issues. Each group was given one topic of air pollution problem and written down all (unlimited) information related to the topic.

Paper-making task enables the prospective chemistry teachers to review information widely. However, the content of the papers presented by the prospective chemistry teachers in the control group, there were some key points that were not covered in the paper. The main points were about the atmosphere and regulations related to air pollution.

The information about atmosphere is very important for the prospective chemistry teachers to be learnt when discussing air pollution. Air and atmosphere are one unity. Similar with government policies related to air pollution, this point is also important for prospective chemistry teachers to learn because it can be used by prospective chemistry teachers as a consideration when proposing solutions to air pollution problems.

It is appropriate to ask prospective chemistry teachers to construct a paper. However, it would be better if expected contents that should be included are enclosed in the assignment worksheet. This strategy enables prospective chemistry teachers to obtain extensive, complete, and precise information. The strategy has been carried out in the experimental group by providing problem sheet. It is estimated that the use of this strategy makes the experimental group superior to the control group for environmental knowledge enhancement.

The problem sheets given to the prospective chemistry teachers are broadly composed of questions about air pollution. The subjects of the material include air pollution, causes and sources of air pollution, regulations on air pollution, as well as impacts and ways of dealing with air pollution. These subjects were discussed by prospective chemistry teachers in groups. The problem sheets are also provided columns to list references used. All groups learnt how to write down sources of articles in the referral list.

Similar to the implementation of STEM education, in the control group, the prospective chemistry teachers discussed the same material. The prospective chemistry teachers in this group were asked to explore some information on air pollution from various sources. The information was then written in paper form. Regarding the list of references in the paper of each group, it is known that all groups used articles from internet as source of information.

Recent references, from books and internet, are called technology. The prospective chemistry teachers in the experimental and control groups utilized internet-based technology in exploring information about air pollution. Thus, the knowledge gained by the prospective chemistry teachers in both groups is ensured similar. This statement is supported by a list of referrals written by the prospective chemistry teachers.

The impact of PBL-STEM education to attitudes towards environment

The implementation of STEM education in Environmental Chemistry lecture can be able to improve prospective chemistry teachers’ positive attitude towards environment. The descriptive data and statistical test results from the aspect of attitudes to the environment are presented in Table 5. The results of statistical tests on the mean of environmental attitudes from both groups showed that STEM education is superior in improving prospective chemistry teachers’ attitudes towards the environment. Mostly 76% of the prospective chemistry teachers in the experimental group had a higher rate of improvement than the control group.

The Table 5 shows that the improvement of mean of environmental attitudes in the experimental and control groups is low. This result is obtained because the pretest scores of attitudes towards the environment on prospective chemistry teachers in both groups are already high. It has been found that from scale of 1 to 5, one item of attitude scale in the pre-test was 51% of prospective chemistry teachers rated 4 and 36% rated 5.

As a result, most of prospective chemistry teachers achieved high score in the pre-test of attitudes towards the environment, so the mean score in Table 5 has a small chance to improve. The findings also indicate that the prospective chemistry teachers in the experimental and control groups have high self-efficacy and they already have good attitude towards the environment. Therefore, these findings serve as a basis for improving scale on instrument of attitude towards environment. This research used a scale of 1 to 5, however, it would be better to use a scale of 1 to 9 for the next research.

Attitudes towards the environment are reflected from the sensitivity and interest on the environment, locus of control, responsibility for environmental balance, and the intention to act positively to the environment. Problem-based project can guide the prospective chemistry teachers directly involved with problems that occur in their environment. This method can stimulate prospective chemistry teachers’ sensitivity towards environment, interest and care, and locus of control. In addition, project work also leads the prospective chemistry teachers to foster their responsibility. Thus, the prospective chemistry teachers are encouraged to think the most appropriate action to balance the environment.

Discussion

The results of this research have proved that Osman et al. (2013) predicted that environmental literacy capabilities can be enhanced by the implementation of STEM education. Besides, these capabilities have also been shown to increase by using the implementation of PBL (Ardianto & Rubini, 2016; Shume, 2016). However, the results of this research have shown that PBL-STEM education can maximize prospective chemistry teachers’ ability.

The dimensions of environmental competence, environmental knowledge, and attitudes towards the environment can be significantly improved by applying STEM education. These three dimensions support each other. The environmental knowledge dimension becomes the central to the other dimensions.

Han et al. (2015) and Jones et al. (2013) stated that STEM education was able to increase students’ knowledge. On the other hand, it was reported that student’s knowledge can be improved by implementing PBL (Mayer et al., 2012; Sandi-Urena et al., 2012). However, this research proves that STEM education can maximize the improvement of prospective chemistry teachers’ environmental knowledge.

The students’ environmental knowledge influences students’ attitudes towards the environment (Pe’er et al., 2007; Yount & Horton, 1992). Students with good environmental knowledge will have positive attitude towards the environment (Özgürler & Cansaran, 2014). The high/low level of the students’ positive attitudes towards the environment is also influenced by the context of the environmental issues discussed (Rohrschneider, 1988). One of the environmental problems studied in the PBL-STEM based lecturing was forest fires. This environmental problem was still highly remembered by the prospective chemistry teachers. Memories of the events and impacts of these disasters encourage the prospective chemistry teachers to show their best for the environment. Thus, experience in lectures has a significant influence on prospective chemistry teachers’ attitudes towards the environment.

The lectures with PBL-STEM education has provided the prospective chemistry teachers experience in assessing environmental issues in theory and practice. The study of environmental problems in theory, both from books and the internet, positively influences prospective chemistry teachers’ attitudes towards the environment (Mobley et al., 2016). In this way, the prospective chemistry teachers get stimulants to care about environmental issues for local, national, and international scale.

The study of environmental problems was practically conducted by the prospective chemistry teachers through applying environmental knowledge. Each member of group work contributed their knowledge to construct an environmental problem-solving design. The ability to design solution of environmental problem is part of the environmental competence dimension. 3) To construct the design, the prospective chemistry teachers firstly identifies, analyzes, and evaluates the environmental problem being studied. Thus, environmental knowledge has a role to improve prospective chemistry teachers’ environmental competence.

Conclusion

STEM education is comprehensively integrated in Environmental Chemistry course. The PBL-STEM based learning is characterized by: (a) Developing prospective chemistry teachers’ environmental knowledge through literature studies from books and internet; (b) Teaching how to apply prospective chemistry teachers’ environmental knowledge through leading them to state their ideas towards environment problems; (c) Developing prospective chemistry teachers’ positive attitude towards environment through literature review and practicing; (d) Prompting prospective chemistry teachers’ enthusiasm in learning process, appreciating others opinions, and becoming critical learners during discussion session in the class.

The implementation of PBL-STEM education in Environmental Chemistry course has significantly enhanced environmental literacy of prospective chemistry teachers. Moreover, the implementation of the innovation course enhanced all of dimensions, e.g. environmental competence and environmental knowledge (strong category) as well as attitude towards environment (medium category).

Acknowledgement. This article is partially supported by the United States Agency for International Development (USAID) through the Sustainable Higher Education Research Alliance (SHERA) Program for Universitas Indonesia’s Scientific Modeling, Approach, Research, and Training for City-centered Innovation and Technology (SMART CITY) Project, Grant #AID-497-A-16-00004, Sub Grant #IIE-00000078-UI-1.

NOTES

1. http//:www.menlh.go.id/informasi-mengenai-adiwiyata/

2. Indonesian Chemical Society (2014). Capaian pembelajaran Kimia dan isu terkait. Sidoarjo: Author.

3. https://cdn.naaee.org/sites/default/files/envliteracyexesummary.pdf

4. http://people.ucalgary.ca/~tamaratt/SciLit_files/SciLit_Review.pdf

5. https://www.uccs.edu/~lbecker/

REFERENCES

Adler, I., Zion, M. & Mevarech, Z.R. (2016). The effect of explicit environmentally oriented metacognitive guidance and peer collaboration on students’ expressions of environmental literacy. J. Res. Sci. Teaching, 53, 620 – 663.

Afriana, J., Permanasari, A. & Fitriani, A. (2016). Project based learning integrated to STEM to enhance elementary school’s students scientific literacy. J. Pendidikan IPA Indonesia, 5(2), 261 – 267.

Alfieri, L., Higashi, R., Shoop, R. & Schunn, C.D. (2015). Case studies of a robot-based game to shape interests and hone proportional reasoning skills. Int. J. STEM Educ., 2(4), 1 – 13.

An, H., Gan, J. & Cho, S.J. (2015). Assessing climate change impact on wildfire risk in the United States. Forests, 6, 3197 – 3211.

Ardianto, D. & Rubini, B. (2016). Comparison of students’ scientific literacy in integrated science learning through model of guided discovery and problem based learning. J. Pendidikan IPA Indonesia, 5(1), 31 – 37.

Benjamin, T.E., Marks, B., Demetrikopoulos, M.K., Rose, J., Pollard, E., Thomas, A. & Muldrow, L.L. (2017). Development and validation of scientific literacy scale for college preparedness in STEM with freshmen from diverse institutions. Int. J. Sci. & Math. Educ., 15, 607 – 623.

Bybee, R.W. (2008). Scientific literacy, environmental issues, and PISA 2006: the 2008 Paul F-Brandwein lecture. J. Sci. Educ. & Tech., 17, 566 – 585.

Ding, L., Wei, X., & Mollohan, K. (2014). Does higher education improve student scientific reasoning skills. Int. J. Sci. & Math. Educ., 14, 619 – 634.

Doerschuk, P., Bahrim, C., Daniel, J., Kruger, J., Mann, J. & Martin, C. (2016). Closing the gaps andfilling the STEM pipeline: a multidisciplinary approach. J. Sci. Educ. & Tech., 25, 682 – 695.

Farwati, R., Permanasari, A. & Firman, H. (2015). Studi deskriptif mata kuliah Kimia Lingkungan di beberapa LPTK di Indonesia. J. Pendidikan IPA Indonesia, 4(1), 1 – 7.

Han, S., Capraro, R. & Capraro, M. M. (2015). How science, technology, engineering, and mathematics (STEM) project-based learning (PBL) affects high, middle, and low achievers differently: the impact of student factors on achievement. Int. J. Sci. & Math. Educ., 13, 1089 – 1113.

Henderson, C., Beach, A. & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: an analytic review of the literature. J. Res. Sci. Teaching, 48, 952 – 984.

Huang, N.-T.N., Chiu, L.-J. & Hong, J.-C. (2016). Relationship among students’ problem-solving attitude, perceived value, behavioral attitude, and intention to participate in a science and technology contest. Int. J. Sci. & Math. Educ., 14, 1419 – 1435.

Ismail, I., Permanasari, A. & Setiawan, W. (2016). STEM virtual lab: an alternative approach practical media to enhance student’s scientific literacy. J. Pendidikan IPA Indonesia, 5(2), 239 – 246.

Jones, L.C.R., Tyrer, J.R. & Zanker, N.P. (2013). Applying laser cutting techniques through horology for teaching effective STEM in design and technology. Design &Tech. Educ., 18(3), 21 – 34.

Kalabokidis, K., Palaiologou, P., Gerasopoulos, E., Giannakopoulos, C., Kostopoulou, E. & Zerefos, C. (2015). Effect of climate change projections on forest fire behavior and values-at-risk in Southwestern Greece. Forests, 6, 2214 – 2240.

Khaeriningtyas, N., Permanasari, A. & Hamidah, I. (2016). STEM learning in material of temperature and its change to improve scientific literacy of junior high school. J. Pendidikan IPA Indonesia, 5(1), 94 – 100.

Koesmawardhani, N.W. (2015, December, 18). 2,61 juta Ha lahan Indonesia terbakar di 2015, 24 jiwa melayang.

Manzanal, R.F., Rodríguez Barreiro, L.M., & Casal Jiménez, M. (1999). Relationship between ecology fieldwork and student attitudes toward environmental protection. J. Res. Sci. Teaching, 36, 431 – 453.

Mayer, R., Moeller, B., Kaliwata, V., Zweber, B., Stone, R. & Frank, M. (2012). Educating engineering undergraduates: effects of scaffolding in a problem-based learning environment. Proceed. Human Factors & Ergonomic Society Annual Meeting, 56, 2507 – 2511.

Mobley, C., Vagias, W.M. & DeWard, S.L. (2016). Exploring additional determinants of environmentally responsible behavior: the influence of environmental literature and environmental attitudes. Environment & Behavior, 42, 420 – 447.

Norman, N., Jennings, A. & Wahl, L. (2006). The impact of environmentallyrelated education on academic achievement: a literature survey. Berkeley: StopWaste.

NRC [National Research Council]. (2012). Education for life and work: developing transferable knowledge and skill in the 21st-century. Washington: The National Academies Press.

Osman, K., Hiong, L.C. & Vebrianto, R. (2013). 21st century biology: an interdisciplinary approach of biology, technology, engineering and mathematics education. Procedia, 102, 188 – 194.

Özgürler, S. & Cansaran, A. (2014). Graduate students, study of environmental literacy and sustainable development. Int. Electronic J. Environ. Educ., 4(2), 71 – 83.

Paden, M. (2012). NAAEE releases framework for assessing environmental literacy being used in 2015 OECD assessment. J. Educ. Sust. Develop., 6(1), 17 – 19.

Pavalache-Ilie, M. & Unianu, E.M. (2012). Locus of control and the proenvironmental attitudes. Procedia, 33, 198 – 202.

Pe’er, S., Goldman, D. & Yavetz, B. (2007). Environmental literacy in teacher training: attitudes, knowledge, and environmental behavior of beginning students. J. Environ. Educ., 39, 45 – 59.

Prabowo, D. (2015, November 6). 7 perguruan tinggi dinilai berperan dalam atasi kabut asap, ini peranannya.

Rohrschneider, R. (1988). Citizens’ attittudes towards environmental issues: selfish or selfless. Comp. Pol. Stud., 21, 347 – 367.

Sandi-Urena, S., Cooper, M. & Stevens, R. (2012). Effect of cooperative problem-based lab instruction on metacognition and problem-solving skills. J. Chem. Educ., 89, 700 – 706.

Shume, T. (2016). Teachers’ perspectives on contributions of a prairie restoration project to elementary students’ environmental literacy. Int. J. Environ. & Sci. Educ., 11, 5331 – 5348.

Storksdieck, M. (2016). Critical information literacy as core skill for lifelong STEM learning in the 21st century: reflections on the desirability and feasibility for widespread science media education. Cult. Studies Sci. Educ. 11, 167–182.

Traphagen, K. & Traill, S. (2014). How cross-sector collaboration are advancing STEM learning. Los Altos: Noyce Foundation.

Wright, K.L., Franks, A.D., Kuo, L-.J., McTigue, E.M. & Serrano, J. (2015). Both theory and practice: science literacy instruction and theories of reading. Int. J. Sci. & Math. Educ., 14, 1275 – 1292.

Yount, J.R. & Horton, P.B. (1992). Factors influencing environmental attitude: the relationship between environmental attitude defensibility and cognitive reasoning level. J. Res. Sci. Teaching 29, 1059 – 1078.

Zollman, A. (2012). Learning for STEM literacy: STEM literacy for learning. School Sci. & Math., 112(1), 12 – 19.