https://doi.org/10.53656/ped2024-1s.11

Резюме. Online and hybrid courses on IT-related topics have been increasing in demand and interest for decades, but the recent pandemic situation has made them critical. This article presents the results of a survey related to online/hybrid learning among engineering students in Bulgarian universities. The results reveal some of the problems and attitudes towards programming education, as well as some challenges and expectations of students regarding online programming courses according to recent educational requirements. Some popular gamification, PBL and GBL strategies were used in the online learning process. Some elements of gamification were introduced into the learning process. At the end of the training, a survey was conducted to assess the opinions of the students regarding the introduction of these elements and their impact on increasing the quality of training. Some of the results are shown in the article.

Ключови думи: programming education; pedagogy aspects of digital and blended learning; gamification-based learning; project-based learning

1. Introduction

The Covid-19 Pandemic crisis enforced rapid changes in all the business and private sectors, especially the education system. Many universities worldwide suddenly had to shift all their educational activities to virtual environment in response to the pandemic. Lecturers’ professional development is crucial, especially in highly connected Internet environment they teach in, and digital course design and development is even more important than in traditional education system. Moreover, the new students’ generations have born in an online era and demand always-on digital access, more profound and more flexible learning experience (Guerrero 2021). We have seen that moving classes online – either blended or fully online – can be done rapidly, but early reports show huge variations in quality, acceptance, completion, and learning (Guàrdia 2021).

On the other hand, online and hybrid teaching brings major challenges to lecturers. In addition, one of the most significant is the numerous complexities involved in moving a face-to-face course into the virtual platforms. A recent study reveals that the barriers to the digital transformation of the educational sphere were lack of funds for the implementation of a comprehensive digital transformation strategy; resistance to changes on the part of the staff; and a low level of confidence in technological solutions used in teaching practice (Mikheev 2021). Therefore, some of the lecturers were unwilling to teach online, although when used wisely, online learning has fewer time and space constraints, 24x7 availability, making learning more flexible for both teachers and learners. As a typical online education form and a powerful substitute for the classroom, MOOC, an acronym for Massive Open Online Course, is an online course for the public and the latest development of distance education (Al-Azawi 2016).

Many universities that are unwilling to change their traditional pedagogical approach have no choice but to move entirely to online educational environment. But since the traditional presence system mostly prevails in many European countries, teachers and students in online education have problems, especially for engineering departments, due to dense curriculum on hand-zone and practical lessons. Other problems center on the experience in online teaching/learning, students’ collaboration, lack of support from lecturers, the complexity of materials provided, and inappropriate homework environment. These facts reduce satisfaction and enthusiasm for both students and educators. Therefore, compliance with online lectures with existing curricula related to engineering studies is highly necessary.

The present study examines the current technological, organisational, and pedagogical trends and challenges in programming teaching in Bulgarian universities. The project focuses on Computer Programming and Algorithms lecture that is a core element for all engineering disciplines in this project. Computer Programming and Algorithms is a complicated course to teach and learn. Today, Industry 4.0 and 5.0 topics are emerging fields and have rising demand during the pandemic period. The increasing interest in artificial intelligence and machine learning applications is closely related to them. Under current circumstances, online Computer Programming and Algorithms courses must be addressed with innovative solutions to support this level with well-educated professionals. In this project, teachers will be trained to overcome online education challenges while preparing adequate course material suits to common “Computer Programming and Algorithms” training. In this way, we aim at designing appropriate online teaching methods for this lecture and integrate them with online technologies to improve students’ learning motivation and interest, maintain students’ concentration, and enhance students’ learning.

The main project objective is to develop a well-structured online teaching method for computer programming and algorithms related courses. For this purpose, educators will get trained on online education pedagogies and didactics to face challenges with the lack of student satisfaction. Adaptation of online pedagogy and educational materials for this course will be one that emphasizes student-centered learning and employs active learning activities. These new course materials will be able to implement for both online & hybrid educational systems. In particular, the project aims at persuading educators to amalgamate immersive technologies in online educational programs while developing their pedagogical knowledge to boost student’s willingness to moreover involvement.

In the context of the above, some popular gamification strategies, project-based learning (PBL) and gamification-based learning (GBL), have been used in the online learning process. Gamification is defined as the process of enriching services with motivational opportunities to induce gaming experiences and additional behavioral outcomes (Lin 2014). In terms of learning, gamification is the process of adding game elements to change existing learning processes.

According to this concept, gamification can be seen as consisting of three main parts:

– Applied motivational opportunities;

– The obtained psychological results;

– Additional behavioral outcomes.

Learning through games is used to encourage students to engage in learning while playing and to make the learning process more interesting by adding fun to the learning process. It has a positive effect on cognitive development [6]. Some elements of gamification were introduced into the learning process, and the results of the study on their impact are shown below.

2. Description of the Applied Methodology for the Survey Related to Online/ Hybrid Learning among Engineering Students in Bulgarian Universities

2.1. Questionnaire overview

The applied methodology is online questionnaire, filled in by 109 respondents, who are students in engineering studies in Bulgarian universities. The questionnaire contains 18 questions – 16 multiple choices and 2 open-answers about the university students study in and their e-mail address.

88% of the respondents (96 answers) pointed out they study bachelor degree in engineering or IT-related fields, while 12 %, answered with “no” are excluded from the rest of questionnaire. Then 72% (78 participants) of the selected sample selected they have already participated in online programming course (live or not) and consequently consist our sample of interest, since they have experience in similar courses.

2.2. Demographic results

The demographic information about participants contains data about their gender, university, bachelor programme and year of their study. Figure 1 contains the obtained data regarding the participants’ gender, which corresponds to the average distribution of men and women in IT and engineering studies:

The prevailing part of the students: 84% (92 participants) are students from Burgas Free University, while the rest 16% are in other universities (Sofia University, South-Western University, etc.). 47,8% indicate they study Software Engineering, 18,8% – Computer Engineering, 7,2% – Electronics, and the rest study various IT-related programmes, including Media Design, Energy, Cyber Engineering, etc. This variety covers a large number of programmes in Bulgarian universities. Figure 2 contains the distribution of respondents’ year of study:

The results indicate that the largest group (37,7%) consists of students of 4th year of study, while 17,4% are 3rd year of study. Therefore, it could be concluded the respondents have enough experience in learning programming to give feasible and reliable feedback regarding their experience with online courses. Students also share information regarding the devices they use in their training (fig. 3 and 4).

It could be observed that the most used device is laptop, followed by desktop computer and smartphone. Devices like tablets and smart TVs are not popular among students for their training activities. In addition, although rarer, students still use pen and paper during training.

2.3. Questionnaire results

It is important to evaluate students’ experience with online courses. Figure 5 introduces results regarding students’ attendance of online courses:

Almost the half of the students (46,4%) have attended often, while 20,3% always have attended the course. There are very small number of students who have not attended (1,4%) or attended only once (1,4%) the course. Therefore, we could assume the sample has enough experience with online programming courses to give reliable and representative feedback.

The next figure 6 introduces the responds results of the question regarding the training type (synchronous, asynchronous or hybrid):

It could be observed that most of the online training was conducted in hybrid or asynchronous mode, while hybrid training is much more presence-oriented. This could be explained with engineering studies peculiarities, where laboratory and project-based training could hardly be replaced with online activities. Therefore, lecturers prefer to conduct presence classes, when possible, rather than shift the entire course online.

It is important to examine what are the students’ preferences regarding their course attendance way. Figure 7 contains an overview of obtained responses regarding students’ opinion:

Both in entirely online and hybrid case students prefer blended approach – synchronous and asynchronous training approach, followed by entirely synchronous learning. In both cases, purely asynchronous learning earns the lowest responses.

Figure 8 presents the elements in the online courses respondents have attended (multiple answers allowed):

The most common training elements are live lectures (76,8%), followed by video guides (68,1%) and electronic tests (58%). The least used elements appear to be social networks (11,6%), animation (14,5%) and games, collaborative whiteboard and programming in pairs (17,4%). This is not a surprise, having in mind that most lecturers in programming follow preliminarily created presentations and/or write source code into selected IDE.

Figure 9 represents respondents’ preferences regarding training elements, present in online programming courses (multiple answers allowed):

Again live lectures gain the most of the responses (63,8%), followed by video guides and simulations (52,2%) and electronic tests and communication (49,3%). The least preferred elements are “will not be online” (1,4%) and social networks (13%). The results at figure 8 and 9 reveal that there is significant coincidence between respondents’ experience and expectations, on one side. On the other side, the respondents may just expect the elements they get used to and have experience with. So it could be expected the lecturers in universities should test various training elements in their courses to enrich the variety of tools in online programming teaching.

Figure 10 contains the results about advantages of online teaching from students’ point of view (maximum 3 choices). It could be observed the students appreciate the most accessibility 24x7 (68,1%), followed by the opportunity to learn in appropriate time/place (62,3%) and read / repeat ability as many times, as needed (52,2%). The least significant advantages are pointed out the presence training is the best (1,4%), possibility of visual contact with other students (4,3%) and “no benefits” answer, selected by 5,8% of respondents. It could be concluded that the prevailing part of students appreciate the online training advantages, although there is still a small number of students who do not see any benefits.

Figure 11 presents the biggest challenges in online training for students (maximum 3 choices).

Respondents point out as the biggest challenge in online training for almost the half of students are difficulties in solving problems on their own (47,8%), followed by lack of social contacts (36,2%) and lack of technical equipment (30,4%). These are intrinsic drawbacks for online training that could be mitigated, but not eliminated at all. As least significant challenges students have selected poorly structured and outside of companies’ requirements material (1,4%) and problems with ADHD treatment in Bulgaria (1,4%). It should be mentioned that significant share (26,1%) see no problems in online training.

Figure 12 represents summary of students’ responses regarding the number of programming courses they have participated in so far:

The distribution of these answers corresponds to the distribution of the question regarding the students’ year of study. In addition, figure 13 represents summary of students’ responses regarding the number of online programming courses they have participated in so far:

It could be seen that respondents have enough experience with online programming courses and all the students have participated in at least one. Figure 14 presents the variety of programming courses students have studied:

The prevailing part of the students (73,9%) have studied C/C++, followed by C# with 39,1% and Java (36,2%). It should be mentioned that the first 5 options were previously available for selection, while the rest were added by the respondents by “Other” option.

Figure 15 presents the results about the question what programming language students would like to start their training.

Again, C/C++ gains the most answers (27,5%), followed by Python (24,6%) and Java (21,7%). So, it could be considered to include Python in earlier programming training, since there is growing interest of students.

In addition, there are also open answers, regarding online training in general:

“I am extremely pleased with the online training. It gives me real freedom of development both at university and at work.”;

“No comments, online is a unique experience!”;

“Personally, I do not consider my house a place of study and I am distracted too much, while at university I am mentally and physically inclined to study and I am concentrated to the maximum.”;

“Thank you to all the teachers for their efforts and for the fact that we have the opportunity for online learning.”

3. Description of the Applied PBL and GBL Methodology

3.1. Preparation and delivery of training

The electronic platform Moodle was selected as a support environment for the training, which allows the introduction of various elements of gamification. Appropriate courses were developed for this purpose. Regardless of the training format or variant, PBL was introduced in each of them.

Students were taught Internet of Things, Computer Networks and Communications and Business Information Systems courses entirely in the online environment. The training in “Fundamentals of Computing and Programming”, “Cyber Security” and “Design and Construction of Computer Networks” was conducted in a hybrid way – part of the training was conducted in person, the other part Online. These subjects were studied in the first, third and fourth year of Bachelor’s and in Master’s degree studies.

In the process of training in the abovementioned disciplines several methods of teaching the studied material are used. In each different method, the timetable of classes scheduled as face-to-face teaching was strictly followed.

The first method involved the use of standard lecture materials, which were delivered by the lecturer live or in video sessions that were recorded. The video sessions exactly duplicated the teaching method of face-to-face teaching and followed the timetable of the teaching hours. Exercises were conducted separately, after the respective lecture material has been delivered.

In the second method, students should be prepared by getting familiar themselves in advance with the material to be presented. In the tutorials, the lecturer presented the lecture material, emphasized the more critical points that would be difficult for the students, answered questions that have been arisen on the material, and set problems for the students to solve. Outside of the scheduled class hours, students studied the lecture materials in detail and sought solutions to the problems posed to them. In the next class, students present their solutions and comment on the results to the audience.

In the third option the learning was 100% in distance way. Students received all the necessary materials – lecture and exercise, as well as detailed instructions on how to complete their assignments. If necessary, they turned to the lecturer for advice on certain problems they encounter.

The training exercises are conducted on software simulators such as Cisco’s Packet Tracer, Autodesk Tinkercad, Circuito.Io, Wokwi, various software languages, programming environments and others that allow 100% implementation and execution of the set assignments. In some cases, for example in practical exercises in the Internet of Things discipline, students get online access to real models on which they can implement their tasks and conduct their research remotely. In faceto-face training, both real models on which students practice and simulations were used.

3.2. Conducting the research

For the purpose of the study a survey was conducted with the participation of students in Bachelor and Master degree studies from Burgas Free University - Burgas and the Higher Naval School “Nikola Ionkov Vaptsarov” in Varna. The students have studied the following disciplines: “Fundamentals of Computer Engineering and Programming”, “Computer Networks and Communications”, “Internet of Things”, “Cyber Security”, “Business Information Systems” and “Design and Construction of Computer Networks”.

The questions in the survey are divided into 9 groups of six questions with four possible answers. The questions cover students’ opinions on:

– the way the training is conducted;

– the method of teaching the course material,

– the method of assessing their progress in the subject; the use of different gamification elements in the learning process and their impact on motivation to achieve better results;

– overall satisfaction with the delivery of the training in terms of knowledge gained, adequacy of assessment, retention of learning and retention of learners’ attention in the subject area.

In processing the results, we must also consider the period in which the training was conducted and the survey was taken. The training was done in the period of October 01, 2020 to April 30, 2022. This period covers four academic semesters in two academic years. The peculiarity of the period is that during this period the country is in pandemic of Covid 19 and in certain periods the training was conducted in online environment only. Another peculiarity is that the processing of the results takes into account in which year of their studies the students studied the respective subject.

3.3. Results

Above are presented the results of the study in graphical form. Survey questions are listed, and results of respondents’ answers are shown.

To make a summary analysis of the answers of the surveyed students we will use the following methodology. To each “Rather – No” answer – 1 point. On “Rather – Yes” – 1 point and on “Yes” – 2 points. We sum the results for each answer and get the following summary graph on Fig. 25.

4. Conclusions

A. About the results of a survey related to online/hybrid learning among engineering students in Bulgarian universities

This study examines the attitudes of the students in engineering field, studying computer programming through online and hybrid mode in Bulgaria. Examining trainees’ experience, expectations and needs helps instructors to adjust the course content and delivery approach in order to achieve the best results. Teaching programming is an integral part of each engineering BSc programme and completes the basis for specialized disciplines.

According to the results obtained from the study, the attitudes of the online and hybrid education students regarding programming were generally positive. Being part of 21st century generation, the majority of students easily took part into online activities and most of them entered the course with relatively high level of computer literacy. Moreover, most of them have already participated in more than one online programming course, which brought them enough experience to give adequate feedback.

Students highlighted online training advantages, like accessibility 24x7, the opportunity to learn in appropriate time/place and the opportunity to read / repeat as many times, as needed. In addition, students reported the issues they encounter during their online training: difficulties in solving problems on their own, lack of social contacts and technical equipment. Some of these obstacles could be avoided or mitigated using additional tools in online environment: live meetings, participation in group projects, group and personal online mentoring sessions, etc.

In addition, it could be observed that a lot of work should be done in the field of digital resources development and various pedagogical approaches implementation. The variety of digital resources could be enriched with collaborative training tools, interactive resources, programming in pairs, participation in real projects, etc. This will bring students not only academic, but also practical experience needed for labor market requirements.

The questionnaire results will be used as a starting point for our future work: to formulate sustainable curriculum implementation for both hybrid and online education; to integrate new online course materials with gamification techniques to the “Computer Programming and Algorithms” curriculum, which helps students attending online; to develop synchronous and asynchronous methodologies to deliver online programming course effectively.

B. About the opinions of the students regarding the introduction of these elements and their impact on increasing the quality of training

From the detailed data shown in the graphs in Fig. 16 to Fig. 24, and from the summary results of the study visible in Fig. 25, we can draw the following conclusions. Overall, students highly value the use of game elements in the learning process. They see particular value in using game-based learning (GBL) or simulators to support learning in education in general.

Particular meaning is found in the use of GBL or simulators in areas and disciplines in which:

‘The materials are dry, technical and boring’, ‘The matter is really difficult to study’, ‘There are routine but complex practical activities to perform’ and ‘Training is given to work in a dangerous or aggressive environment’. Also, well rated use of GBL or simulators in area where ‘Complex analyses are needed’.

Similarly, is the rating for the questions ‘The cognitive development of the learners’ and ‘Desire to contribute to the common goal’ are valued most highly.

The impact of ‘Degree of coping’, ‘Time to deal’ and ‘Class attendance’ was rated as insignificant.

This is best seen in the low score obtained for “Do you agree that the use of gamification elements (PBL triad – stars, points, rankings) has an effect on: ‘Feeling of the support of the team in the training’, ‘Increasing the concentration of students’ and ‘Desire to contribute to the common goal’.

The questions related with – ‘The cognitive development of the learners’: ‘Consolidation of already learned things through repetition’ and ‘Increasing class attendance’ were rated as positive but with low impact.

Strongly impressive was the negative assessment of students on the extent to which their school supports PBL and GBL development policies. The assessment is that the university management ‘Has a positive attitude towards introducing gamification as a factor to increases the quality of teaching’ and ‘Motivates teachers for development’, but ‘Does not have a formal development policy’, ‘Does not incentivize development’, ‘Does not allocate funds for development’ and ‘Does not post formal information on its website’.

The findings give us an insight into the overall picture of the impact of PBL and GBL training in a higher education institution. The detailed analysis and evaluation of the results, taking into account the forms of learning, the method of teaching, the different disciplines and which course the students are in, can provide us with important information on how useful gamification is in improving the quality of learning in higher education institutions. An evaluation of the responses can give us guidance in which areas and which elements of gamification would be most useful to use.

Acknowledgement

This work presents some of the results of project “Redesigning Introductory Computer Programming Using Innovative Online Modules (RECOM)”, 2020-1-TR01-KA226-HE-098258, Erasmus+ programme.

REFERENCES

AL-AZAWI R., F. Al-Faliti, M. Al-Blushi, 2016. Educational Gamification Vs. Game Based Learning, International Journal of Innovation, Management and Technology, vol. 7, no. 4, pp. 132 – 136, doi: 10.18178/ijimt.2016.7.4.659.

GUERRERO, M.; URBANO, D. 2021. The entrepreneurial university in the digital era: looking into teaching challenges and new higher education trends. A Research Agenda for the Entrepreneurial University”, pр. 143 – 167, ISBN: 9781788975032.

GUÀRDIA, L.; CLOUGHER, D.; ANDERSON, T., & MAINA, M., 2021. IDEAS for Transforming Higher Education: An Overview of Ongoing Trends and Challenges. The International Review of Research in Open and Distributed Learning, vol. 22, no. 2, pp. 166 – 184. https://doi.org/10.19173/irrodl. v22i2.5206.

LIN, W. C., J. Y. Ho, C. H. Lai, B. S. Jong, 2014. “Mobile game-based learning to inspire students learning motivation. ISEEE, vol. 2, pp. 810 – 813, DOI: 10.1109/InfoSEEE.2014.6947779.

MIKHEEV, A.; SERKINA, Y. & VASYAEV, A., 2021. Current trends in the digital transformation of higher education institutions in Russia. Educ. Inf. Technol, no. 26, pp. 4537 – 4551. https://doi.org/10.1007/s10639-021-10467-6.