ER4STEM Publications

TITLE LINK REFERENCE YEAR AUTHORS ABSTRACT KEYWORDS CONFERENCE
Overview and interim evaluation of the project ROBIN: Robotics for integration View/Download Vittori L., Schuster L., Weinert N., Koppensteiner G. & Lepuschitz W. (2016). Overview and interim evaluation of the project ROBIN: Robotics for integration. 2016 IEEE 8th International Conference on Engineering Education (ICEED). doi:10.1109/iceed.2016.7856096 2016 Lisa Vittori, Lisamarie Schuster, Nicole Weinert, Gottfried Koppensteiner and Wilfried Lepuschitz Statistics in Austria show a significantly higher unemployment rate of people with migration background compared to those without such a background. This situation is further aggravated by the increased application of modern technologies that replace jobs with automated systems. People with migration background in Austria tend to have a lower qualification and corresponding jobs that are in higher danger of being automated. This paper presents the concept of the project ROBIN that employs educational robotics for inspiring young people towards engineering and technology. In this context, a focus is specifically put on supporting young people with a migration background. Also the results of an interim project evaluation are presented and a few lessons learned are discussed. International Conference on Engineering Education (ICEED) 2016
Game Kits: Metadesign considerations on game modding for learning View/Download Yiannoutsou N. & Kynigos C. (2016). Game Kits: Metadesign considerations on game modding for learning. In Proceedings of the The 15th International Conference on Interaction Design and Children (pp. 583-588). ACM. 2016 Yiannoutsou Nikoleta, Kynigos Chronis Today’s technologies that blur the distinction between users and designers have empowered end-users to engage with powerful learning activities like game modding. In this paper we discuss the characteristics of modding tools as expressive media that support teachers and students alike, to integrate in games their knowledge and ideas without being restricted by tools bound to the way the game is implemented; i.e. mainly through programming and STEM knowledge. We present work in progress on "Sus-X", a GameKit that generates SimCity like games and provides pedagogically designed modding tools. We explore the expressive power of "Sus-X", through two studies with students that engaged in modification of two different games created with "Sus-X": one game involved urban sustainability and the other involved nutrition. Game modding; End-Users, Learning; Half-baked microworlds Interaction Design and Children (IDC) 2016
Enhancing students’ interest in STEM through educational robotics (in Greek) View/Download Grizioti M., Xenos M. & Kynigos C. (2016) Enhancing students’ interest in STEM through educational robotics (in Greek), Presented In Hellenic Conference on Innovating STEM Education 2016 (HiSTEM 2016), 16-18/12/2016, University of Athens, Greece 2016 Grizioti Marianthi, Xenos Marios, Kynigos Chronis During the last few years there has been a shift of the educational community towards STEM while several studies have indicated the benefits of STEM in the development of important learning skills. Educational robotics, dute to their interdisciplinarity, can be a strong tool for the design of STEM activities. The presented research focuses on educational robotics as a means for motivating students to engage with STEM. As part of this research, there was a series of workshops organized in different greek schools. The aim was to study how and to what level educational robotics activities can streghtain students' interest towards STEM subjects and careers and how they can contribute to meaning generation about STEM concepts. Educational Robotics, STEM, Constructionism Hellenic Conference on Innovating STEM Education (HiSTEM) 2016
Activity Plan Template: a mediating tool for supporting learning design with robotics View/Download Yiannoutsou N., Nikitopoulou S., Kynigos C., Gueorguiev I. & Angel-Fernandez J. (2016). Activity Plan Template: A Mediating Tool for Supporting Learning Design with Robotics.n M. Merdan, W. Lepuschitz, G. Koppensteiner & R. Balogh (Eds.) Robotics in Education. RiE 2016. Robotics in Education Advances in Intelligent Systems and Computing, 3-13. doi:10.1007/978-3-319-42975-5_1 2016 Nikoleta Yiannoutsou, Sofia Nikitopoulou, Chronis Kynigos, IvayloGueorguiev, Julian Angel Fernandez (UoA, ESI -CEE, TuWien) Although Robotics have been designed for education for several decades now, only recently they started being broadly used in education, formal and non formal. In this context many different technologies have emerged accompanied by relevant learning material and resources. Our observation is that the vast number of learning activities is driven by multiple “personal pedagogies” and thus it results in the fragmentation of the domain. To address this problem we discuss in the paper the construct of “activity plan template”, a generic design tool that will facilitate different stakeholders (teachers, instructors, researchers) to design learning activities for different robotic toolkits. In the paper we discuss the characteristics of the activity plan template and the research process of generating such a template. Since we report work in progress, we present here the first version of the activity plan template, the construction of which is based on a set of best practices identified. Activity Plan Template; learning design; educational robotics International Conference on Robotics and Education RiE 2016
Offering Multiple Entry-Points into STEM for Young People View/Download Lammer L., Lepuschitz W., Kynigos C., Giuliano A., Girvan C. (2017) ER4STEM Educational Robotics for Science, Technology, Engineering and Mathematics. In: Merdan M., Lepuschitz W., Koppensteiner G., Balogh R. (eds) Robotics in Education. Advances in Intelligent Systems and Computing, vol 457. Springer, Cham 2017 Lara Lammer, Wilfried Lepuschitz, Chronis Kynigos, Angele Giuliano and Carina Girvan Robotics is a popular vehicle to introduce young people to science, technology, engineering and mathematics (STEM) with various approaches worldwide that use robotics to teach or entertain or both, accompanied by various tools and repositories. However, the stakeholders involved have different goals and methods, thus difficulties in finding common ground; e.g. the focus in most cases is on increasing interest in STEM, but research methods are unspecified or vague; or despite the vastness of offerings, teachers are reluctant to incorporate activities in the classroom. In this paper, we introduce the Educational Robotics for STEM (ER4STEM) project that will realise a creative and critical use of educational robotics to maintain children’s curiosity in the world: an open and conceptual framework will bring three main stakeholders of educational robotics – teachers, educational researchers and organizations offering educational robotics – together in a user- and activity centered repository. Educational robotics; framework; teachers; repository International Conference on Robotics and Education RiE 2016
Offering Multiple Entry-Points into STEM for Young People View/Download Lepuschitz W., Koppensteiner G., Merdan M. (2016). Offering Multiple Entry-Points into STEM for Young People. In M. Merdan, W. Lepuschitz, G. Koppensteiner& R. Balogh (Eds.) Robotics in Education. RiE 2016. Robotics in Education Advances in Intelligent Systems and Computing, vol. 457. Switzerland: Springer International Publishing, 95-101. doi:10.1007/978-3-319-42975-5_4 2017 Wilfried Lepuschitz, Gottfried Koppensteiner and Munir Merdan Enrollment in the STEM fields (science, technology, engineering and math) is not keeping pace with the need. Recent reports indicate a decrease in the number of graduates from STEM fields and a shortage on the job market. Considering these issues, particular attention has been paid developing innovative methods and tools for improved teaching of STEM themes. This work presents an approach involving multiple entry points for young people to engage in the STEM fields. This approach is manifested in the non-profit association Practical Robotics Institute Austria (PRIA) with its activities designed to fill STEM gaps in the Austrian education system and to bring innovative engagement that cannot be found in the classrooms. Thus, STEM literacy is fostered as well as the development of systems thinking, problem solving, and teamwork skills. STEM, after-school program, camps, workshops, non-profit association, research, education International Conference on Robotics and Education RiE 2016
Hedgehog Light – A Versatile, White Box Educational Robotics Controller View/Download Koza C., Lepuschitz W., Wolff M., Frank D. & Koppensteiner G. (2017). Hedgehog Light – A Versatile, White Box Educational Robotics Controller. Educational Robotics in the Makers Era Advances in Intelligent Systems and Computing, 229-232. doi:10.1007/978-3-319-55553-9_19 2017 Clemens Koza, Wilfried Lepuschitz, Martin Wolff, Daniel Frank and Gottfried Koppensteiner Robotics curricula that include programming generally require a robotics controller as core platform. The controller and its capabilities shape the offered curriculum in terms of programming means, and therefore age appropriateness. A closed, black-box controller also limits the uses outside the context of such a curriculum. The Hedgehog Light controller aims to imply as few limitations as possible by using open source methodology, the open and well-documented Raspberry Pi, a multi-platform protocol and programming interface, and that can mostly be built in fabrication laboratories (fab labs). controller, Raspberry Pi, white box, open source, programming EduRobotics 2016 Educational Robotics in the Makers Era
Constructing the shortest path on a cylindrical surface View/Download Kynigos C. & Zantzos I. (2017). Constructing the shortest path on a cylindrical surface, CERME 2010 Conference, 1-5 February 2017, Dublin 2017 Chronis Kynigos, Ioannis Zantzos Digital media warrant a reappraisal of established conceptual fields and a search for new ones densely providing access to powerful mathematical ideas. This study reports secondary students' meaning making around the notion of intrinsically defined curvature in space by means of a tool integrating programming, dynamic manipulation of variable values and a simulation of 3D space. The study involved 15 ninth grade school students’ attempt to design the shortest path between two points on a cylindrical surface are presented in this paper. Camera perusal and zoom allow for a change of viewpoints of the constructed figure. The findings yield meanings around concepts notoriously difficult even in undergraduate mathematics, such as differential stereometry, limits and curvature as systematic trihedron state change. Curvature, helix, stereometry, meaning-making, programmable media CERME10
Learning Programming with Educational Robotics: Towards an Integrated Approach View/Download Xenos M., Yiannoutsou N., Grizioti M., Kynigos C. & Nikitopoulou S. (2017). Learning Programming with Educational Robotics: Towards an Integrated Approach. Educational Robotics in the Makers Era, 560, 215. Springer, Cham. doi: 10.1007/978-3-319-55553-9 2017 Xenos Marios, Yiannoutsou Nikoleta, Grizioti Marianthi, Kynigos Chronis, Nikitopoulou Sofia Despite the fact that it has been a few years since robotics entered the school and offered new learning opportunities, educational robotics usually is offered in the context of extra-curricular activity (e.g. a “club”) which addresses a limited number of students and participation is based on student personal interest. In this paper we explore the potential of ER when it is integrated in the typical school curriculum. In the study we report here, we integrated ER in the computer science curriculum and all students of a 9th grade class engaged with robotics activities. The rationale underlying the study is that robotics can be used as a medium to motivate students in engaging with programming and support them to negotiate real life problems. Analysis of the data collected, indicate that ER when integrated with the computer science curriculum, can create a rich learning environment where programming is contextualized and students are highly motivated to engage and negotiate important STEM concepts. Constructionism · Programming · Educational robotics EduRobotics 2016 Educational Robotics in the Makers Era
RobIn: a half-baked robot for electronics in a STEM context View/Download Kynigos C., Grizioti M. & Nikitopoulou S. (2017, June). RobIn: A Half-baked Robot for Electronics in a STEM Context. In Proceedings of the 2017 Conference on Interaction Design and Children (IDC) (pp. 521-526). ACM. 2017 Kynigos Chronis, Grizioti Marianthi, Nikitopoulou Sofia Robotic kits in designs for STEM education are becoming popular during the last few years. However, many of them focus mainly on programming leaving aside the electronics and engineering parts of robotics. In this paper we present RobIn (Robotic Insect), a robotic design that supports the equivalent coexistence of programming, construction and argumentation processes in corresponding educational robotic activities. RobIn challenges students to change it, improve it and expand its prototype robotic skeleton by using every day and affordable materials. We also present a study where RobIn functioned as a boundary object among students from different specializations of a Vocational Technical School. Educational Robotics; STEM; Boundary Crossing; Interaction Design and Children (IDC) 2017
Determining the Effect of Programming Language in Educational Robotic Activities View/Download Fernandez J.M. & Vincze M. (2017). Determining the effect of programming language in educational robotic activities. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 658-663. 2017 Julian M. Angel-Fernandez and Markus Vincze Robotics has been suggested as a field of high potential in education and with high expectancy to impact teaching from kindergarten to university. This paper presents a study conducted with the purpose to have a better understanding of the impact of programming languages among participants of a workshop. An activity, which encompasses ten exercises, was designed and we used three different programming languages (i.e. visual, blocky and text) to program the Thymio robot. A total of six workshops were held using this activity two for each programming language. Qualitative and quantitative data were collected in each workshop. The results suggest that despite the programming language used, participants enjoyed working with robots. Moreover participants with previous experience on programming prefer more advance programming languages. 6th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).
The ER4STEM Repository for Educational Robotics View/Download Duca A., Giuliano A., Nikitopoulou S., Yiannoutsou N. & Kynigos C. (2018) The ER4STEM Repository for Educational Robotics. Constructionism 2018. Vilnius, Lithuania, August 21-25, 2018. 2018 Annalise Duca, Angele Giuliano, Sofia Nikitopoulou, Nikoleta Yiannoutsou, Chronis Kynigos Using robotics in education is an engaging method for student motivation towards STEM subjects and more. Teachers, educators and researchers who are newly experimenting with the use of robots in the classroom are all asking a very similar question. “Where can I find inspiration to introduce constructionism in my teaching?”, “What can I do to teach my subject using robotics?” The answer to this is “The ER4STEM Repository” which will be full of educational resources, activity plans and suggestions for educators. “The ER4STEM Repository” has been underpinned by the basic pedagogical theory underlying its’ design in constructionism. This happens through an Activity Plan Template. This template is aligned with the Repository and provides a generic design instrument that identifies critical elements of teaching and learning with robotics based on theory and practice and is expected to contribute to the description of effective learning and teaching with robotics. repository; educational robotics; STEM; robots; sharing ideas; collaboration; Educational Robotics for STEM; Educational Activities; Constructionism; OER Constructionism 2018
Educational Robotics for Communication, Collaboration and Digital Fluency View/Download Gueorguiev I., Todorova C., Varbanov P., Sharkov P., Sharkov G., Girvan C., Yiannoutsou N. & Grizioti M. (2017) Educational Robotics for Communication, Collaboration and Digital Fluency. In: Lepuschitz W., Merdan M., Koppensteiner G., Balogh R., Obdržálek D. (eds) Robotics in Education. RiE 2017. Advances in Intelligent Systems and Computing, vol 630. Springer, Cham 2018 Ivaylo Gueorguiev, Christina Todorova, Pavel Varbanov, Petar Sharkov, George Sharkov, Carina Girvan, Nikoleta Yiannoutsou, Marianthi Grizioti This paper is the experience-based summary of the work with the design, implementation and results from an “Educational Robotics and Creativity Workshop” under the EU funded Horizon 2020 project „ER4STEM – Educational Robotics for Science, Technology, Engineering and Mathematics“. This paper gives an overview of the empirical data obtained from the post-workshop questionnaires, completed by the participants from 13 educational robotics workshops, per-formed in 7 schools (public and private) in Bulgaria with 312 students (142 girls and 170 boys) in the time period from February 16, 2016 until May 31, 2016. The students were between 7 and 14 years old with the majority of them aged be-tween 9 and 10 years old. educational robotics, creativity, collaboration, communication, digital fluency, Arduino, visual programming, robotics in education International Conference on Robotics and Education RiE 2017
Visualizing mathematics with the MathBot: a constructionist activity to explore mathematical concepts through robotics View/Download Todorova C., Girvan C., Yiannoutsou N., Grizioti M., Gueorguiev I., Varbanov P. & Sharkov G. (2018) Visualizing mathematics with the MathBot: a constructionist activity to explore mathematical concepts through robotics. Constructionism 2018. Vilnius, Lithuania, August 21-25, 2018. 2018 Christina Todorova, Carina Girvan, Nikoleta Yiannoutsou, Marianthi Grizioti, Ivaylo Gueorguiev, Pavel Varbanov and George Sharkov In this practice paper, we aim to share our experience with the design and implementation of constructionist educational robotics activities tailored to primary school students (4th grade, age 911 years) implemented in a series of robotics workshops, which took place within a real school setting in Sofia, Bulgaria. Through this contribution, we will further present an activity plan, which involves student engagement with mathematical concepts (angle measuring and properties of the circle) in order to program the behaviour of a robot. Our paper reports insights on the implementation of the activity plan focusing students’ evaluation of their experience during the workshop. These insights are drawn from quantitative data from 131 participants (63 boys and 68 girls), capturing the overall student attitude. The activity plan behind this set of educational robotics workshops was designed, adapted and piloted in alignment to the guidelines of the Bulgarian national curriculum for mathematics for the 4th grade. However, it could function as a practical example, which could be adapted, enriched and modified to benefit other age groups, nationalities and desired learning outcomes. educational robotics; mathematics; programming; Scratch; primary education Constructionism 2018
Towards a Generic Curriculum for Educational Robotics in STEM: From Scientific Concepts to Technologies and Powerful Ideas View/Download Gueorguiev I., Todorova C., Yiannoutsou N., Gkreka C., Girvan C., Varbanov P., Sharkov G., Duca A., Vittori L. & Angel Fernandez J. (2018) Towards a Generic Curriculum for Educational Robotics in STEM: From Scientific Concepts to Technologies and Powerful Ideas. Constructionism 2018. Vilnius, Lithuania, August 21-25, 2018. 2018 Ivaylo Gueorguiev, Christina Todorova, Nikoleta Yiannoutsou, Christina Gkreka, Carina Girvan, Pavel Varbanov, George Sharkov, Annalise Duca, Lisa Vittori and Julian Angel Fernandez This paper and its corresponding poster present a “work in progress” concept for the visualization of 19 activity plans, into a generic curriculum map for teaching STEM concepts through constructionist robotics activities. There are six educational paths that represent potential use cases and these have been validated through 148 educational robotics workshops implemented with children between the ages of 7 and 18 in six European countries. educational robotics, curricula, robotics, workshop, science, technology, engineering, mathematics, learning, STEM Constructionism 2018
Architectural Overview and Hedgehog in Use View/Download Koza C., Wolff M., Frank D., Lepuschitz W. & Koppensteiner G. (2017). Architectural Overview and Hedgehog in Use. In W. Lepuschitz, M. Merdan, G. Koppensteiner, R. Balogh & D. Obdržálek (Eds.) Robotics in Education. RiE 2017. Robotics in Education Advances in Intelligent Systems and Computing, vol. 630. Switzerland: Springer International Publishing, 238-249. doi:10.1007/978-3-319-62875-2_21 2018 Clemens Koza, Martin Wolff, Daniel Frank, Wilfried Lepuschitz, and Gottfried Koppensteiner Robotics is a versatile tool for teaching STEM topics, as it supports various disciplines, skill sets and target audiences. However, controllers used in Educational Robotics are often limited in their use cases. In this regard, Hedgehog tries to be flexible by design. This paper introduces Hedgehog’s architecture, currently implemented and future use cases, and experiences from our first Hedgehog workshops. Raspberry Pi, Controller, Visual programming, Blockly, Python International Conference on Robotics and Education RiE 2017
Hedgehog: A Versatile Controller for Educational Robotics View/Download Klein M., Koza C., Lepuschitz W. & Koppensteiner G. (2018) Hedgehog: A Versatile Controller for Educational Robotics. Constructionism 2018. Vilnius, Lithuania, August 21-25, 2018. 2018 Markus Klein, Clemens Koza, Wilfried Lepuschitz and Gottfried Koppensteiner We describe Hedgehog, an educational robotics controller designed to foster interest in STEM subjects. Hedgehog allows building robots out of common actuators and sensors, and can be combined with Lego for a beginner-friendly experience. Through building robots, students can learn and apply engineering, electronics, planning and teamwork skills. The controller facilitates learning programming at different age levels through both textual and visual programming support. For advanced students, Hedgehog's open source ecosystem allows delving into subjects such as microcontroller programming or cooperative robots as well. Hedgehog has been used in numerous workshops and also in robotics competitions with great success. Robotics, Programming, Visual Programming, STEM Constructionism 2018
Studying real-world societal problems in a STEM context through robotics View/Download Kynigos C., Grizioti M. & Gkreka C. (2018). Studying real-world societal problems in a STEM context through robotics. Conference on Interaction Design and Children (IDC) 2018. arXiv preprint arXiv:1806.03245. 2018 Kynigos Chronis, Grizioti Marianthi, Gkreka Christina In this paper, we will discuss the design and implementation of three educational robotics workshops that focus on the use of robotics for addressing real-world societal problems. Through the workshops, we aimed to investigate how the integration of real-world problems to robotics can provide a multidisciplinary approach for ‘child-robot interaction’ that allows students to engage with different STE(A)M concepts. In addition, we examined if this integration increased or strengthened students’ interest in STEM/robotics education and careers. The workshops were designed and implemented in the context of the European project Educational Robotics for STEM (ER4STEM) with students from different age groups using Lego Mindstorms and Arduino kits. Educational Robotics, STEM education, Lego Mindstorms, Arduino Interaction Design and Children (IDC) 2018
Discussing digital artefacts under design as a process aiming towards the centre of TPaCK View/Download Diamantidis D. & Kynigos C. (2018) Discussing digital artefacts under design as a process aiming towards the centre of TPaCK. InRe(s)sources 2018. 2018 Diamantidis Dimitris, Kynigos Chronis The study addresses the illumination of TPaCK knowledge of postgraduate mathematics education students and their progression towards the TPaCK centre by means of a threaded forum discussion concerning their on-going designs of digital artefacts for pupil meaning-making. We perceived those digital artefacts as living documents under change and studied both students' constructions and their written exchanges to identify how their initial ideas were placed in the TPaCK model and how these could progress. We studied the exchanges between them and their instructors as a boundary crossing process considering the artefacts as boundary objects. Our preliminary results show that the process was effective in helping us with our understanding of both how the students started and how they were challenged to progress towards TPaCK intersections and centre. Professional development, instrumental approach, boundary crossing, turtle geometry. Re(s)sources 2018
Storytelling as educational strategy for STE(A)M activities that aim to motivate students: A case study Panagiotou E. & Diamantidis D. (2018) Storytelling educational strategy for STE(A)M activities that aim to motivate students: A case study (in Greek). In proceedings of 11th International Conference “ICT in Education”, 19-21 October 2018, Thessaloniki, Greece 2018 Panagiotou Evriklia, Diamantidis Dimitris In recent years there has been a shift in the educational community towards STEAM fields and their benefits for students in developing important skills for the 21st century. Many studies have highlighted the contribution of educational robotics in the design of STEAM activities. The study we present in this paper focuses on the use of storytelling as an educational method in a STEAM context in order not only to motivate students to be informed about societal issues, such as the problems of the disabled, but also to encourage them to create their own robots, utilising their knowledge of STEM sciences and their creativity (Arts), while they are discusing about their choices. story telling, multidisciplinarity, creativity, educational robotics 11th International Conference “ICT in Education”, 19-21 October 2018, Thessaloniki, Greece
Engineers Designing and Implementing Activities for School Students: What have we learned? View/Download Angel-Fernandez J. & Vincze M. (2018) Engineers Designing and Implementing Activities for School Students: What have we learned? II IEEE World Engineering Education Conference (EDUNINE2018). Buenos Aires, Argentina, March 11-14, 2018. 2018 Julian M. Angel-Fernandez and Markus Vincze Robotics kits are spreading out in the market, becoming more convenient, but more is required to determine their impact on educational activities, such us participants' perceptions and factors that could affect the activity. This paper presents an educational activity, which was designed with two main goals in mind. i) Determine if there is any impact of the programming language used in the activity and ii) create a baseline to change various factors in upcoming workshops. It was started with a series of six workshops held in the campus of the university and taken by 124 participants. Results indicate that in general working with robots is interesting and fun, there is no difference between males and females besides the question/factor of using technology to learn, and working in teams. Nevertheless, they also suggest that participants' previous experience on programming must align with the difficulty level of the workshop. II IEEE World Engineering Education Conference (EDUNINE2018)
Introducing Storytelling to Educational Robotic Activities View/Download Angel-Fernandez J. & Vincze M. (2018) Introducing Storytelling to Educational Robotic Activities. IEEE Global Engineering Education Conference (IEEE EDUCON 2018). Santa Cruz de Tenerife, Canary Islands, Spain, on April 18-20, 2018. Best paper in Area 3: Attracting, Engaging and Retaining Human Talent to Engineering 2018 Julian M. Angel-Fernandez and Markus Vincze Creativity is a skill that has been recognized as one of the 21st century skills. Likewise robotics has been recognized as technology with several features to enthrall children and be used to teach a variety of topics (e.g. Mathematics and programming). This paper presents a study done to verify the impact of introducing a storytelling session in an activity with children from 6 to 18 years old in Austria. A total of 196 participants participated in the workshops, held in the campus of the university. Quantitative and qualitative data were collected and analyzed. The results showed that the most difficult task for old participants was the collaboration between groups created. Participants also did not mention the use of creativity during the design and implementation of the story. Instead participants referred to the work done with robots and technology. Moreover, participants think that working with robots is interesting and fun. Storytelling, Creativity, Robotics, Educational Robotics, Informal Education, Thymio II IEEE Global Engineering Education Conference (IEEE EDUCON 2018)
Towards a Formal Definition of Educational Robotics View/Download Angel-Fernandez J. & Vincze M. (2018) Towards a Formal Definition of Educational Robotics. Sixth Austrian Robotics Workshop (ARW 2018). Innsbruck, Austria. May 15 and 16, 2018. 2018 Julian M. Angel-Fernandez and Markus Vincze There is an increasing number of articles, web pages, robotic kits and other materials that are using the term Educational Robotics (ER) to refer to the use of robots in education, however the current definition of ER is still vague and open to misinterpretation. Therefore, anyone can claim that their work falls in the category of ER just because robots are involved. Despite all benefits of robotics, its incorrect use may be counterproductive. Therefore, the incremental use of the term ER is meaningless if it is not used correctly. Consequently, a concrete and precise definition of ER is required to support the development of it. This paper presents a first attempt to develop a concrete definition of ER, which describes all fields of study that constitutes it and how they are related between them. The definition is the result of the experience acquire during the participation of the European project Educational Robotics for STEM (ER4STEM). Sixth Austrian Robotics Workshop (ARW 2018)
Towards a Framework for Educational Robotics View/Download Angel-Fernandez J., Yiannoutsou N., Kynigos C., Girvan C. & Vincze M. (2018) Towards a Framework for Educational Robotics. Constructionism 2018. Vilnius, Lithuania, August 21-25, 2018. 2018 Angel-Fernandez, Julian M. & Yiannoutsou, Nikoleta & Kynigos, Chronis & Girvan, Carina & Vincze, Markus Educational robotics has been considered as a field with a good potential to teach difficult concepts (e.g. friction) in appealing way. As a consequence, the interest in educational robotics has grown in the last decade, which is reflected in increasing number of robotic platforms, kits, and programming interfaces now available. Nevertheless, researches still fail on describe activities that could be used by teachers and other people with no technological fluency, who are scared by the overwhelm amount of information that made them avoid the use of robotics to teach. Moreover, most of the activities developed until now do not consider pedagogical methodologies to inform the design and implementation of them. As a direct consequence of the misinformation about the correct use of pedagogical methodologies and robotics' multidisciplinary, the number of people who master the use of robotics in education is still scant. This paper presents ongoing work on the development of a framework in the European project Educational Robotics for Science, Technology, Engineer, and Mathematics (ER4STEM). The framework aims to make evident the connection between 21st century skills, robotics and pedagogical methodologies to support the creation of pedagogical activities, which is defined in ER4STEM as an activity that has clear learning outcomes and evidence of learning, use of one or more pedagogic methodologies during the activity, and detail description of the activity. This is achieve through the critical use of tools and examples of activities developed ER4STEM. Educational Robotics, Framework for Educational Robotics, Pedagogical Activities, Educational Activities, Educational Robotics for STEM, and Constructionism Constructionism 2018
Deliverable 1.1: Best Practices and Requirments View/Download 2016 Angel-Fernandez Julian M., Lammer Lara, Kynigos Chronis, Gueorguiev Ivaylo, Varbonov Pavel, Lepuschitz Wilfried, Duca Annalise, Pullicino Joanna, Grizioti Marianthi, Yiannoutsou Nikoleta, Nikitopoulou Sofia, Girvan Carina, Vrba Pavel This deliverable presents the process that was executed to determine best practices and requirements in educational robotics. The extensive research shows that existing approaches have gaps in the description and implementation of activities to be comparable, replicable and engaging to all children. Additionally, approaches are mostly outside of schools, defragmented and unconnected; knowledge is scarcely well transferred. The educational robotics community needs a common solution that addresses these gaps. The ER4STEM framework will be based on the identified needs and become the catalyst to improve young people’s learning experience through the use of robotics in formal and informal spaces. It will bring the educational robotics community forward by creating processes, tools and artefacts that allow the use of robots in learning spaces so that a creative and critical use of educational robotics is realized to maintain children’s curiosity in the world. A first step for a common solution is the understanding of the different stakeholders and their needs. Thus, this document analyses the stakeholders that affect and are affected by educational robotics activities. Furthermore, the need for a common process to identify best practices is addressed. The newly developed activity plan for educational robotics (a tool of the framework) is based on profound research and experience on pedagogical methodologies, and has been used by all consortium partners to describe their educational robotics activities. Following the identification of the stakeholders’ needs and the development and testing of the activity plan, newly developed parameters and criteria are presented to determine good practices in educational robotics. Based on the identified needs, first ideas on the ER4STEM framework are given as an outlook.
Deliverable 1.2: Framework View/Download 2016 Angel-Fernandez Julian M., Gueorguiev Ivaylo, Varbonov Pavel, Duca Annalise, Lepuschitz Wilfried, Kynigos Chronis, Stola Miroslav, Girvan Carina, Lammer Lara This deliverable describes the specific objective of the framework, which is to help people create robotic activities. To achieve this, the framework introduces a macro-process that later is adapted to each one of the stakeholders’ needs (i.e. teachers, organizers of educational activities, and educational researchers) identified in the D1.1. In this first version of the framework just two processes are developed (i) designing a pedagogical activity and (ii) implementing a conference.
Deliverable 1.3: Towards an extended definition of ER4STEM Framework View/Download 2017 Angel-Fernandez Julian M., Kynigos Chronis, Todorova Christina, Pullicino Johana, Grizioti Marianthi, Girvan Carina, Yiannoutsou Nikoleta, Lepuschitz Wilfried The purpose of this deliverable is to present the work done during the second year towards the creation of a framework for pedagogical activities in educational robotics. A general idea of the framework was presented in D1.2. However this definition required detailed description of its characteristics and literature analysis on other frameworks. A revision of available frameworks in educational robotics showed that there is no other framework on educational robotics that provides a comprehensive guideline to the main stakeholders in educational robotics. As a consequence it was then review frameworks in technology to identify features that are relevant on those frameworks. Considering these factors and the stakeholders, identified in D1.1, a requirement analysis of these stakeholders was done in order to correctly pin-point areas in which the framework should focus. Therefore the framework is envision as a guidelines for educationally comprehensive use of robotics, making evident the connection that was not visible in most of the works reported on educational robotics: pedagogy, 21st century skills, specific knowledge in robotics and general knowledge. Base on this analysis, industry requirements, reported in D1.1, and suggestions provided in D6.3, it was done a literature review on critical thinking, creativity and collaboration to determine current tendencies on the evaluation and improvement of these skills. Additional, it is presented the conceptualization of an ontology 1 on educational robotics field, which in the future is intended to be implemented in a knowledge base. This base will be helpful on the creation of semantic search systems that could be also used in the repository once it is implemented.
Deliverable 1.4 - ER4STEM Framework: Final Version View/Download 2018 Angel-Fernandez Julian M., Girvan Carina, Yiannoutsou Nikoleta, Jäggle Georg, Nikitopoulou Sofia, Kynigos Chronis, Duca Annalise, Gueorguiev Ivaylo, Todorova Christina, Grizioti Marianthi, Greka Christina, Saidlova Lenka, Lepushitz Wilfried, Hopkins-Waver Sarah This deliverable reports on the final version of the ER4STEM framework, which is the result of three-years of improvement and iteration based on the continuous contribution of WP 2, WP3, WP4, WP 5 and WP6. Consequently, this document could be considered as an organizer and a window to the work done throughout the ER4STEM project’s implementation period. The framework aims to make explicit the connection between pedagogical strategies, robotics and 21st century skills providing processes and tools to guide stakeholders along the critical use of robotics in education. Moreover, it situates the educational use of robotics in constructionist learning activities for children both inside and outside of school in which, they:
- Collaborate within and between teams;
- Creatively engage with challenges in STEM domains;
- Engage in critical thinking through reflection;
- Have opportunities to learn how to recover from failure;
- Gain a sense of achievement.
These activities are taking place in shared spaces with mediating artefacts (aka robots), teachers and children are able to challenge existing classroom norms and attitudes towards learning (in STEM and in general), in order to develop and maintain young people’s interest in STEM subjects and careers. To achieve this, the ER4STEM framework provides four components that are the result of the work done throughout the project:
- ER4STEM Glossary, (ER4STEM Glossary);
- Tools: ER4STEM web-repository (WP 5) (ER4STEM Repository), ER4STEM activity plan template and activity blocks (WP 4) (ER4STEM Activity Plan Template), and ER4STEM generic curriculum (WP 2) (ER4STEM Generic Curriculum);
- ER4STEM pedagogical principles (ER4STEM Pedagogical Principles);
- Processes for the delivery of Educational Robotics Educational Robotics Workshops (WP 2 – D2.3) and Conferences (WP 3 – D 3.3) for young learners (ER4STEM Processes).
Deliverable 2.1: Robotics Workshops 1st Year View/Download 2016 Girvan Carina, Todorova Christina, Kynigos Chronis, Sharkov George, Gueorguiev Ivaylo, Pullicino Joanna, Angel-Fernandez Julian M., Grizioti Marianthi, Stola Miroslav, Yiannoutsou Nikoleta, Varbanov Pavel, Nikitopoulou Sofia, Lepuschitz Wilfried The purpose of this deliverable is to inform on the process of WP2 “Educational Robotics workshops” describing the curricula created and used to deliver Educational Robotics Workshops (henceforth: ERW) and presenting the quantitative data obtained during the ERWs delivery. The report creates a baseline, which will be used for further modification and improvement of the ERW curricula throughout the ER4STEM project lifecycle. Quantitative data in this deliverable is based on the information reported in the workshop information forms, provided by each partner.
Deliverable 2.2: Robotics Workshops 2nd Year View/Download 2018 Todorova Christina, Sharkov George, Gueorguiev Ivaylo, Angel-Fernandez Julian M., Varbanov Pavel The purpose of this deliverable is to inform on the process of WP2 “Educational Robotics Workshops” describing the generic curriculum applied to deliver Educational Robotics Workshops (henceforth: ERW), as well as to present the progress achieved so far along with the quantitative data obtained during the ERWs delivery. The report provides an update on the process, which will be used for further modification and improvement of the ERWs’ curriculum throughout the ER4STEM project lifecycle. Quantitative data in this deliverable is based on the information obtained through the collection of data from the workshop participants, reported through the workshop information forms, provided by each partner.
Deliverable 2.3: Robotics Workshops 3rd Year View/Download 2018 Carina Girvan, Todorova Christina, Sharkov George, Gueorguiev Ivaylo, Angel-Fernandez Julian M., Varbanov Pavel The purpose of this deliverable is to provide information about the progress of WP2 “Educational Robotics Workshops”. In this deliverable, we describe the improvements made on the design of the generic curriculum on educational robotics, where we connect the elements and processes, which the ER4STEM team applied to deliver Educational Robotics Workshops (henceforth: ERWs), as well as to present the progress of the delivery of the ERWs. The report provides an update on the process, which will be used for ER4STEM Curriculum description in D2.4. Quantitative data in this deliverable is based on the information reported through the workshop information forms, provided by each partner.
Deliverable 2.4: ER4STEM Curriculum View/Download 2018 Todorova Christina, Sharkov George, Gueorguiev Ivaylo, Angel-Fernandez Julian M., Grizioti Marianthi, Varbanov Pavel The purpose of this deliverable is to provide a final report on the complete ER4STEM Generic Curriculum as developed by the ER4STEM project partners during the ER4STEM project implementation period. In this deliverable, we provide an overview of the ER4STEM Generic Curriculum on educational robotics, where we connect the elements and the processes, which the ER4STEM team applied to deliver Educational Robotics Workshops (henceforth: ERWs). This report is intended to be used by the ER4STEM target groups, including but not limited to teachers, educators, researchers and academia, as reference material for the design, adaptation, and organization of ERWs following the ER4STEM approach. The quantitative data in this deliverable is based on the information reported through the workshop information forms, provided by each partner in Y1, Y2 and Y3 of the project.
Deliverable 3.1: ECER 2016 Proceedings and Report View/Download 2016 Lepuschitz Wilfried & Varabanov Pavel This document presents the preparations, description and proceedings of the European Conference on Educational Robotics (ECER) 2016, which took place from 11th to 15th April in Vienna, Austria.
Deliverable 3.2: ECER 2017 Proceedings and Report View/Download 2017 Todorova Christina, Gueorguiev Ivaylo, Koza Clemens, Sharkov Georg This document presents the preparations, description and proceedings of the European Conference on Educational Robotics (ECER) 2017, which took place from 24th to 28th April in Sofia, Bulgaria.
Deliverable 3.3: ECER 2018 Proceeding and Report View/Download 2018 Lepuschitz Wilfried, Koza Clemens, Duca Annalise This document presents the preparations, description and proceedings of the European Conference on Educational Robotics (ECER) 2018, which took place between the 16th to 20th April in Qawra, Malta.
Deliverable 3.4: ER4STEM Confference Plan View/Download 2018 Lepuschitz Wilfried, Gueorguiev Ivaylo, Varbanov Pavel, Hopkins-Weaver Sarah This document presents the ER4STEM conference plan, which is derived from the ECER organization within WP3. The purpose is to provide a generic plan on how to organize a student conference. Thus, organizers have guidelines that they can follow, even though the intended event might differ from the ECER organized within WP3.
Deliverable 4.1: First Version of the Activity Plans View/Download 2016 Yiannoutsou Nikoleta, Kynigos Chronis, Gueorguiev Ivaylo, Angel-Fernandez Julian M., Lammer Lara, Vittori Lisa, Xenos Marios, Bitas Christos, Pahouli Agni, Spachos Vasillios, Mavratzas Nikolaos, Baras Giannis, Kitakis Konstantinos, Nikitopoulou Sofia This deliverable addresses the first year of work in WorkQpackage 4: “Pedagogical Design and Innovation for Educational Robotics”. In accordance to the Description of Work (DoW), D4.1 reports on the first version of activity plans developed for ER4STEM and implemented in the first round of workshops. This version of activity plans has been designed with the use of an activity plan template developed by UoAQ ETL aiming to provide tools to the stakeholders of educational robotics to design activity plans that integrate key learning activities like the following:
- transition from individual to collaborative learning perspectives;
- an identification of key skills including collaboration, argumentation, taking individual responsibility in groups, creativity and innovation, constructionism, coding/programming robot behaviours, interactions, field properties;
- a focus on skill development through the sharing of productions;
- a focus on problems which the students perceive as real world;
- the encouragement for students to come up with robotics ideas to solve real problems;
- teacher roles and strategies encouraging the generation of meanings and self-regulation.
The activity plan template was generated from a study of best practices in Educational Robotics (see D.1.1) and from ETL’s experience in designing innovative activity plans for digital learning environments (the method followed for the design of activity plan template is described in detail in D.2.1).
Deliverable 4.2: Operational Release of Activity Plans View/Download 2018 Yiannoutsou Nikoleta, Kynigos Chronis, Todorova Christina, Angel-Fernandez Julian M., Diamantidis Dimitris, Baras Giannis, Girvan Carina, Schuster Lisamarie, Hackenberger Christoph, Gueorguiev Ivaylo, Nikolaos Mavrantzas, Xenos Marios, Nikitopoulou Sofia, Pachouli Agni, Spahos Vasileios This deliverable addresses the second year of work in Work Package 4: “Pedagogical Design and Innovation for Educational Robotics”. In accordance to the Description of Work (DoW), D4.2 reports on the second (operational) version of activity plans developed for ER4STEM to be implemented in the second round of workshops. The operational version of activity plans is based on the first year evaluation, on partner interactions with the ETL research team, who is coordinating the design process, and on the revised version of the activity plan template. The activity plan template was revised according to a) partner comments during the first year of designing activity plans b) Scientix evaluation which was performed during the first year of the project and which highlighted some sections of the activity plan as very academic c) ETLs analysis of the first year activity plans. The revised activity plan template informed the main structure and functionalities of the activity design instrument included in the educational repository (WP5). Furthermore, as we mentioned in D.4.1 the activity plan template was developed by UoA-ETL with the aim to provide tools to the stakeholders of educational robotics to design activity plans that integrate key learning activities like the following:
  • transition from individual to collaborative learning perspectives
  • an identification of key skills including collaboration, argumentation, taking individual responsibility in groups, creativity and innovation, constructionism, coding/programming robot behaviours, interactions
  • a focus on skill development through the sharing of productions
  • a focus on problems which the students perceive as real world
  • the encouragement for students to come up with robotics ideas to solve real problems
  • teacher roles and strategies encouraging the generation of meanings and self-regulation
Deliverable 4.3: Final Release of Activity Plans View/Download 2018 Yiannoutsou Nikoleta, Kynigos Chronis, Pullicino Joanna, Annalise Duca, Gueorguiev Ivaylo, Todorovar Christiana, Angel-Fernandez Julian M., Diamantidis Dimistris, Baras Giannis, Girvan Carina, Schuster Lisamaire, Hackenberger Christoph, Giuliano Angele, Nikolaos Mavrantzas, Xenos Marios, Nikitopoulou Sofia, Pachouli Agni, Spahos Vasileios This deliverable addresses the third year of work in Work Package 4: “Pedagogical Design and Innovation for Educational Robotics”. In accordance to the Description of Work (DoW), D4.3 reports on the final version of activity plans developed for ER4STEM to be implemented in the third round of workshops. The final version of activity plans is based on the second year evaluation recommendations, on the activity blocks developed in year 2 and enriched in year 3, on partner interactions with the ETL research team, who is coordinating the design process, and on the revised version of the activity plan template.
Deliverable 8.1: Data Management Plan View/Download 2018 Lammer Lara, Girvan Carina, Angel-Fernandez Julian M. This Data Management Plan outlines how the research data collected was handled during and after the ER4STEM project. It describes the data set, how it was archived and preserved as well as how it will be shared. It also provides a description on how the data was treated before been uploaded in the Zenodo repository.
Deliverable 8.2: Report on Scientific Paper View/Download 2018 Yiannoutsou Nikoleta, Kynigos Chronis, Angel-Fernandez Julian M., Vincze Markus The purpose of this deliverable is to report on the scientific publications produced during the life of the ER4STEM project. Scientific publications are one of the evaluation criteria of the success of ER4STEM project “Number of peer reviewed articles related to the project published in scientific magazines or at scientific conferences” (DoW pp 7). Thus the role and main objective of this deliverable is to show how ER4STEM orchestrated the scientific dissemination of the project outputs so that a) we ensured wide coverage in terms of targeted audiences, topics and venues b) we highlighted all the different facets and aspects that constitute the ER4STEM approach and c) we identified venues for publications that were relevant to our work but they could also broaden our horizons and help us enrich, refine and evolve our ideas.
Deliverable 8.3: Report on Non-Scientific Dissemination View/Download 2018 Lepuschitz Wilfried, Giuliano Angele, Varbanov Pavel, Angel-Fernandez Julian M., Harner Patrick, Girva Carina, Hopkins-Weaver Sarah, Kynigos Chronis, Grizioti Marianthi, Stola Moroslav The purpose of this document is the collection and description of non‐scientific dissemination activities.
Deliverable 8.4: Report on Established Links with Scientix View/Download 2018 Bugeja Jacquelline, Giuliano Angele, Angel-Fernandez Julian M., Lepuschitz Wilfried, Todorova Christina This document is a report that brings together a collection of activities and collaborations of the ER4STEM project with the SCIENTIX community over the 3 year period of the project.
Overview and Evaluation of a Workshop Series for Fostering the Interest in Entrepreneurship and STEM 2018 Jäggle Georg, Lepuschitz Wilfried, Girvan Carina, Schuster Lisamarie, Ayatollahi Iman, Vincze Markus This document descirbes the ER4STEM activities at the workshops from PRIA and the results of the evaluation. The results show that the workshops foster entrepreneur skills, collaboration and the interest in STEM.
Why do young people lose the interest in STEM subjects? View/Download 2018 Jäggle Georg, Angel-Fernandez Julian, Vincze Markus This is a document desrcibes the several factors, which influence the interest in STEM as a white paper for the repository.
Deliverable 5.1: Mobile Robot Controller Prototype View/Download 2016 Lepuschitz Wilfried, Gueorguiev Ivaylo, Koza Clemens This document describes the prototype of Pocket Bot, a graphical programming environment for the educational robotics controller Hedgehog to be used on smartphones. In the beginning, the Hedgehog controller is introduced, which was mainly developed during the project SCORE! (funded by the Austrian Research Promotion Agency). Then, insights about Pocket Code are given, which acted as basis for Pocket Bot. Finally, details about Pocket Bot are reported.
Deliverable 5.2: Prototype of SLurtles for Kids View/Download 2016 Girvan Carina & Klima Martin This documents presents the SLurtles prototype for kids. SLurtles, having never previously been implemented with young learners will be used for the first time using the ER4STEM framework. To realise the potential of SLurtles with young people they need to be redesigned and implemented in a secure virtual world environment. Once created, SLurtle-based learning activities which follow the ER4STEM Framework will be designed and packaged for teachers working with young learners in a variety STEM contexts. This deliverable reports on the redevelopment of SLurtles and the creation of the virtual environment in which they are used, which has resulted in a prototype ready for piloting with end users.
Deliverable 5.3: Report of the Repository Workshop View/Download 2018 Duca Annalise, Giuliano Angele, Angel-Fernandez Julian M. This document is a report based on the 2 workshops completed by AcrossLimits together with the SCIENTIX ambassadors and/or SCIENTIX National Contact Points to explain how to use the educational repository works ” of Work Package 5, titled “Technology for Educational Robotics”. As part of this work package a state-of-the-art repository was built in support to other deliverables. The aim of the workshops was to ensure that educators, teachers and researchers across Europe get to know about how they can make use of the repository, as well other outcomes of the project.
Deliverable 5.4: ER4STEM Repository View/Download 2018 Duca Annalise, Giuliano Angele, Gueorguiev Ivaylo, Todorova Christina, Varbanov Pavel, Angel-Fernandez Julian M., Lepuschitz Wilfried This document is a report which outlines and explains all the steps undertaken to identify, create and deploy the ER4STEM repository in relation to Work Package 5, titled “Technology for Educational Robotics”. The ER4STEM educational repository is aimed at teachers, robotics practitioners and researchers across Europe to help make resources for robotics in education easily available and helps educators make use of robotics during their lessons.
Deliverable 6.1: Pre-kit for Evaluation View/Download 2016 Girvan Carina & Lammer Lara This documents presents the development and contents of the pre-kit which acts as a prototype to pilot the evaluation tools and methods used in WP6 during year one of the project. It also provides an opportunity to collect base-line data on existing activities involving educational robotics within the consortium. The pre-kit represents the first stage in the development of an evaluation framework and tools for ER4STEM activities (O.6.1) and was completed through Task 6.1.
Deliverable 6.2: Evaluation Tool Kit View/Download 2016 Girvan Carina & Todorova Christina This document presents the development and contents of the evaluation tool kit to be used in the evaluation of workshops during years 2 and 3 of the project. It is designed to be implemented by any partner in any ER4STEM context to collect data which will be analysed and form part of the yearly evaluation.
Deliverable 6.3: Year 1 Evaluation View/Download 2016 Girvan Carina, Yiannoutsou Nikoleta, Grizioti Marianthi Deliverable 6.3 presents an evaluation of the first year of the ER4STEM project. The primary aim of this deliverable is to provide baseline data for the analysis of the Framework in project years 2 and 3. The secondary aim is to inform the development of the Framework and Activity Plans. This deliverable presents data form 48 workshops implemented in four European countries by project partners. The evaluation is not intended to be exhaustive and analysis of year 1 data will continue in project year 2 in tandem with the development of the Framework. Analysis of the data occurs at four levels and from these, 11 recommendations for the Evaluation, Activity Plans, Repository and Framework are made for the start of project year 2.
Deliverable 6.4: Year 2 Evaluation View/Download 2017 Girvan Carina, Grizioti Marianthi, Angel-Fernandez Julian M., Jäggle Georg, Yiannoutsou Nikoleta Deliverable 6.4 presents an evaluation of the first year of the ER4STEM project. The primary aim of this deliverable is to provide an evaluation of the activities implemented in the second year of the project, specifically the workshops and competitions. The secondary aim is to inform the final development of the Framework and Activity Plans. This deliverable presents data form 70 workshops implemented in six European countries by project partners. The evaluation is not intended to be exhaustive and analysis of year 2 data will continue in project year 3. Analysis of data collected in year 2 occurs at three levels looking across competitions, individual workshops, country-level data and project level. From these findings, final recommendations for the Evaluation, Activity Plans, Repository and Framework are made for the start of project year 3.
Deliverable 6.5: Year 3 Evaluation View/Download 2018 Girvan Carina, Hopkins-Weaver Sarah, Saidlova Lenka, Schuster Lisamarie, Greka Christina, Nikitopoulou Sofia, Jäggle Georg, Angel-Fernandez Julian M., Todorova Christina, Lepuschitz Wilfried, Kynigos Chronis Deliverable 6.5 presents an evaluation of the third year of the ER4STEM project, industrial evaluation and whole project evaluation. The primary aim of this deliverable is to provide an evaluation of the year 3 activities implemented in the third year of the project, specifically the workshops and competition. To achieve the primary aim, this deliverable presents the evaluation of 72 workshops implemented in five European countries by project partners and one competition in the third year of the project. The secondary aim is to provide an overall evaluation of the three years of the project, against the research aims, objectives and specific questions outlined in the DoW.