Photograph of Harrie EijkelhofPhotograph of Berenice Michels

The development and implementation of an interdisciplinary STEM course for upper secondary schools

Berenice I. Michels and Harrie M.C. Eijkelhof
Freudenthal Institute,
Utrecht University

Abstract

i

Since 2007, a new interdisciplinary course ‘Nature, Life and Technology’ (NLT) has been taught in more than 200 Dutch senior high schools by teams of teachers, in addition to the regular courses in mathematics, physics, chemistry, biology and earth science. The goals of NLT are to motivate students for a career in science and technology and to show them the coherence between the traditional STEM subjects. Teams of teachers and experts from research and industry developed more than 80 interdisciplinary modules. This paper describes the development and implementation process, focusing on the design principles in the development of standards, development of materials, professional development (PD), implementation at schools and monitoring activities. From this experience, several lessons can be learned for comparable challenges.

Introduction

ii

In the years 2005 – 2015 a new interdisciplinary course for upper secondary called ‘Nature, Life and Technology’ (NLT) was successfully developed and implemented in the Netherlands. The process started in late 2005, with no course description, no curriculum and no materials. In September 2007 the course was offered for the first time at 175 schools and at the end of 2015, 235 schools (approximately 40% of all upper secondary schools in the Netherlands) offered the course to around 17000 students each year.

The project to develop and implement the new subject was not a research project. Neither time nor budget allowed the governing Steering Committee and the executive Project Committee to follow the usual steps of design-based research or educational design research. Nevertheless, we think it relevant to report on this process. Plomp (2013) describes educational design research as having two explicit goals: firstly to solve the complex educational problem it is addressing and secondly to advance our knowledge about the characteristics of these interventions and the processes to design and develop them. In reporting on the process of development and implementation (D&I) of NLT, we seek to contribute to this advancement of knowledge and to offer insights from this D&I process that are relevant for comparable design challenges.

The goal of this paper therefore is to analyse the way in which several design principles were applied to the different stages in the development and implementation of NLT, and which lessons can be learned from this process for comparable challenges in other contexts.

Relevance of the paper

1

The relevance of this article is found in a set of challenges of the D&I process for NLT, with which other design projects might also be confronted. Firstly, as stated above, the project had to be completed under strong time pressure. Secondly, the course was new, so the developers could not rely on an existing curriculum, nor re-use existing materials. Moreover, the unfamiliarity of the new subject poses a greater challenge to make it a durable success. Furthermore, the subject had to be interdisciplinary and of current topical interest. This made it difficult to develop the course with a small group of (experienced) educational designers, as they would not have the required knowledge on a broad range of recent STEM topics. Topicality also complicates the maintenance of the developed materials, as it can cause materials to become out-of-date very quickly. Finally, the course needed a thorough system of quality control, as students would be allowed to take the course as part of their high school examination. This is a high-stake examination because successful Netherlands high school graduation grants general access to higher education, without further entrance exams. (Certain subjects may be required for specific studies, but NLT is not required for any bachelor study.) The course would not have a central or external refereed test: the examination would be fully school-based.

Outline of the paper

2

After a description of the context in which the D&I process took place, we start our analysis by listing the design principles used for this process. Thereafter, we will describe the key stages of the D&I process: the development of standards, the development of teaching materials, professional development of teachers, implementation of the course in schools, and the monitoring and evaluation activities performed throughout the D&I process. In the descriptions we will pay attention to the way the design principles guided the process and to adaptations that were made, based on experiences during the process.

Context description

iii

Why a new course?

Around the world, science education is in motion. Science curricula are in the process of revision or have recently been renewed (Eilks & Hofstein, 2017; Galvão & Abrantes, 2005; Mamluk-Naaman, 2007; Millar, 2006; Next Generation Science Standards (NGSS) Lead States, 2013; National Research Council, 2012; Parchmann, Grasel, Baer, Nentwig, Demuth & Ralle, 2006; Royal Society, 2014; Szybek, 2005; Wei, 2009) This development has been going on since the 1960s (Atkin & Black, 2007; Ogborn, 2005). There are a variety of reasons behind the dynamics of curriculum development, which can be described from a utilitarian, a democratic, an economic and a cultural perspective (Millar & Osborne, 1998; Osborne & Dillon, 2008). In modern societies, many new science related issues have emerged, for instance in the fields of information technology, risk and safety, water management, energy production and consumption, health issues and environmental affairs. Many of these issues require combined insights from various disciplines. From all three perspectives, STEM students need insight into basic interdisciplinary knowledge and into the science behind these issues.

Around the beginning of the century, concern grew in the Netherlands about the number of students choosing a career in STEM fields. In 2004 the Dutch government therefore launched a national action plan for STEM (Ministry of Education Culture and Science (OCW), 2004), with the goal to increase the number of STEM students by making the programmes more attractive to upper secondary students. In line with these developments a government advisory board, including highly respected scientists, in 2005 recommended the introduction of a new course in the upper secondary curriculum: ‘Nature, Life and Technology’.

A priori key features of the course

2

The aim of the new course was firstly to motivate students for a career in STEM and secondly to show them the coherence between the traditional STEM subjects. The advisory board required the course to be interdisciplinary, combining and broadening the curricula of biology, chemistry, earth science, physics and mathematics. It had to address current STEM issues in a modular form. This meant that the curriculum would consist of separate, independent teaching modules, each addressing a different topic in a fixed amount of study time.

The new course targeted students in Upper Secondary Education (age range of 15 – 18 years), following a STEM profile in the pre-university or pre-college track. In the Dutch system, in both tracks, students choose one of four profiles in upper secondary, two of which are considered STEM profiles. The profile determines the subjects followed by the students, with just one or two electives. All students within this target group follow chemistry and mathematics courses. Besides that, they study either physics or biology or both. Some students take earth science as well. The pre-college students would spend 320 hours on NLT over the course of two years; the pre-university students would spend 440 hours in three years. These hours include all time spent on the subject in class and at home. The course is elective, both for schools and for students: schools are free to decide whether to offer the course and students are free to decide whether to take the course.

Context for development and implementation

3

Some challenges arose at the start of the development process, originating from both the political constraints and the Dutch educational tradition. Firstly, there was the short timeframe: as the introduction of NLT was part of a larger scale reform of the upper secondary programme, schools had to be able to offer the new course from 2007. This prohibited a pilot phase with a small number of schools.

Secondly, no obligatory professional development (PD) course could be required: every teacher certified to teach a STEM course in upper secondary (either biology, chemistry, physics, mathematics or earth science) would be allowed to teach the new course. However, the teachers could not all be expected to be aware of the new developments in STEM, which they were going to be teaching in this new course. Therefore, PD would surely be necessary, though not officially required.

Thirdly, in the Netherlands, schools have considerable autonomy (Organisation for Economic Co-operation and Development, 2016, p. 119) about many aspects of the curriculum. Therefore, from all the products resulting from the D&I process of NLT, schools would only be required to follow the standards. Schools had to be seduced rather than compelled to use other key parts of the new curriculum, such as the teaching materials, the modularity and the implementation tools.

Finally, the attitude towards interdisciplinarity among science and mathematics teachers in the Netherlands is ambivalent: some teachers are strong supporters of more coherence between the STEM subjects at school; others are very suspicious as they fear that interdisciplinarity will mean the end of their own subject or may lead to superficial teaching of subject knowledge. There is no tradition of interdisciplinary STEM teaching in the Netherlands, as often, even in lower secondary schools, STEM subjects are taught as separate subjects, teachers function in separate departments at their schools and the national examinations have separate exams for each subject. Thus, combining knowledge from different STEM subjects is generally left to the students.

Starting situation

4

In January 2006, the government appointed a Steering Committee and gave it the task of developing and overseeing the implementation of the new course. The committee consisted of university representatives from the different STEM subjects (mostly members of the committees that were redesigning the other STEM curricula), a representative of the colleges (universities of applied sciences), and members of the national STEM teacher associations. Furthermore, an executive Project Committee was formed, consisting of STEM curriculum experts, STEM teacher trainers and STEM teachers. The Steering Committee received funding (2 million euros to develop and implement the new course over five years (2006 – 2010). Later, additional funding (1.6 million euros) was granted for the years 2011 – 2015, to anchor the course durably in the Dutch educational system.

Many schools showed interest in the new course from the very beginning. Both school boards and teachers were interested to follow the process and to take part in the development of the new subject. For school boards the main reason for this interest lay in the opportunity to stimulate teachers from different STEM departments to cooperate, to create more coherence between the different subjects, and to develop new teaching strategies. Teachers were mostly interested to join the development of the new course because of the opportunities it offered to get in contact with new developments in STEM, to develop themselves professionally and to teach new and interesting topics not included in their regular curricula.

Most research universities showed great interest in NLT. They saw the new course as an opportunity to valorize research to students, teachers and the general public; to attract more motivated and better prepared students; and to raise the entrance level of new students. Most colleges showed less intrinsic motivation to participate in the new development at the start of the process, possibly due to high teaching loads of their staff and less involvement in research.

Appendix 1 gives a list of all certified NLT modules at the beginning of January 2018, and links to student materials in Dutch and English. A sample module ‘Molecular Gastronomy’ is available here.

Appendix 1 gives a list of all certified NLT modules at the beginning of January 2018, and links to student materials in Dutch and English. A sample module ‘Molecular Gastronomy’ is avaiable from educationaldesigner.org/ed/volume3/issue12/article48/pdf/molecular-gastronomy-manual.pdf.

Design principles

iv

To develop the new course according to the requirements and within the constraints given, the Steering and Project Committees used a set of design principles. The Steering Committee formulated some of these principles explicitly in the outline of the course (Steering Committee NLT, 2008), and other principles emerged during the process. Following Van den Akker (1999), we discern substantive principles (focusing on the characteristics of the intervention – the new course) and procedural principles (focusing on the design activities for the intervention – the new course). The ten resulting principles are summarized in Table 1 and discussed more fully below.

The full outline document is available here.

The full outline document is available from educationaldesigner.org/ed/volume3/issue12/article48/pdf/molecular-gastronomy-manual.pdf.

Table 1: Ten design principles for the development and implementation of NLT.
Principles Relevant for
Development Implementation
Substantive principles
1 Covering a broad spectrum of topical S&T developments X
2 Interdisciplinary X
3 Cooperative X
4 Modular X
5 Context-based and activity-oriented X
Procedural principles
6 Cyclic where possible X
7 Involvement of Secondary teachers and Higher Education X X
8 Quality enhancement measures X X
9 Monitoring and evaluation activities X X
10 Effective communication X

Substantive principles

1

Five substantive design principles were defined, following from the goals of the new course, the a priori key features of the course and general curriculum expertise.

  1. The first goal of the course, to motivate students for a career in STEM, should not result in the course being a ‘commercial’ for specific university majors. The Committee therefore operationalized this goal as engaging students with a broad spectrum of current developments in STEM. In that way, students would be able to discover for themselves how attractive (or not) a career in STEM could be and what possibilities exist.
  2. The second goal of the course, to show the coherence between the STEM subjects, is covered by the design principle of the course being interdisciplinary (English, 2016; Rennie, Wallace & Venville, 2012). This also contributes to the first goal, as it allows students to become involved in a whole new array of STEM topics, generally not covered by the traditional disciplines.
  3. The interdisciplinarity principle also incorporates cooperation between the contributing disciplines, e.g. by teachers from different disciplines. This is necessary, as one single teacher will not have expertise of the broad spectrum of current developments in STEM.
  4. Furthermore, the Steering Committee decided the course had to be modular, to give teachers (and students) the opportunity to personalize the course to their interest by choosing particular modules. Modularity also helps the longer-term challenge to maintain currency as out-of-date modules can be replaced with new ones as the need arises.
  5. The modules in the course had to be context-based and activity-oriented, as this would support the interest and active participation of students in the course (Bennett, Lubben & Hogarth, 2007; Hake, 1998).

Procedural principles

2

Besides the five substantive principles, five procedural design principles guided the development and implementation process of NLT.

  1. From a development perspective, the Steering and Project Committees wanted the design process to be cyclic where possible, though this was limited due to the time constraints.
  2. Another important principle for the development was the involvement of both teachers from secondary schools and experts from higher education. Involvement of teachers in the development process would enhance ownership, which would contribute to a durable success. Teachers would bring their knowledge of the students’ potential into the development process: teachers from different subjects would know which knowledge and skills students might have or lack, and experienced teachers could tell what to expect from students when learning about a new topic. Furthermore, this involvement could function as a form of PD. Involvement of higher education was necessary to ensure the goal of orientation on STEM careers and majors, and to give a realistic image of STEM studies and careers. Experts from universities and some research centres, museums, etc. would bring state-of-the-art knowledge into the development process.
  3. To ensure the quality of the course, quality enhancing measures would be necessary throughout the D&I process.
  4. Likewise, monitoring activities had to be carried out, as far as time and finances allowed. Most of the procedural design principles formulated from a development perspective are important from an implementation perspective as well. Involvement of secondary teachers in the development process simplifies the introduction of the new subject in their schools, higher education professionals could play an important role in the professional development of teachers, and quality enhancing measures and monitoring activities support both successful development and implementation.
  5. From an implementation perspective, another design principle was effective communication. This was to enable the Project Committee to communicate well and support the schools with the implementation process. The first step of this communication was a registration system for all schools offering the new course, which also allowed monitoring of the implementation process.

Development and implementation process

v

In this section, we describe the main stages of the D&I process of NLT: the development of the standards and the teaching materials, the professional development of teachers and the measures and directives supporting the implementation of NLT at schools. We show how the main design principles (see Table 1 ) guided these processes and how monitoring activities provided input for decisions on the D&I activities.

Standards

1

One of the main tasks of the Steering Committee was to deliver to the Ministry of Education advice on the standards of the new subject, the only official requirements for the examination of the course. Standards for school-based exam subjects must be very general and broad; the specification is left to teachers and authors of educational materials. All subject standards for upper secondary education contain a skills domain and several content domains.

In 2007, a first version of standards was needed so schools could offer the course. The Steering Committee considered this version as a pilot version, which was tested during the years 2007 – 2010. In 2010 the Committee produced with a revised set of standards as advice to the Ministry, based upon the experiences during the pilot.

The first set of standards was developed in close co-operation with stakeholders from higher education. The skills domains of the NLT standards were aligned with standards for biology, chemistry, and physics; the content domains of NLT were formulated so that every STEM major at a general or applied university was represented in the NLT standards.

During the first years of implementation, the Project Committee conducted school visits to get an impression of the implemented curriculum. During these visits, the Committee discovered that many teachers interpreted NLT mainly as a ‘skills course’: due to time pressure in regular physics, chemistry and biology lessons, they covered skills such as doing research, modeling and designing in those subjects only superficially. NLT gave them the chance to treat these skills more thoroughly. Broadly, however, teachers and students did not recognize some essential features such as (learning about) interdisciplinarity and orientation for STEM majors as characteristics of NLT. Therefore, the Steering Committee decided to define explicitly four essential characteristics of NLT, and to include these in the new standards. The defined essential characteristics of NLT are:

This 2010 version of the NLT standards therefore consists of three parts described below. Table 2 gives examples of these standards and the way the characteristics of NLT are included.

Table 2: Examples from the 2010 NLT standards for the pre-university track (Krüger & Eijkelhof, 2010).
Domain Standard
Science skill -
A3: Modelling
The candidate can analyze a relevant problem in contexts, limit it to a manageable problem, translate it into a model, generate and interpret model outcomes, and test and evaluate the model. For this, the candidate uses consistent arguments, and relevant arithmetic and mathematical skills.
NLT specific skill –
Collaboration
The candidate can collaborate with others while working on interdisciplinary problems, thereby actively stimulating, evaluating and using the input of others as well as having a recognizable own contribution to the final result.
NLT-specific:
Interdisciplinarity
The candidate can apply to the context relevant conceptual knowledge and approaches from earth science, biology, physics, chemistry and mathematics to interdisciplinary issues, relating to the [content] domains [of these standards]
NLT-specific:
Interplay between science and technology
The candidate can describe the interplay between the development of scientific knowledge and that of technology and clarify this interplay with examples relating to the [content] domains [of these standards]
Content –
C1: Processes in living nature, Earth and space
The candidate can apply scientific and mathematical concepts to explain interdisciplinary processes in the fields of living nature, Earth Sciences and Space Sciences.

Development of materials

2

The first Steering Committee decided that by 2010, 50 modules had to be available for schools: 20 for the pre-college track and 30 for the pre-university track. This would make it possible to cover a wide spectrum of STEM developments and allow schools enough options to offer a good choice of modules. Pre-college students must follow 6, 7 or 8 modules and pre-university students follow 8, 9, 10 or 11 modules.

For the development of these modules, the Steering and Project Committees designed a strict procedure. First, the whole development process was designed to follow a so called ‘wave model’: every half year the development of a new set of modules was started, the nominal development time of a module being 18 months. Secondly, all modules had to be developed by teams consisting of science or mathematics teachers from at least two different schools, and at least one content expert, e.g. from a university, a research institute or industry. Each team had to follow the same four-step procedure (see Figure 1):

  1. Outlining: an outline of the module was developed and submitted for approval, including learning goals and forms of assessment.
  2. Drafting: the team developed a draft version of the module.
  3. Validating: the draft version was tested at one or two schools and evaluated by a content expert and a science educator. Neither the schools nor the experts were involved in the development of the module. Teachers and students reported about their experiences with the module.
  4. Finalizing: a final version of the module was developed, taking into account the comments on the draft version from students, teachers, content expert and science educator.
Figure 1. The wave module for the development of 50 NLT modules The wave module for the development of 50 NLT modules

Each wave consisted of the development of some precollege modules (PC) and some pre-university modules (PU) according to the same four-step procedure.

The multi-perspective validation of the draft materials by teachers, students, and content and educational experts appeared to be very useful as they provided different types of feedback (Eijkelhof and Krüger, 2009). Teachers commented mainly on the quality of the student assignments, the feasibility for the target group, the costs, and the required content and pedagogical support. The students reacted on difficulty level, wording, clarity of assignments, and preferences for practical work. The scientists and engineers had much to say about the attractiveness of the field, wordings, and complexity of the topics. They suggested many additions in the modules and referred to external sources of information. Comments by the science educators dealt with the feasibility for the target group, the teaching approach, the coherence of the module and the relation with regular school subjects. The main challenge in rewriting was to select and combine this variety of comments. When the feedback was extensive or complex, sometimes step three and four were repeated before the development team delivered the final version. In Table 3 we provide as an example some feedback and the resulting changes for the module Molecular Gastronomy, which was developed as an external module by Wageningen University. We selected this module, as it is one of the four modules that have been translated into English. Finally, the final version was submitted to the Steering Committee for certification. A list of the certification criteria is given in Table 4.

Table 3: Examples of feedback provided and processed for the module Molecular Gastronomy
Feedback from Given feedback Resulting change in module
Teacher at test school The module is designed for Grade 12, but is too easy for these students. The module was redesigned for Grade 10/11.
In task 3 (p. 22) students recognized the food from its texture. Added to the teacher manual that the different foods should be mashed before use in the task.
Student at test school Too much introduction; wording too ‘childish’. Texts are edited with special attention to the level of the language.
Educational expert Try to pay more attention to the majors and professions, related to this topic, without making a ‘commercial’ for specific majors at your own university. Profession boxes are added, recruiting information on specific majors is deleted from student manual.
Some terms are missing from the list of definitions Added.
Content expert Wonders if the topic is too difficult for Grade 12; refers to teachers to decide The teachers found the module too easy, not too difficult for Grade 12.
Some errors e.g. in the percentage of fat in milk Fixed.
Table 4: Certification criteria for NLT modules.
Certification criteria NLT modules

The module…

  1. Was developed by a team consisting of of science or mathematics teachers of at least two different schools, and at least one content expert.
  2. Was developed following the four-step quality procedure
  3. Consists of student material, teacher material and (exemplary) assessment material
  4. Incorporates all of the following characteristics:
    • interdisciplinary: integration of two or more of the disciplines Physics, Chemistry, Biology, Mathematics and Earth Science
    • student activity based
    • focus on current state of knowledge in the outside world
    • connection to actual developments in the field
    • interesting/challenging for students
    • language use suitable for the target group
    • connection with the curriculum of regular high school science and mathematics subjects
    • feasible for 40 hours of student work
    • feasible across the country, not too local
    • copyrights: Creative Commons
  5. Limited in size (not too bulky)*
  6. Pays attention to the four characteristics of NLT: interdisciplinarity, orientation on S&T majors in Higher Education and careers, the role of mathematics in S&T, and the interaction between sciences and technology*
  7. Maintenance agreements with Regional Support Centres on digital components, if applicable, on yearly small maintenance and on revision after five years*

* Criteria 5, 6 and 7 were added later, based on evaluation results.

The wave model proved to be very useful for several reasons. First, it enabled the Project Team to handle the large number of modules to be developed, by chopping the development into feasible portions. Another advantage was the fact that procedures in subsequent waves could be adapted based upon the experiences in the first wave. Indeed, several adaptations were made. Firstly, the Project Committee provided PD courses for teachers who were developing educational materials as it became clear that they did not all have the necessary skills. Secondly, the Project Committee selected commonly recurring feedback from different draft modules and gave this to new development teams, so that they could take into account these points of attention beforehand. Thirdly, the Project Committee accepted more flexibility in the roles within development teams. For example, different ways were found to fulfill the expert role: (1) as principal writer, (2) as one writing member of the development team, and (3) lecturing the teachers before they started writing the module with the expert in a ‘consultant’ role.

The Steering Committee designed the certification criteria given in Table 3 to ensure the quality of the modules developed by their teams. Later, however, it facilitated a key aspect in the development NLT: the development of so called ‘external modules’ by others such as universities, companies and other institutions. The Steering Committee could certify these modules, developed without their prior involvement, with the same set of certification criteria. By 2010, besides the 50 modules developed by the Steering Committee, 15 external modules had been developed and certified. By June 2016, 26 of the 78 certified modules had been externally developed. Appendix 1 gives an overview of all modules available on January 1, 2018 and links to student materials.

After the first five years of development, the Steering Committee made a change in the certification procedure. From 2011 on, certification was granted only for a period of five years. After these years, the modules had to be checked extensively, to see if they were still accurate and topical. Modules could be revised, re-certified for five years or de-certified.

Professional development

3

Due to NLT’s innovative character it was evident right from the start that professional development of teachers was very important to establish the new subject in Dutch secondary schools. This professional learning took place in a variety of forms. In the first place, teachers in development teams learned from developing modules: content experts shared their knowledge with them and during the development process the teachers gained curricular and didactical experience. Teachers from 150 schools were actively involved in developing new modules and on working groups, in close cooperation with experts from 50 institutes.

Secondly, during the implementation in schools, all teachers had to dive into new topics and by cooperating with colleagues in NLT teams, they became familiar with teaching content and methods in other subjects. Thirdly, special PD activities were designed including an annual conference and special PD courses organized by regional support centers at universities or colleges.

Implementation at schools

4

In the Netherlands, regular STEM-subjects are assessed by national examinations, based on detailed syllabi. Although the purpose of these exams is to provide students with access to higher education, they are also used to monitor the quality of education in these subjects: schools where students achieve relatively poorly on these exams are subject to controls by the inspectorate.

As NLT is assessed only within the school, the Steering Committee had to find other ways to ensure the quality of teaching NLT. Due to Dutch school autonomy, the Committee did not have the power to enforce regulations. Therefore, with a few early exceptions, the Committee granted registration and easy access to modules for those schools that committed themselves to comply as much as possible with certain requirements.

Firstly, NLT had to be taught by a team of at least three teachers, with different disciplinary backgrounds, to avoid superficial lessons, to promote real interdisciplinarity, and to show students the contributions from various disciplines. Secondly, schools had to use certified modules for at least 75% of their NLT curriculum, to ensure the use of quality teaching materials. Thirdly, school boards had to facilitate NLT teachers with time for team meetings, for professional development and for establishing contacts with external institutes such as universities and industry. Finally, schools had to cooperate and take part in evaluation and monitoring activities from the Project Committee.

Later, as school visits showed that some schools used one-sided means of assessment, a fifth requirement was added that assessment had to consist of both paper and pencil tests and other methods, such as presentations, reports of investigations and design activities.

Monitoring activities

5

During the D&I process, the Project Committee undertook several monitoring activities. Firstly, members of the Committee visited 23 schools from 2008 to 2014. Aims of the one-day visits were to find out why schools were offering NLT, if teaching was in line with the views of the Steering Committee and how NLT was anchored within the schools. During the visit, school practice was discussed with the head of the NLT department, NLT teachers, technicians, school administrators and students. The committee members observed lessons and collected information about scheduled hours, module use, grading and experiences. In 2013/2014, they collected information on how teachers and school administrators envisioned the future of NLT after 2015. Secondly, registration data of 2007 and 2014 was collected and analyzed on the composition of and cooperation within NLT-teacher teams, on facilitation by school administrators and on the use of certified modules. Thirdly, data of school examinations provided by the Ministry of Education were analyzed on student subject choices and final grades of NLT. Finally, a small number of first year undergraduate students who took NLT at high school were interviewed, and in two consecutive years experiences of graduating high school students were collected in short surveys (N=182 in 2013 and N=390 in 2014). Some of their reactions are presented in Table 5. Reports on all these evaluation activities can be found (in Dutch) at the NLT website (National Coordination Committee on NLT, undated). The results of all these monitoring activities contributed to decisions on the D&I process.

Table 5: Some reactions of students after completing the course.
Question: Why would you (not) advise a grade 9 student to take NLT next year?

“I advise other students to take NLT because it is about the connection between different sciences and technology, so it is more like the real world”

“It makes the choice of an undergraduate programme more difficult: you get in touch with a broad scope of S&T programmes, so you might end up with 3 programmes you'd like to choose”

Question: In what way did NLT support you by choosing your major?

“NLT showed me my favorite way of learning, which didn't influence the choice of my major, but did influence the choice of my university”

Question: In what way do you expect NLT to be of benefit for your subsequent education?

“In NLT I discovered that math is about more than high school lessons”

“NLT is the most important course I took. I am going to study physics, astronomy & mathematics, which I wouldn't have done without NLT: physics in high school is not challenging enough. NLT gave me a scent of real physics”

Discussion

vi

During the development and implementation of NLT we used substantive and procedural design principles. Now, we discuss the experiences with each of these principles, to finish with the formulation of the main lessons learned.

Substantive Principles

1
  1. Covering a broad spectrum of topical STEM developments appeared to be ambitious but feasible. It was done through the standards and the modules developed. The principle was introduced to motivate students for a career in STEM, one of the goals of NLT. In evaluations, several students reported that studying NLT modules helped them choose further studies.
  2. Many STEM research fields are currently interdisciplinary so covering many STEM fields in the modules supported this principle, though it raised discussions about the nature of interdisciplinarity.
  3. The importance of interdisciplinarity within NLT was also relevant for the cooperation of teachers in schools as experience with co-creating and co-teaching is not common in the Netherlands. At almost all schools, teams of three or more teachers are responsible for teaching NLT. Often teachers within teams distribute modules amongst them, according to their expertise: one teacher teaches the module, the others provide support from their own disciplinary background. An advantage of this team teaching appears to be that teachers become more familiar with teaching practice in other subjects, which contributes to the NLT goal to show the coherence between separate STEM subjects. Moreover, students experience through the team-teaching of their teachers that interdisciplinarity requires cooperation: within the modules, they have to cooperate themselves while in the classroom their teachers cooperate as well.
  4. The modular format of the course appeared to be very useful: it made specific cooperation with external experts possible and it offered teachers the opportunity to choose.
  5. Contexts for the modules were abundantly available and teachers and students made successful pleas to include many student activities to avoid a lecture-type course.

Procedural Principles

2
  1. The cyclic-where-possible principle was vital, especially at the start of the development process when many modules had to be developed concurrently and experiences with developing modules were new and had to be shared. During the whole process of D&I, a cyclic approach was applied, as results from different monitoring activities were being used to improve the process. Very positive experiences were gained with the four-step procedure of developing modules, including the need to trial draft modules and the importance of collecting comments from teachers, students, external experts and science educationalists and the requirement of certification by the Steering Committee. In classical educational design research, new interventions are first designed in a strict cyclic way on a small scale and implemented when the design is successful. For NLT this approach was not possible, as the implementation had to take place semi-simultaneous to the design. However, this proved to be of advantage as well: in educational design research, designing for scalability is an issue that has proven hard to tackle. Pedagogical strategies successful in one or two classrooms will not automatically succeed in different classrooms with different students and different contexts (e.g. Clarke, Dede, Ketelhut & Nelson, 2006). For NLT, scalability was less of a problem, since we worked on a very large scale (175 schools) from the very beginning.
  2. Very essential appeared to be the involvement of both teachers and external experts in the development process, as both used their specific experience: teachers in the target group, and experts on topical developments in the STEM field of the module. The involvement of higher education was not only of importance for the development of materials. The regional support centers that evolved during the D&I process, took up the important task of maintaining and revising modules, to keep them up-to-date. Moreover, they play an important role in PD for NLT. At some universities hosting a regional support center, research has been initiated into the details of PD for NLT (Visser, 2012) and the development of materials by teacher design teams (Binkhorst, 2017). Most regional support centers extended their involvement with secondary education to all STEM subjects and to both teacher development and student activities (Michels & Eijkelhof, 2018).
  3. To ensure quality, the Steering Committee defined both certification criteria for the modules and implementation criteria for schools. These criteria proved to be essential for the success of NLT as they made different stakeholders trust the quality of the new subject.
  4. Different monitoring activities (e.g. school visits and investigations) appeared to be useful for adjusting the process in almost all stages of the D&I process. It might have been better to plan concurrent research right from the beginning, despite the time pressure and limited finances.
  5. Finally, the decision to ask schools to register and to commit themselves to follow certain requirements as much as possible, had positive effects for teachers and avoided an ‘anything goes’ approach in a course without national examination. Furthermore, this registration allowed the Project Committee to provide schools with targeted communication, through newsletters and a database.

Lessons learned

3

In the D&I process of NLT, we faced several challenges, as described in the introduction of this paper. For other design tasks, facing comparable challenges, five main lessons can be learned from the D&I of NLT.

  1. Need to formulate design principles. When developing and implementing a new course semi-simultaneously, design principles should be formulated at the start of the process. One of the principles should be cyclic when possible, making adjustments during the process possible. An advantage of the semi-simultaneous development and implementation is the absence of scalability issues, as the work is done on full scale from the beginning. Furthermore, despite time pressure, an achievable concurrent research plan for monitoring and evaluation should have been drawn at the start.
  2. Need of cooperation. When developing and implementing an interdisciplinary subject for secondary schools, cooperation between teachers and higher education experts is sensible: teachers know what to expect from students, experts are familiar with the topical issues in an interdisciplinary field. Higher education can play an important role as well in maintaining materials and in the PD of teachers. Finally, the subject must be taught by teams of teachers with different backgrounds. They provide an example to students of interdisciplinary cooperation and together a team of teachers can cover the broad content of an interdisciplinary module.
  3. Development of modules in a four-step procedure. When developing different modules on a broad range of topics, the quality of the modules developed by different teams can be ensured by using a four-step procedure (outlining, drafting, multi-perspective validating and finalizing), and concluding with certification according to well defined criteria.
  4. Development of sets of modules in waves. When developing many modules in a limited time, subsets of modules can be developed in separate waves. Each wave follows the same procedure, different waves starting consecutively but overlapping with each other in time. In this way, the different tasks are spread over time and subsequent waves can profit from experiences in the first waves.
  5. Quality through implementation requirements. When implementing a course with no external quality control (e.g. no external exams in a context with large autonomy for schools), quality can be encouraged by stimulating schools to comply with important implementation requirements. For this, schools can be requested to register to get access to relevant information or materials.

NLT after 2015

vii

In 2016, the Steering Committee completed its work to develop and implement NLT in schools. However, much work is still required to keep schools aboard and to strengthen the quality of NLT education. Fortunately, schools recognized this need and an association of NLT schools has been formed. Currently 180 schools have registered and pay annual fees, making the continuation of the NLT development possible. Important tasks for the NLT Association are updating existing modules and teaching guides, developing new modules, supporting professional development both at the regional and national level, stimulating peer audits by schools, and maintaining a network with universities, industry, teacher associations, and supporting research.

In the Netherlands, NLT has been fairly successful in bringing new STEM developments into schools, in stimulating cooperation between teachers of different disciplinary background and in setting-up networks between universities, industry and schools. Cooperation at the international level would be useful to share experiences.

References

viii

Atkin, J.M. & Black, P. (2007). History of science curriculum reform in de Unites States and the United Kingdom. In: S.K. Abell & N.G. Lederman (Eds), Handbook of Research on Science Education (pp. 781-806). Mahwah N.J.: Lawrence Erlbaum.

Bennett, J., Lubben, F., & Hogarth, S. (2007). Bringing science to life: A synthesis of the research evidence on the effects of STS and context based approaches to science teaching. Science Education, 91, 347–370.

Binkhorst, F. (2017). Connecting the DOTS, Supporting the implementation of Teacher Design Teams. Enschede: Doctoral thesis, University of Twente, Netherlands.

Clarke, J., Dede, C., Ketelhut, D.J., & Nelson, B. (2006). A design-based research strategy to promote scalability for educational innovations. Educational Technology, 46(3), 27-36.

Eijkelhof, H. & Krüger, J. (2009). Improving the quality of innovative science teaching materials. Paper presented at the ESERA 2009 Conference, Istanbul, August 31 – September 4, 2009. Retrieved from: http://betavak-nlt.nl/dmedia/media/site-files/097e4/7477f/63a78/0201d/99f9c/ESERA-2009-Proceedings.pdf.

Eilks I. & Hofstein A. (2017). Curriculum Development in Science Education. In: K.S. Taber & B. Akpan (Eds) Science Education. New Directions in Mathematics and Science Education (pp 169-181) Rotterdam: Sense Publishers.

English, L.D. (2016). STEM education K-12: perspectives on integration. International Journal of STEM Education, 3, 3, 1-8.

Galvão, C. & Abrantes, P. (2005). Physical and Natural Sciences - a new curriculum in Portugal. In P. Nentwig & D. Waddington (Eds), Making it relevant. Context based learning of science (pp. 175 - 194). Münster: Waxmann Verlag.

Hake, R.R. (1998). Interactive-engagement vs. traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64–74.

Krüger, J. & Eijkelhof, H. (2010). Advies Beproefd Examenprogramma NLT; eindrapportage Stuurgroep NLT (in Dutch) [Advice on tested Programme of Examination NLT; final report Steering Committee NLT]. Stuurgroep NLT. Retrieved from: http://betavak-nlt.nl/dmedia/media/site-files/76dc8/70435/1b11e/e1164/734ad/Advies_beproefd_examenprogramma_NLT_december_2010.pdf.

Mamluk-Naaman, R. (2007). 'Science and Technology for All'- an Israeli curriculum based on new standards in science education. In D. Waddington, P. Nentwig & S. Schanze (Eds), Making it comparable. Standards in science education (pp. 199-220). Münster: Waxmann Verlag.

Michels, B. & Eijkelhof, H. (2018). High school students engaging with researchers within a pre-university programme: motivations and experiences. Research for All, 2(1), 43 – 61.

Millar, R. & Osborne, J. (Eds) (1998). Beyond 2000: Science education for the future. London: Kings College, School of Education.

Millar, R. (2006). Twenty First Century Science: Insights from the design and implementation of a scientific literacy approach in school science. International Journal of Science Education, 28, 1499-1521.

Ministry of Education Culture and Science (OCW). (2004), Deltaplan beta/techniek; actieplan voor de aanpak van tekorten aan bèta’s en technici (in Dutch) [Deltaplan science/technology; action plan for tackling deficits in natural scientists and technicians]. Retrieved from: http://www.scienceguide.nl/media/1936364/onderwijs-deltaplan-deltaplan.pdf

National Coordination Committee on NLT. (undated). Vereniging NLT. Natuur, Leven, Technologie. Feiten en cijfers. Retrieved from: https://betavak-nlt.nl/nl/p/over-nlt/feiten-en-cijfers/

National Research Council (2012). A Framework for K–12 Science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.

Next Generation Science Standards Lead States (2013). Next Generation Science Standards: For states, by states. Washington, DC: The National Academies Press. Retrieved from: http://www.nextgenscience.org/

Organisation for Economic Co-operation and Development (2016). PISA 2015 Results (Volume II): Policies and Practices for Successful Schools. Paris: OECD Publishing. Retrieved from: http://www.oecd.org/education/pisa-2015-results-volume-ii-9789264267510-en.htm

Ogborn, J. (2005). 40 Years of curriculum development. In K. Boersma, M. Goedhart, O. de Jong, & H. Eijkelhof, Research and the Quality of Science Education (pp 57-65). Dordrecht: Springer.

Osborne, J. & Dillon, J. (2008). Science Education in Europe: Critical Reflections. London: Nuffield Foundation.

Parchmann, I., Grasel, C., Baer, A., Nentwig, P., Demuth, R., & Ralle, B. (2006). Chemie im Kontext: A symbiotic implementation of a context-based teaching and learning approach. International Journal of Science Education, 28, 1041–1062.

Plomp, T. (2013). Educational Design Research: An Introduction. In: Tjeerd Plomp & Nienke Nieveen (Eds), An Introduction to Educational Design Research (pp 10 - 51). Enschede : SLO. Retrieved from : http://international.slo.nl/publications/edr/

Rennie, L., Wallace, J., & Venville, G. (2012). Exploring curriculum integration: why integrate? In L. Rennie, G. Venville, & J. Wallace (Eds), Integrating science, technology, engineering, and mathematics (pp. 1–11). New York: Routledge.

Royal Society (2014). Vision for science and mathematics education. London: The Royal Society.

Steering Committee NLT (2008). Outline of a new subject in the sciences. A vision of an interdisciplinary subject ‘Nature, Life and Technology’. Enschede: SLO. Retrieved from http://betavak-nlt.nl/dmedia/media/site-files/08c51/cdf62/04332/627db/e26dd/Vision-outline-NLT.pdf.

Szybek, P. (2005). The Swedish Science Curriculum. In P. Nentwig & D. Waddington, D. (Eds), Making it relevant. Context based learning of science (pp. 195 - 210). Münster: Waxmann Verlag.

Van den Akker, J. (1999). Principles and Methods of Development Research. In J. van den Akker, R.M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds), Design approaches and tools in education and training (pp 1-14). Boston: Kluwer Academic.

Visser, T.C. (2012). Professional Development as a Strategy for Curriculum Implementation in Multidisciplinary Science Education. Enschede, Institute for Teacher Education, Science Communication & School Practices (ELAN), University of Twente. ISBN 978-90-365-3405-5

Wei, B. (2009). In search of meaningful integration: The experiences of developing integrated science curricula in junior secondary schools in China. International Journal of Science Education, 31, 259–277.

About the Authors

ix

Harrie Eijkelhof is an emeritus professor of physics education at Utrecht University. He was chairing the steering committee of the project Nature, Life and Technology (NLT) from 2006 - 2016. Currently he is involved in various curriculum, professional development and quality assurance programmes.

Berenice Michels studied meteorology and earned her teacher's degree in physics. She worked at the Netherlands Curriculum Institute (SLO) for 10 years. There, she was member of the NLT project group from 2006 - 2010 and thereafter project leader of the NLT project as well as secretary of the Steering Committee. Now, she is director of the U-Talent programme at Utrecht University (www.u-talent.nl)

Michels, B.I., Eijkelhof, H.M.C. (2019) The Development and Implementation of an Interdisciplinary STEM Course for Upper Secondary Schools. Educational Designer, 3(12).
Retrieved from: http://www.educationaldesigner.org/ed/volume3/issue12/article48/