Note: If you browse this site it will use temporary 'session cookies' to assist navigation and keep an anonymous record of page visits. More info…

Introduction

2

Any production of new materials for education must be driven by a desire to change. The scope of that change can span a spectrum from the radical, e.g to completely reorganise the structures and systems of schooling, to the almost trivial, to e.g. produce a more attractive textbook similar to that of publisher A to be marketed as a rival by publisher B. Thus, one of the questions which I want to address here is “How far can you go?” i.e. how far should the move towards innovation be tempered by practicality. The answer will require a review of the relevant constraints and affordances. The list of issues to be considered is formidable. Two of them, the power to attract resources to design and produce and disseminate, and the resource of expert people to do the work, I shall only consider marginally. Another, that of leverage, i.e capacity to attract publishers, to convince those who control teachers, the power to convince teachers to use the material, the likelihood that teachers may be able to use them to good effect (aka ‘the teacher-proof material’ syndrome), will be more salient in the discussion.

What I take to be obvious is that any new materials are but one component in a systemic reform, and the many features of the context in which the materials will be planted and nourished have to be appraised in the initial planning of the work. There are obvious examples of the risks of neglecting the context: the teaching approach required to use the materials may turn out to be too challenging for most teachers, or statutory requirements of high-stakes testing might not reward the aspects of learning which the materials were planned to promote.

In this paper, I shall develop a view that the strategic approach to design of educational material should focus on five main issues, namely research, curriculum (C), assessment (A) pedagogy (aka instruction) (P), and sustainability.

The close inter-interrelationship of three of these, C, A and P, requires a preliminary analysis. The curriculum, whether centrally prescribed, or implicit in the practice and culture represented in textbooks, is seen as a list of things to be taught, and which the teacher may ‘deliver’ by a pedagogy that he/she thinks appropriate. In most systems, assessment becomes the lens through which the curriculum is interpreted; this interpretation may depend on the appraisal of the aims of the curriculum that the test-system and its controllers adopt for their work. Such an interpretation may be benign, in that their instruments reflect and reward powers to understand and apply, or malign because it reflects a view of the curriculum as a collection of things to be known, with assessment recording mere recall, thereby constraining the pedagogy to ‘teach-to-the-test’, rather than to the original curriculum aims. The strategic choice for the designer of new materials is to anticipate how they will fit into the existing equilibrium between C, A, and P, or perhaps to seek to change this to create a more favourable environment.

I propose to explore this framework in the spirit of ‘notes towards a theory of ’ rather than in one of defence of a definitive theory. I shall pursue this exploration by discussing four cases of innovation, in which I have been personally involved, in order to show how they illustrate the differing priorities of the five issues. My main reason for choosing these examples is that the differences between them are such that they might illuminate the diverse complexity of the issues, and at the same time I know them intimately because I was involved in all of them.

Because this paper aims to consider the strategy rather than the details of design implementation, it will not discuss design at the level of the actual design of the various artefacts produced, such as printed pages. Whilst effective design at this level is a necessary condition for eventual quality, it is not a sufficient one, and it cannot redeem a design for which the strategy has serious flaws.

Research-based materials: developing primary school science.

3

This work started from a concern that this area was weak despite several past attempts to give it a strong base. An underlying reason was that development was trapped between two inadequate approaches (Black, 1980). One was to conceptualise primary science as a diluted version of high-school science, whilst the other was to propose activities which might engage young children in general reasoning skills, but developing thereby a content-free ‘process’ science. As Wastnedge put it ‘At present we must concern ourselves more with how children learn than with what they learn’ (2001 p.50).

To escape this trap, the vision was to design activities which would challenge children’s thinking and help them to develop some of the elementary insights which might serve as a basis for high-school science. To inform this vision, a research project was set up to explore the ideas about the natural world which children might already have. An example would be in the study of light, where children might first identify the various sources from which light comes, and then, through activities with torches (flashlights) and mirrors try, as they draw pictures of what is happening, to explore the different ways in which light may be represented on paper. The most ambitious of the targets was to help children realise that we see only by the effect in our eyes of the light which enters them from outside (Osborne et al.1993). The outcome was a set of data, evidence about the effectiveness of interventions based on them, and proposals for the elementary levels of conceptual development which might serve as a basis for the future learning of science (Black and Harlen, 1993).

On the basis of this work, funding and support from publishers was secured for writing and trialling new materials with selected teachers. Whilst the weakness of the science education of most primary teachers was a well-known problem, here, the nature of the activities and concepts was such that the new training in science which was needed was modest. The sole proviso was that when children raised unexpected questions which they were not equipped to answer, which would be bound to happen, teachers would have to admit this but perhaps use such questions as issues for discussion - e.g. asking “How could we test that idea?”.

The outcome was two sets of booklets, one for ages 5 to 7 and one for ages 8 to 11, with 11 for teachers and 11 for pupils in each set. At the time when these materials were being written, the future national curriculum was just coming over the horizon in England, and high stakes external tests were not established. The guides for teachers (Nuffield Primary Science 1993) were able to include links to the first version of the national curriculum for science, and second editions referred in more detail to the revised 1995 version. Built in to the activities set out in the teachers’ guides were suggestions for pupils to express their ideas in drawings and writing (Figure 1), and the guides contained samples of such work to help teachers judge such products in relation to the national curriculum levels of achievement. Thus, for example, the booklet on materials for 5-7 year olds included a page presenting a well-known children’s story about a ginger-bread house, at a simple reading level and with coloured pictures, but then asking children what would happen to this house if it were to rain. This was followed by a closer look at a real house (Figure 2).

When it was clear that the 1998 national curriculum was to be revised again, further development of the publications was halted, because of the uncertainty about the outcome. New statutory written tests, to which the materials were not aligned, were externally set and marked, the status of the assessments by teachers of any practical work was down-graded, and the addition, in the late 1990s, of a new stress on literacy and numeracy, with national prescriptions for teaching methods and testing goals for these topics, all served to undermine the priority given to science education (Harlen, 2007). Thus the innovation did not survive, and no comparable project has emerged to serve the changed context. Yet a recent review of materials for teachers about young pupils’ misconceptions in science, advised:

Probably the most valuable source is the Nuffield Primary Science Teachers’ guides (Dabell et al. 2006, p.137)

Inventing a new subject - Design and Technology in England’s national curriculum.

4

This field of study has been represented by a patchwork of subjects in the school curriculum, and its definition varies widely between different national systems (Black, 1998). An integrating rationale for the field was proposed by Black and Harrison (1985) who argued that technology was a field of practical action aimed at meeting human needs, and would require inputs from a variety of resources rather than those provided by any one school subject. The diversity of relevant subjects, such as home economics, craft design and technology, art and design, engineering studies, and so on, meant that this vision had implications for curriculum planning. Two projects then explored ways in which several models of cross-curricular work might be implemented in schools (Black et al. 1988; Murray et al. 1994).

The 1998 national curriculum designated Design and Technology as a single compulsory subject, but the inherent problems with this new subject are illustrated by the fact that it was not until 1995, after six years of confusion, that an acceptable specification for the national curriculum was settled (Barlex, 1998).

A detailed description and evaluation of the material was published by Givens and Barlex (2001): one of their findings was that only the most successful schools saw the need for, and made use of, the Student’s Study Guide. A most significant problem was that the four main constituent areas, namely Food, Textiles, Resistant Materials and Systems and Control, were usually taught by four different teachers, with pupils rotating between them in a carousel model, so that a coherent system of progression in the subject was hard to achieve despite the emphasis on these aspects in the Student’s Study Guide and the Teacher’s Guide.

Nuffield A-level Physics: systemic reform in a friendly environment.

5

The origin of this project lay in concerns, both in the U.K. and the U.S.A., about the state of science education (Jenkins, 1979), and about cold-war threats linked to the shortage of scientists (Rudolph, 2002). Such pressures encouraged the Nuffield Foundation to set up reform projects, first in the science courses for all secondary pupils, and later for the A-level sciences. (After age 16, students in England aiming at university entry spent a further two years in advanced A-level courses, in specialised study of only three subjects).

The aims of the course were outlined as follows :

The construction of a course is itself a piece of engineering, a job to be done despite inadequate knowledge of how some of the basic components in the learning process work. We have conceived the task as one of weaving topics together to make a connected story which makes sense and arouses interest in its own right, which has discernible themes and connects these in fruitful ways, and which at the same time serves these deeper aims of learning in the future, of understanding physics, of understanding how physics works, of learning to inquire oneself, and of seeing applied and social implications.
p.303, Black and Ogborn, 1970

This challenged the existing norms of both curriculum, pedagogy and examinations. The course was designed as a sequence of ten topics, each drawing on the work of its predecessors, with the final topic serving as a grand finale, drawing together many of the concepts developed in the earlier topics. This was physicists’ physics, reflecting, in its web of connections, one of the most compelling features of the subject. Other topics led in different directions, principally to explore technological and social effects of physics.

One key feature was the need to introduce topics not previously studied at A-level. This called for ‘pedagogic transformation’, i.e. ways to present ideas, previously considered too demanding at this level, to make them both authentic and intelligible. For example, a new way to teach the second law of thermodynamics involved consulting the different approaches in many sources (Ogborn, 1978), then setting up a new approach, requiring new methods and visual aids (Black & Ogborn, 1978; Black et al. 1971, 1972), then checking with several university physicists that the approach was authentic.

For the pedagogy, a core aim was that students should become independent learners. So the final publications did not include pupil textbooks, a feature which worried publishers and many teachers. There were detailed guides for teachers (Figure 7), and small booklets for students of background reading (Figure 6) and of questions (Figure 8).

Teachers were asked to provide a range of existing textbooks for students to consult, the aim being to help students to become less dependent on a sole textbook. Much of their learning was laboratory-based, with experiments designed to provoke discussion rather than to ‘prove’ established results. In one topic, different groups conducted one each of a suite of experiments, and then reported to a plenary in which all could draw together the full picture. All were required to do two small independent investigations, each taking two to three weeks, with the second being teacher-assessed to contribute about 10% of the A-level examination.

Both the integrity of the course and the interests of the students called for development of a new A-level examination. To validly reflect and support the aims of the course, a tailor-made set of six instruments was developed as follows :

• a multiple choice test;
• a test with a set of short, structured questions each calling for a set of short answers;
• a test offering a choice of essay-type questions;
• a passage about a topic not in the course, with questions assessing understanding of it;
• a practical test composed of about eight tasks using laboratory equipment;
• a report on an individual open-ended experimental project, assessed by the teacher.

Entry to university depended on the results of A-level examinations and acceptability of the results of this examination had to be secured in negotiations to establish that it would lead to a qualification which would be recognised and accepted beyond schools, notably by university admissions officers (Black & Ogborn, 1977).

There were numerous other aspects to the engineering, e.g involving schools who would trial the course, equipment manufacturers, publishers, teacher trainers, school district authorities, and the funding agency. One example will illustrate the complexity of the advanced planning required: trials of the new approaches in schools could not start until any novel equipment was available, so manufacturers had to make it – so they needed to see working prototypes a year ahead of the trials, which prototypes had first to be developed and tested: thus decisions about and development of such equipment had to be settled at a time when the course was no more than a set of aims with mere outlines of the material to be written.

The success of the course depended on meeting all of these the multiple requirements. However, a key feature of the context was the team’s freedom when the curriculum was not seen as an area for government control – the innovators were free to innovate, and succeeded or failed by their power to attract voluntary participation. Governments in the UK allowed more freedom for work at this level, in part because of their concerns about poor recruitment to those advanced studies which lead to qualifications in science and engineering.

The materials were revised and re-printed after a few years, and at this point it was decided, under pressure from publishers and schools, to produce a pupil textbook. The new materials had significant effects on the development of many conventional syllabuses and text-books. The course and its special examination continued for over 25 years until a quite new course, Advancing Physics (Ogborn, 2003) replaced it.

Formative assessment: materials to support only pedagogy

6

This particular piece might seem a surprising inclusion here. It did not produce either curriculum or assessment materials: it was solely about pedagogy. Yet by any standards it has been influential, producing a very widely cited research publication, two booklets for teachers — each of which has sold over 50 000 copies — and a book which has been a ‘best-seller’ according to both the publishers and Amazon. What makes it relevant here is this sharp contrast with the other examples.

The second phase of activity was to set up a project to tease out how the evidence might be used to change the pedagogy of teachers. Teachers in six schools, about 40 in all, accepted invitations to work with us in turning some of the ideas from the research into classroom practice. We emphasised that research could not provide recipes for teaching - it only provided ideas which might suggest new approaches that teachers might incorporate in their own daily work; in so doing they would be inventing new practical knowledge, not ‘applying’ research in any direct sense (Black & Wiliam, 2003). A further feature was that the project was to explore such invention within the context of the existing constraints of curriculum and summative testing in their schools, so that our concern was restricted to provoking and evaluating changes, and supporting the teachers - mainly by providing feedback from observing their teaching and also by providing regular project meetings, one every five weeks over two years, in which they could all meet and share experiences.

One outcome was that test scores, on the normal tests that were being used in their schools, were significantly higher that those of comparable control classes (Wiliam et al., 2004). Another was that the teachers expressed strong satisfaction with the changes in their own work. A third was our finding that such changes took time. Indeed, after one year of work in which the teachers had been happy to be involved, our classroom observations showed that little had changed. It was only in the next six months that the project ‘took off’ in that teachers were now teaching in different ways. We could then produce a second short booklet and a book for teachers (Black et al. 2002, 2003), in both of which we made extensive use of quotations from the written reflections of the teachers.

The core activities developed and promoted here were at the heart of the ways in which teachers interact with their students, whereby they elicit and then respond to students’ own thinking about ideas central to their learning. Such interactions could occur in oral dialogue, the minute by minute exchanges in a classroom, or in the more measured composing of guidance in responding to students’ written work. Both require a shift of orientation, from framing responses as grading or judging, to making them serve, and be seen by students to serve, as guidance to respect, yet challenge and improve, the learner’s thinking about the tasks that the activity had revealed. This could enrich the teaching of almost any curriculum, and is in principle quite separate from the long-term checking up of assessment of learning.

However, widespread dissemination has involved dilution to the extent of undermining the idea. Adoption has been superficial by many who do not grasp that behind the surface tactics lies a need for most teachers to re-think their beliefs about, and personal role, in students’ learning. The use of the title ‘Assessment for Learning’ as a simpler alternative to ‘Formative Assessment’ has not helped, and both in the USA and in recommendations of the government in England this title has been applied to regimes of frequent summative testing, which the original evidence for formative assessment does not support.

Reflections and conclusions

7

It is not easy to distil common themes from these examples because of their diversity. What I want to suggest here is a set of strategic questions which must be answered in the first stages of designing educational materials, using the four cases to illustrate a range of answers.

What research evidence is needed – and is it available ?

For Formative Assessment, the research evidence was the essential starting point, both by showing its potential to improve attainment, and in developing ways to realise this potential. Research also provided an essential basis for the Primary Science in helping to develop activities which would be both feasible and effective in securing its aims. Design and Technology called for a new configuration of curriculum topics, and whist some relevant research basis was being developed, the project had to move rapidly into practice because of national legislation. The A-level physics, being less radical and looking to evolution of existing good practice, did not need a new basis in research evidence.

Is the curriculum an established constraint – and is it likely to change?

This dimension was irrelevant for the Formative Assessment example. The A-level Physics was pushing the limits of existing curriculum norms, exploiting the curriculum flexibility allowed in its context to promote new aims. Such freedom was not enjoyed by Design and Technology, for whilst it successfully helped to define the new subject, its development was dogged by changes in the national curriculum specification. Primary Science established a new approach to the subject’s curriculum, in a context which initially allowed freedom, but changed later in ways which made it unviable.

Does summative assessment provide an opportunity or a straitjacket?

This issue was marginally relevant to the Formative Assessment, in that the evidence that it could lead to higher scores on existing tests helped make it attractive to teachers. Whilst the A-level Physics had freedom of manoeuvre to compose a summative assessment system that supported its aims, the Primary Science innovation did not formulate a summative assessment system, and when such a system was imposed, it became hard to sustain. For assessment of Design and Technology, the national requirements were not unhelpful, in that emphasis on teacher assessments of pupils’ design-and-make tasks was acceptable.

What style of classroom pedagogy is intended and how is it to be achieved ?

One aspect here is evident in Primary Science, which faced the obstacle that nearly all of the teachers lacked confidence in their knowledge of the subject, so that they felt unable to cope with the imaginative questions that their pupils might raise. Hence, the guides for teachers had to be supplemented by a book about the science itself, whilst explicit texts for pupils were also essential.

Design and Technology faced a similar problem in that being new, the collegiality and shared culture on which established subjects could draw did not exist: teachers of diverse experience had to forge a new identity. Here there was a clear need for materials for teachers to support them with guidance about the rationale of the course and about the way in which the materials reflected this rationale, with a unified approach across the various topics which would be taught by different teachers. But there was an equally strong need to have a rich set of materials speaking directly to students to complement this support. At the same time, because of the distinctive nature of the work, conventional textbooks would be inadequate. The combination of files of capability and resource tasks, with small books that provided ‘craft resources’ and background information, met the need.

The A-level Physics project also had problems, for it both presented novel topics and called for big changes in teaching practices. Many teachers who found the new content attractive showed, in trials, that they had not grasped that the new practices required radical changes in their teaching style. The publication of a pupils’ book for the two very new units was essential, yet the other eight demanded a new style and a new articulation of the unusual resources: the change to more conventional pupils’ books at the first revision may have been inevitable.

Pedagogy was the central concern of the Formative Project: promoting teacher change in this dimension was the focus of its work. It succeeded with its design challenge of achieving texts which combined brevity, clarity, practicality, and, through the use of teachers’ own writing, credibility.

Designing for sustainability?

It is inevitable that new educational materials may either become non-viable because of changes in the national context, or decline in popularity as they are replaced by later novelties. Their success may then be measured by the extent to which their ideas are incorporated in rival materials – imitation being the most sincere form of flattery.

Whilst it is hard to see how design can, or even should, make materials less fragile to such changes, it is likely that the survival of their most valuable features will eventually depend on these becoming part of the valued practice of teachers. They will achieve their common aim, of engaging pupils in successful learning, insofar as they can enhance the essential determinants of successful learning, the quality of the close interaction between teacher and pupils, and between pupils themselves, and thereby expand the capacity of the teacher to work creatively with the ways that pupils respond to the tasks and challenges set before them. My Formative Assessment example is particularly significant for it is only by working to challenge and so help develop these crucial features that teachers can promote learning in any innovation. So the question for the designer to face is: what types of teaching and learning activity do you envisage, what theories, of learning and of pedagogy, provide warrants for your choices, and in what ways do your products help such activity to happen?

Any set of materials must match and support the classroom work of teachers, but many under-estimate the difficulty of achieving teacher change, and it is in this dimension that the design of any materials becomes a central and subtle issue. One element which these four had in common was to take seriously fundamental aims in learning, aims to which none could object but few seemed committed to achieve in practice. It is a well documented finding that achieving teacher change is slow steady work (Schulman and Schulman, 2004) – to imagine that a short in-service course will be enough is usually a fantasy. The most subtle task of designers is to catalyse such change, within any materials, through work on the context in which they will be used, and through sustained support for the teachers who are translating their aims into practice.

References

8

Diagrams

9

© ISDDE 2008 - all rights reserved