Terry Freedman, who worked on the assessment of ICT at the QCA, looks at the tricky area of summative assessment.
In
Making Judgements, we looked at Assessment for Learning (AfL)
techniques, and what they might mean in terms of ICT. AfL, as its name
suggests, is an approach designed to help the teacher help the pupil
learn more -- and learn better.
It is, to risk stating the obvious, assessment for learning.
One
of the benefits to the teacher of AfL is that she doesn't have to pin
down exactly what level a child is working at in order to help the
child progress. Indeed, research suggests that if the teacher awards a
grade for individual pieces of work, the pupil actually suffers as a
result, in terms of not benefiting as much as if a comment had been
made instead.
However, there comes a point where the teacher
really does need to make a judgement one way or the other as to the
level the pupil has reached in ICT. This is where summative, or
assessment of learning, comes in.
Summative assessment also has
another side to it: it is, or should be, an objective measure of
progress that means something to, and is trusted by, agencies that are
"outside" the school: ofsted, putative parents, and the DfES for
example.
That really does make life harder. ICT is a subject in
which there is still a dearth of subject experts, and even fewer
experts who agree with each other about levels of attainment of pupils.
The fact of the matter is that ICT is still quite a young subject and
so there is not (yet) a solid body of knowledge or wisdom concerning
pedagogy and standards. But it is starting to emerge, as we'll see in a
moment.
Add to this subject-specific difficulty a requirement
for objectivity that floors even the most experienced teacher in other
subjects, and you could be forgiven for believing that the task is
impossible.
The situation is not aided by the fact that the
Level Descriptions in the ICT Programme of Study have been written in
such a way that it's hard to pin down exactly what a level 5, say,
actually looks like. The general view is that the Level Descriptions
are "vague". However, the truth of the matter is that they were written
in such a way as to be future-proofed against advances in technology
and software. They are also intended to be used in a "best fit" kind of
way which relies on a teacher's professional judgement. That in itself
is problematic in ICT: there aren't that many teachers of the subject
who have the self-confidence to stand up and declare that they know how
to assess ICT properly!
Fortunately, though, QCA recognises
these problems and have come up with a couple of creative solutions.
The first is to encapsulate what the Level Descriptions mean in terms
of a few key characteristics, and the second is to develop a website
containing pupils' work which exemplifies different Levels.
Conveniently enough, you can find the former on the latter! Go to http://www.ncaction.org.uk/subjects/ict/progress.htm
and you'll come to the page called Progression in ICT. More of that in
a moment: for now, let's consider the NC Action website itself.
The
point of the site is to help teachers actually see what a particular
Level or, to be more accurate, what working at some aspects of a
particular Level, looks like. It's hard -- impossible in fact -- for
teachers to be able to say categorically that a pupil has reached Level
X when there is no objective form of assessment available -- or at
least, no generally accepted form of assessment (see the box on
validity and reliability also). The Level Descriptions describe what is
expected; the examples of work on the NC Action website show you.
The
work is organised very well. Each piece consists of several elements:
screenshots of the work, a description of what the pupils were asked to
do, a commentary and further analysis. You can search for work at
particular Levels, or particular key stages, or both.
When you
read the commentary for a piece of work, you'll see that it does not
say "This is a Level X", but "This illustrates some aspects of Level
X": these aspects are given in bold print. By the same token, the text
does not read "To get to Level Y the pupil should…", but "To make
further progress…". The reason for this style of writing is to avoid
misleading teachers into thinking, say, that if the pupil were to look
at 6 websites instead of 3 she has reached a higher Level. It also
recognises that, in real life, pupils' work does show elements of
different Levels. The idea, as stated earlier, is to achieve "best fit".
The
progression document, which lists the key characteristics of each Level
Description, with examples, is a life saver. Take Level 6, for example,
where the key characteristics are described as follows:
Level
6 is characterised by increased integration and efficiency in the use
of ICT tools. A greater range and complexity of information is
considered.
This means that when you are assessing a pupil's
ICT capability (over a range of work rather than just one item), you
can ask yourself whether integration and efficiency are in evidence. If
the answer is no, then they are not demonstrating Level "six-ness". You
can then see if they are demonstrating Level "five-ness" through the
same process. This is much easier and more accurate than trying to
match the pupil's work to the original Level Descriptions!
A
word of caution though: do read the examples, and all of the notes on
the website to do with assessing ICT, because it's very easy for
teachers to misinterpret what is meant. Take the concept of
"efficiency" for example. Some teachers, understandably, assumed it
meant using keyboard shortcuts or touch typing. It doesn't. But to find
out what it really means, you'll have to look it up for yourself!
Top tip #1: Validity and reliability
To
be certain that a test (or any other form of assessment) -- in any
subject --actually measures what it purports to measure, it has to be
both valid and reliable.
There are several types of validity, but
the most important one, at least initially, is face validity. That is,
it must at least look as though it measures what it says it measures,
or will not have any credibility.
A test can be said to be reliable if it produces consistent results between groups of similar pupils.
If
you are thinking about buying or creating your own objective assessment
of ICT, you should take these factors into account. On the subject of
validity, for example, is the test really measuring a pupil's ICT
capability, or her reading skills?)
Bear in mind also that the
commercially available tests that have been developed will almost
certainly have been trialled with teachers. That sounds good, until you
realise that most teachers award a higher Level than is really the case.
Obviously,
this may lead you to ask, quite reasonably, who has decreed what
constitutes working at a particular Level, if there is no objective
assessment available? The answer is the QCA. And before you protest
that your interpretation of the Level Descriptions is much more
accurate or robust, for whatever reason, you need to take into account
the fact that OFSTED, BECTa and the Key Stage 3 ICT Strategy all base
their assessment-related work on the QCA's interpretation. There are
ongoing discussions between those groups and practitioners, which
results in the guidance being clarified as time goes by, but QCA is the
ultimate arbiter. (For this reason, LEAs who have their own exemplar
sites and moderation procedures do need to ensure that they use the QCA
examples (see main text) as a baseline for their work, rather than
simply ignore it.)
Getting back to the tests, they may be very
helpful in suggesting what Level a pupil is working at, but you will
still need to exercise your own judgement, and come up with other ways
of assessing the pupils' ICT capability (see box called Better
Assessment). Even the test which the QCA is developing, which has gone
through very rigorous procedures to ensure validity and reliability,
and the correct evaluation of the contribution to the final outcome of
the pupils' mastery of skills and techniques, is not expected to
replace teacher assessment.
Top tip#2: Better Assessment
Beware
the one-off assessment in the form of a written test, or a
computer-based exercise. To be worthwhile, the assessment should take
place over time, and should not impose a glass ceiling on what the
pupil can show he knows, understand and can do.
The best way to
achieve this, especially with older pupils such as at Key Stage 3, is
to run projects. To be successful, these projects should:
* Be interesting
* Run for several weeks
* Involve pupils working independently, but with help available should they need it
Involve
the pupils documenting what they have done, and saving versions of
their work as they progress: taken together, these create an audit
trail to aid you later on, or to help you justify your decisions; it
also provides a record of who did what if pupils have to share
computers.
A prerequisite is to make sure that the pupils are
capable of working independently. This has to be built up gradually.
For example, start them off by having single lessons of working
independently (micro projects), then two or three lessons (min
projects).
It's important to realise that pupils don't learn how
to work independently simply by doing it. You need to teach them how to
organise their work and their time, and how to document it. You can
provide templates for these activities. You will almost certainly need
to take them through the process as a kind of worked example.
Finally,
some of the help available should be in the form of technical help on
using the software, because pupils shouldn't be prevented from doing
what they want to do just because they don't know how to do it in a
particular application. Teaching pupils how to use the Help in
applications, and providing crib sheets and posters on how to do the
basics in various applications, are fundamental to the success of
project-based assessment.
Of course, no matter how well you run
this sort of activity, you should never assess the pupil on the basis
of one item of work, but on a range of activities throughout the Key
Stage.
See also the articles on record keeping.
A variation of this article was first published in InteracTive in 2003.
What do you
think? Please leave a comment.
© Terry Freedman Thu, 19 Feb 2004
Comments are moderated.
If you found this article useful,
share it with a colleague via email. You can also share it on other websites using the "Share or Retweet" button below
Subscribe to article summaries
Share or Retweet
"educational technology"
"ICT in Education"