Star wars

Performance ratings for social services
departments are set to be more transparent than ever thanks to Alan
Milburn’s proposals for star-ratings, and could see epartments
being pitted against each other in the battle for resources.
Frances Rickford reports.

Remember dunces? Teachers made them stand in
the corner with a tall hat marked with a D for getting their sums
or spellings wrong – at least, according to the picture books I
used to read. Now we don’t do that to children. Humiliating or
punishing people because their performance doesn’t reach our
expectations has gone out of fashion. It’s thought to be
demoralising and demotivating – counter-productive, in fact. But
doing it to organisations – particularly public sector
organisations – is all the rage. Not only do league tables of
schools, education authorities, hospitals and now social services
departments reveal to the world who is top and bottom of the class,
but the government is now proposing to heap more rewards on the
glorified and more penalties on the shamed.

A new star ratings system will award zero,
one, two or three stars to social services departments depending on
a formula bringing together their performance indicator scores with
data from Best Value, joint reviews and Social Services
Inspectorate reports. Social services departments which score
highly will have extra freedoms and extra money – their share of a
£50m “performance fund” – and those deemed poor performers
face a battery of possible interventions including the indignity of
being taken over by a neighbouring authority. Not surprisingly
social service managers are frantically asking themselves, in the
words of the gospel song, “Who will wear the starry crown? Oh Lord,
show me the way.”

Thirty departments got a foretaste last autumn
at the social services conference when Alan Milburn jumped the star
ratings gun by naming and praising or shaming departments on the
basis of performance indicators alone. This drew protests from the
Association of Directors of Social Services who pointed out that
the Department of Health has itself said that “the indicators can
only paint part of the picture and must be considered as part of a
broader set of performance information about social services” and
that “indicators only indicate; assessing the performance of a
council is complicated…”¹

The rationale for performance assessment,
according to Milburn, is that “the publicÉ have a right to
know how well those services are doing in comparison with
others.”

But his argument begs a string of questions.
First, and most obvious, who decides what a good service is?

Second, can something as complex as a social
service really be quantified, or even graded by external inspection
teams?

Third, if services can be measured by
indicators, are the current performance indicators the right
ones?

Fourth, are they robust enough to resist
manipulation, and are social services departments’ data collection
systems thorough and consistent enough to ensure like is being
compared with like?

Fifth, is the assessment framework measuring
things which social services managers can control, because if not
it is both unfair and unlikely to improve anything.

And sixth, can a regime as top-down as the
performance assessment framework invigorate and improve services
which rely fundamentally on the skill, energy and goodwill of
front-line staff?

Tony Hunter is director of social services in
the East Riding of Yorkshire and chairperson of the ADSS’s
standards and performance committee. Although furious about
Milburn’s own performance at Harrogate, he has no argument with the
principle of performance assessment. “We do support the development
of systems which enable the public to have an overall perception of
how a council is performing. That system should measure what really
matters to users and carers most, but you’ve got to start
somewhere. And a system which is useful overall is less likely to
be in place if social services directors are not involved in
developing it.”

But Hunter is acutely aware that the current
indicators are flawed, and is also conscious that front-line staff
in many departments need a lot of convincing that the system is
really about improving services for users. Front-line scepticism
about the usefulness of many of the indicators is endorsed by the
detailed work undertaken by the Social Services Research Group
(SSRG) which has published a 120-page, two-volume study of the
personal social services performance assessment framework and Audit
Commission performance indicators for social services. The SSRG is
an independent organisation with members from within local
government, health, and academia. It was actively involved in
developing the original PAF indicators four years ago, and still
works closely with the Department of Health. Last November, SSRG
chairperson David Allen wrote to social services chief inspector
Denise Platt to express the group’s concern about the current
direction and development of performance management of social
services. It warns that a number of the PAF indicators are seen by
the professional community as “weak or very questionable” and
expresses fears that things are set to get worse rather than
better. “Effort and resource in the performance management arena is
becoming increasingly fragmented and dispersed, and decisions about
future performance indicators are being made that are not being
informed by good practices on the ground.”

The question of who decides what is to be
measured is also highlighted by Peter Beresford, professor of
social policy at Brunel University and chairperson of the user
organisation network Shaping Our Lives. “On whose views are you
setting the criteria? What things are seen as important, what
weights get attached to different things, and who rates them? We
already know that there are significant differences between the
perceptions of managers and practitioners on the one hand and of
service users on the other about what works and what constitutes
good practice.”

Beresford points out that service-user
organisations are most concerned about the nature of interactions
between themselves and practitioners, the culture and values of an
organisation, and the issue of continuing cuts to services.

“You need to have safeguards, but the
evaluation has to be right. Doing it this way invites organisations
to cover their backs – to look for ways to subvert or manipulate
these bureaucratic standards and measures because if they don’t
they will be penalised.”

Ian Sinclair heads the social work research
and development unit at the University of York. He argues that the
performance assessment framework could have a positive influence in
acting as a counterbalance to social work’s inevitable focus on
individuals. “Performance indicators measure the general so they
could spread people’s focus more widely, which would be a good
thing. But there are many dangers including the risk that that they
will be allowed to interfere with professional judgment.” So for
example a child could be left in a placement in which they are
unhappy because the department is trying to improve its score on
placement changes.

Sinclair points out that the indicators are
not risk-adjusted, so for example the larger number of difficult
children your department is working with the worse your score is
likely to be. And he also suggests that many of the indicators
measure things which are largely outside of the control of the
directors whose heads will be on the block.

“For example, the indicator on long-term
foster placements depends on the quality of your foster carers, and
it is extremely difficult to influence that. They are supposed to
train foster carers but there is little or no evidence that
training does in fact improve the quality of foster carers.”

Even the quality of directly employed staff,
such as the heads of children’s homes, is very difficult for senior
managers to influence especially when, as now, there is a labour
shortage across the social care sector.

Other indicators measure factors which are
even further outside the influence of social service managers.
“They depend on things like the supply of educational psychologists
or the actions of the courts, or the performance of the health
service or local schools for example.

“The danger is that the star ratings system
will punish organisations for things beyond their control and
because they get a bad smell from their poor rating the situation
will be exacerbated. If this is combined with giving more resources
to the successful, it will be an extremely dangerous policy
prescription.”

¹ Department of Health,
Introduction to Social Services Performance Assessment
Framework Indicators, DoH, 2001