How can we be sure?

October 3, 2005 in Community Care

About 500 Sure Start programmes have been set up in recent years
in the most deprived areas of the country to transform the life
chances of younger children through improved support. The gains are
being measured as part of a longitudinal study by the National
Evaluation of Sure Start but it will be some years before the full
picture is known.

However, programmes are required to measure their performance
through local evaluation and monitoring. They are coming under
pressure to show outcomes of the innovative services they have
developed as well as progress towards national and local targets.
Often it appears to be difficult for local programmes to quantify
outcomes, impacts and benefits to those families involved with
services through the collection and analysis of data which show
change.

It is difficult to evaluate efforts to tackle complex social
problems that have multiple causes. The more an intervention tries
to tackle the root of the problems and build lasting solutions from
the bottom up, the longer it takes for any measurable impact to be
felt – and the more difficult it is to tell whether any
changes are a result of the original scheme or other factors.

This has been problematic in many cases. One professional in a
local programme in the North West said of its pop-in service: “It
is difficult with human behaviour to know what intervention has had
what outcome. Is it because someone smiled at you as you walked
down the road? It is difficult to isolate what the pop-in is doing
for families. All we can do is constantly evaluate our approach and
talk to families who attend the pop-in about what it means to
them.”

Although quantitative data analysis may be necessary to confirm
large-scale change, the value of individual and personal accounts
of change can be valuable. Such accounts can offer local
programmes, evaluators and policymakers deep insights into the
impacts that local programmes have had on individuals and
consequently inform research practice and policy.

Since its establishment, Sure Start has boasted a strong
“consumer focus”. That is to say, there is an expectation that all
local programmes will involve parents in the design, management and
evaluation of the programme.

One mother said of her involvement with Sure Start Blacon in
Chester: “In 2000 I gave birth to my third child. My health visitor
told me about a government initiative which a group of professional
bodies and parents were putting together, and asked if I would like
to join the partnership board. I nervously agreed and many months
and meetings later everything was finally agreed and we set up Sure
Start Blacon. I was a key member in the planning process and was on
the interview panels for the majority of the Sure Start team. It
worried me that someone would come in without the experience of
working with children and without knowing Blacon and set up
services. I wanted to make sure the programme worked for local
people.

“Within the four years that I was a member of the partnership
board I spent two of them as vice-chair and chair of the executive
group, where I have worked alongside many professional agencies,
including social services, NCH, West Cheshire primary care trust
and Chester Council.

“During my time on the partnership board I also decided to
become a volunteer as I wanted to help more than just sit in
meetings. As a volunteer I underwent a range of training and
awareness days, including post-natal depression, domestic abuse,
disability awareness and basic sign language. I also helped in the
planning stages of the teenage pregnancy strategy. In December 2004
I started work at Sure Start Blacon as an employed member of the
team as speech and language therapy assistant.

“Sure Start Blacon has made me more confident and more
determined to have a voice in my community because I have always
felt valued and respected. Whenever I talked I was listened to. As
a parent I was thought to be equal to the professionals. I
wasn’t just the ‘token’ parent to make up the
numbers. I was a real local parent with first-hand experience of
being with children and living within the community, and I had real
local concerns.

“I was offered guidance yet not patronised, supported yet not
dictated to. This helped me to become an outspoken member of the
community and gave me the power to get involved and try new things.
People think Sure Start is just for children, but Sure Start Blacon
has changed my life, improved my family’s quality of life and
I hope my continued involvement can help the programme help many
other local families.”

This extract from a personal account of how a local programme
has affected the life of this mother and her family shows the value
of listening to individuals’ narratives when evaluating Sure
Start local programmes. While policymakers may favour hard data in
establishing levels of local success, it may be difficult for local
programmes to produce such figures.

In part this appears to be due to the difficulties programmes
experience in collecting, storing and analysing data on individuals
with what are often complex social problems. By collecting
information on these individuals, professionals within local
programmes are often aware of the paradox inherent in Sure Start
services – data are required to evaluate the effectiveness of
Sure Start and enhance future service and policy development
– yet the act of data collection may deter some vulnerable
and particularly “hard-to-reach” residents from engaging with the
service.

This emphasises the value of using qualitative methods in the
evaluation of Sure Start. Sure Start programmes are dealing with
complex communities with complex problems. Just as each Sure Start
area is unique, so is each individual’s denotation of their
situation and their Sure Start experience. Therefore it is timely
that an approach to conceptualising these very personal experiences
be used to show impact and change. Furthermore, it may be possible
for qualitative accounts of the changes which have occurred for
people to be used to begin to define quantitative measures, against
which programmes can measure their future success.

Training and Learning

The author has provided questions about this article to guide
discussion in teams. These can be viewed at www.communitycare.co.uk/prtl
and individuals’ learning from the discussion can be
registered on a free, password-protected training log held on the
site. This is a service from Community Care for all GSCC-registered
professionals.

Abstract

This article looks at the issue of measuring the impact of Sure
Start programmes on family life. It uses a case study from Sure
Start Blacon in Chester to illuminate the value of personal
narratives in understanding the impacts of local programmes on
families. This shows the importance of qualitative personal
narratives in evaluating a complex intervention such as Sure
Start.

Further Information