There is “little evidence” that the Signs of Safety (SofS) model – used in some form by two-thirds of English councils – leads to better social work practice or reduced risks to children.
That was the verdict of a government-commissioned evaluation of the ten authorities funded through round 2 of the Department for Education innovation programme to implement the approach with support from consultancy Munro Turnell & Murphy (MTM).
One of the researchers – the head of What Works for Children’s Social Care – said the findings should give councils considering adopting SofS pause for thought, however MTM said they reflected problems of implementation, rather than with the model itself.
The evaluation found that there was:
- No evidence from a staff survey or analysis of national data to suggest SofS had resulted in improvement in staff wellbeing or retention. Around half of social workers in the four pilots surveyed thought turnover was a problem in their authority.
- No evidence from case file reviews that a more detailed application of SofS led to more thorough assessments.
- No moderate or high-strength evidence that SofS decreased the probability of a child being re-referred within six months, or on the impact of SofS on the probability of a child being re-referred and their case escalating.
- Moderate strength that SofS reduced the probability of kinship care, compared with non-kinship care, contrary to the aims of the programme.
- No apparent impact from using SofS on the numbers of children in need or in care.While pilot sites had fewer children in need, on child protection plans and in care than similar authorities, this did not change over time.
- No difference between two of the pilots and two comparable non-SofS authorities from observations of practice in any of the indicators used.
‘Little evidence’ of impact
Researchers also pointed out that, of the eight authorities that took part in this pilot and a previous trial funded through round 1 of the the innovation programme, just two were rated ‘good’ or ‘outstanding’ by Ofsted by the end of 2019.
Of the model used in full by one-third of English councils and in part by another third, they concluded: “Adopting SofS may contribute to strengthening an agency, but it is just one part of what is required to improve outcomes for children, young people and their families. It may lead to more consistent recording of cases but there is no evidence that it leads to consistent and improved practice…
“In summary, we found little evidence to support the claim that SofS leads to better practice or reduced risk for children.”
What is Signs of Safety?
The evaluation report says that Signs of Safety is an approach to child protection underpinned by a commitment to work collaboratively with families and children, and to conduct risk assessments and draw up safety plans that focus on families’ strengths, networks and resources. These are based on an assessment of what is working well, what services are worried about and what needs to happen.
However, there was a lack of consensus across the pilots about the nature of Signs of Safety. Some viewed it as a practice framework, and others as part of a wider framework that may encompass different approaches, such as reflective, systemic or trauma-informed practice. Some practitioners viewed it as a value system and a way of working differently with families, others as an assessment tool and a third group as a tick-box exercise for navigating recording systems.
The lead researcher for the evaluation, King’s College London’s Mary Baginsky, stressed on Twitter that the study was not a verdict on SofS in all circumstances and that most of the data from the evaluation had come from just five of the pilots.
The chief executive of What Works for Children’s Social Care, Michael Sanders, who also took part in the research, said that the study did not find “very substantial harms” associated with SofS, and that it was “probably safe” for authorities using the model to continue.
However, he added that “anyone thinking of starting should probably pause for thought”.
‘Challenges of implementation’
MTM – which consists of former government children’s services adviser Professor Eileen Munro, SofS co-founder Andrew Turnell and Terry Murphy, who led SofS’s implementation in Western Australia in the 2000s – admitted that “the outcomes we
set ourselves — transformation of the organisations and their child protection case results — were not matched by our interventions”.
However, it said that the study and its own research, published last year, showed that “Signs of Safety was only partially and to varying degrees implemented in the ten local authorities and so the studies tell us about the challenges of implementation not the impact on families of receiving a Signs of Safety service”.
MTM said that the evidence showed that results depended on “leadership and the extent to which they aligned organisational systems with the practice”.
It added: “MTM takes this as a sobering reminder that a poor implementation of Signs of Safety could make things worse. This should be no surprise as introducing another set of demands into a busy and complex environment without clear commitment and making it fit and thus allowing space for the work, will produce as much contention as traction.”
How was the research conducted?
The evaluation sought to examine three issues:
- how pilots had implemented SofS, and how faithfully they had followed the model, in relation to aligning their quality assurance systems with SofS practice and using safety planning at all stages of families’ contact with social workers;
- whether it had improved outcomes for families, including the impact of SofS on the likelihood of families being re-referred, the use of kinship care, the quality of assessments and job satisfaction for social workers;
- the costs of implementation and the cost-saving implications of changes in outcomes.
The fieldwork took place from April 2018 to March 2020, and included a staff survey, an examination of assessments, a contrast study examining practice in two SofS and two non-SofS children’s service departments, an examination of implementation plans and profiles, and an analysis of national outcomes data, including an analysis of how these have changed in SofS sites compared with similar authorities that had had not implemented SofS.
While some data was conducted from all nine pilots that completed the pilot (one council dropped out during the process), most was collected from five authorities.
‘No reason to stop using Signs of Safety’
In its response to the research, the DfE said there was no reason for councils to stop using SofS.
“When used alongside other elements of social work practice and tools, the Signs of Safety tool has proven to benefit councils, including bringing together agencies that work to protect vulnerable children, and there is no evidence that councils should stop using it,” a spokesperson said.
Association of Directors of Children’s Services president Jenny Coles said: “As the evaluation report states, of the local authorities piloted, the implementation of Signs of Safety varied markedly and many local authorities across the country are adapting it to meet their own needs or local context, allowing for improvement as they each learn what works best.
“Measuring the value and impact of these approaches is rarely easy but where this is done well both practitioners and families tell us this works for them. However, we also know that the effectiveness of any practice model depends on organisational values, leadership and the skills and confidence of practitioners.
“It is important that we continue to explore and evaluate ways of supporting children and families and to give local authorities the financial means to appropriately adapt these methods to meet local needs and context.”