There is “no evidence” a popular practice model reduces the numbers of children entering care, a new study has found.
The systematic review was one of two research pieces published this week by the government’s What Works Centre for children’s social care. It found “concerning” question marks over whether and how Signs of Safety’s impact, including its cost-effectiveness, could be measured.
The approach, which originated in Australia and focuses on partnership building and on families’ strengths, “implicitly and explicitly” aims to reduce numbers of children in care and has been enthusiastically embraced in the UK. It was the subject of a broadly positive 2017 Department for Education (DfE) evaluation of 10 pilot authorities.
The new study report hedged its bets, saying that “lack of basic evidence in relation to Signs of Safety does not mean we should conclude it does not work”.
But, it concluded: “Robust evaluations based on a clearly specified intervention theory are needed to adequately assess whether Signs of Safety can achieve its outcomes when delivered well.”
The systematic review was carried out by academics at the What Works Centre’s research partner, Cardiff University’s CASCADE children’s social care centre.
It analysed 38 previous publications relating to Signs of Safety, including qualitative and quantitative academic studies, some of them from outside the UK and of limited relevance.
Also included in the total were examples of ‘grey literature’ such as local and national government reports.
Researchers aimed to investigate “whether, how, for whom and under what conditions Signs of Safety works to safely reduce the number of children entering and re-entering care, and to increase the number of children re-unified with their family”.
They found only one paper, from 2009, which had directly showed a lower rate of entering care among children receiving Signs of Safety interventions than those who did not.
Even then, the study warned that “considerable caution” should be exercised regarding that earlier research because Signs of Safety was “only part of a long and complicated set of changes within a whole system”.
The systematic review identified three potential reasons as to why the hypothesis that Signs of Safety reduces the need for children to be in care appeared unproven:
- That it is effective but has been poorly evaluated thus far
- That it is not being delivered well, and therefore it is hard to know whether it works
- That it in fact has no impact.
The paper described the first two as “plausible” and the latter “troubling”, noting that it was possible that families’ tough situations could limit the ability of “skilled workers using strengths-based approaches” to make a difference.
While arguing for assessment of a “well-implemented” Signs of Safety case study in order to grow the evidence base, the study acknowledged some inherent challenges in doing so.
“Evidence relating to implementation, primarily from evaluations, grey literature and qualitative research, demonstrates the complexities of putting Signs of Safety into practice for an organisation,” it said. “Signs of Safety is not a clearly defined intervention, and this leads to difficulties in assessing the relative success of implementation.”
A project at Wigan council, one of the What Works Centre’s partner authorities, is examining the challenges of putting Signs of Safety in place – including how impetus can slow as novelty wears off – and the systematic review identified a number of barriers employers face.
These include different levels of training, which can be affected by shrinking budgets, incompatible IT systems, having to change assessment processes and simply instigating a learning culture.
“Given the complexity of moving to a Signs of Safety approach, it is not surprising that it may take some time to embed,” the paper said. “Yet there is no evidence to suggest [its] impact increases over time.”
Questions need answering, the study concluded, as to what a “good” implementation of Signs of Safety looks like and the extent to which its principles and practices can be sustained once they have been adopted by an employer.
As well as the Signs of Safety study, the What Works Centre issued a broader scoping review of evidence around practice that safely reduces numbers of children and young people in care.
The research grouped results into nine key intervention activity areas:
- Family/child skills training
- Service integration or coordination
- Worker practice changes
- Changes of therapeutic approach
- Child welfare system changes
- Meetings involving family members
- Interventions that affect families’ finances
- Social worker supervision
The centre is now due to consult on the findings within the children’s social care sector via policy and practitioner panels and special events, with families and frontline practitioners among those being asked to contribute.
The What Works Centre’s chair, Sir Alan Wood said the new studies “represent an important step in the centre’s development as well as important contributions to discussions about the evidence that exists for children’s social care practice and the strength and weaknesses of this evidence”.
He added: “Given these identified gaps in the evidence base, we’re more like the ‘don’t know as yet What Works Centre’ but the picture will become clearer as our research begins in earnest next year.”