‘Concerning’ question marks over Signs of Safety effectiveness, review finds

Research from children's social care What Works Centre argues practice model's impact on reducing care admissions is unproven and that evidence base needs 'urgently developing'

files

There is “no evidence” a popular practice model reduces the numbers of children entering care, a new study has found.

The systematic review was one of two research pieces published this week by the government’s What Works Centre for children’s social care. It found “concerning” question marks over whether and how Signs of Safety’s impact, including its cost-effectiveness, could be measured.

The approach, which originated in Australia and focuses on partnership building and on families’ strengths, “implicitly and explicitly” aims to reduce numbers of children in care and has been enthusiastically embraced in the UK. It was the subject of a broadly positive 2017 Department for Education (DfE) evaluation of 10 pilot authorities.

The new study report hedged its bets, saying that “lack of basic evidence in relation to Signs of Safety does not mean we should conclude it does not work”.

But, it concluded: “Robust evaluations based on a clearly specified intervention theory are needed to adequately assess whether Signs of Safety can achieve its outcomes when delivered well.”

‘Troubling’ possibilities

The systematic review was carried out by academics at the What Works Centre’s research partner, Cardiff University’s CASCADE children’s social care centre.

It analysed 38 previous publications relating to Signs of Safety, including qualitative and quantitative academic studies, some of them from outside the UK and of limited relevance.

Also included in the total were examples of ‘grey literature’ such as local and national government reports.

Researchers aimed to investigate “whether, how, for whom and under what conditions Signs of Safety works to safely reduce the number of children entering and re-entering care, and to increase the number of children re-unified with their family”.

They found only one paper, from 2009, which had directly showed a lower rate of entering care among children receiving Signs of Safety interventions than those who did not.

Even then, the study warned that “considerable caution” should be exercised regarding that earlier research because Signs of Safety was “only part of a long and complicated set of changes within a whole system”.

The systematic review identified three potential reasons as to why the hypothesis that Signs of Safety reduces the need for children to be in care appeared unproven:

  • That it is effective but has been poorly evaluated thus far
  • That it is not being delivered well, and therefore it is hard to know whether it works
  • That it in fact has no impact.

The paper described the first two as “plausible” and the latter “troubling”, noting that it was possible that families’ tough situations could limit the ability of “skilled workers using strengths-based approaches” to make a difference.

Implementation questions

While arguing for assessment of a “well-implemented” Signs of Safety case study in order to grow the evidence base, the study acknowledged some inherent challenges in doing so.

“Evidence relating to implementation, primarily from evaluations, grey literature and qualitative research, demonstrates the complexities of putting Signs of Safety into practice for an organisation,” it said. “Signs of Safety is not a clearly defined intervention, and this leads to difficulties in assessing the relative success of implementation.”

A project at Wigan council, one of the What Works Centre’s partner authorities, is examining the challenges of putting Signs of Safety in place – including how impetus can slow as novelty wears off – and the systematic review identified a number of barriers employers face.

These include different levels of training, which can be affected by shrinking budgets, incompatible IT systems, having to change assessment processes and simply instigating a learning culture.

“Given the complexity of moving to a Signs of Safety approach, it is not surprising that it may take some time to embed,” the paper said. “Yet there is no evidence to suggest [its] impact increases over time.”

Questions need answering, the study concluded, as to what a “good” implementation of Signs of Safety looks like and the extent to which its principles and practices can be sustained once they have been adopted by an employer.

Scoping review

As well as the Signs of Safety study, the What Works Centre issued a broader scoping review of evidence around practice that safely reduces numbers of children and young people in care.

The research grouped results into nine key intervention activity areas:

  • Family/child skills training
  • Service integration or coordination
  • Worker practice changes
  • Changes of therapeutic approach
  • Child welfare system changes
  • Meetings involving family members
  • Interventions that affect families’ finances
  • Mentoring
  • Social worker supervision

The centre is now due to consult on the findings within the children’s social care sector via policy and practitioner panels and special events, with families and frontline practitioners among those being asked to contribute.

The What Works Centre’s chair, Sir Alan Wood said the new studies “represent an important step in the centre’s development as well as important contributions to discussions about the evidence that exists for children’s social care practice and the strength and weaknesses of this evidence”.

He added: “Given these identified gaps in the evidence base, we’re more like the ‘don’t know as yet What Works Centre’ but the picture will become clearer as our research begins in earnest next year.”

More from Community Care

6 Responses to ‘Concerning’ question marks over Signs of Safety effectiveness, review finds

  1. Mick November 15, 2018 at 11:47 am #

    It’s no reflection at all on Signs of Safety – but David Cameron’s DfE early on had this terrible tendency to buy in whatever mumbo-jumbo they could find overseas, often licensed from private-sector institutes who had produced their own evidence bases, and then present it as a magic bullet. Even if all of them got great results when applied in the UK, it’s a bad way to go about things. But going forward I’m not even sure we’ll have the luxury of basing everything on evidence: what if the problems are evolving faster than the evidence can be collected?

  2. N November 16, 2018 at 12:49 am #

    Why is this a surprise? Signs of Safety and other models (Strengthening Families Framework, Systemic Social Work, etc) are, at best, cheat sheets. They are a tool that good social workers can use to help streamline their work but they have to know how to do the job in the first place.

    For some reason Local Authorities think that they constitute ‘good social work’ in and of themselves like some sort of magic pill that you can mix with an overstretched workforce and under-trained staff and Just Fix Things.

    Getting a service to use a particular model does nothing more than give mediocre social workers something to say when Ofsted turns up and maybe standardise recording techniques.

  3. Louise November 19, 2018 at 10:04 am #

    Finally some research that says what practitioners have being saying about SOS. It is just good social work practice repackaged in different language. Andrew turnells retirement fund is not looking so good now.

  4. Anon November 19, 2018 at 12:13 pm #

    It was simply rubbish, just a paper exercise to show that local authority is doing the job but in reality it was achieving nothing.
    Such a waste of tax payers money – government seriously need to reconsider the rationale of this service/disservice.

  5. Sam November 21, 2018 at 11:42 am #

    Signs of Safety – like other training based organisational interventions – is ineffective unless the organisational system is looked at as a whole.

    What matters to the people the organisation exists to help? What things get in the way of the organisation and it’s staff doing what matters? And how do you remove those things in a systemic way?

  6. Nora November 21, 2018 at 8:33 pm #

    Interesting comments added to those generated today at a round table discussion of a recent research based article on signs of safety ‘s effectiveness (Reekers et al 2018) . The research found that SoS was as good as the usual practice in reducing risk of child abuse and parental empowerment was associated with this. Most in the discussion had been trained to use a version of SoS. Some were proponents, others sceptics. The critical research left some frustrated and several comments were about methodological limitations. Others were relieved that at least SoS does not make things worse. Some said that perhaps the reasons why social workers liked SoS (the evidence supports this) was because it gave them a shared language and a way to reduce complexity , maybe that it gave more confidence in uncertainty. Is this good for social work and children, carers and adult service users? To make the complex simple, to scale (and potentially set) social problems? How people understand and use SoS is largely unknown. If service users asked – what’s the evidence for the approach you use to support me ? What would we say?