‘Little evidence’ Signs of Safety improves practice or children’s outcomes, finds pilot evaluation

What Works chief says local authorities considering adopting practice model used by two-thirds of councils should pause for thought, however consultancy behind model say evaluation results reflect implementation problems

Description_of_image_used_in_if telecare_doesn't_work_then_why_are_councils_still_using_it_magnifying_glass_over_word_evidence
Photo: Fotolia/aquarious83men

There is “little evidence” that the Signs of Safety (SofS) model – used in some form by two-thirds of English councils – leads to better social work practice or reduced risks to children.

That was the verdict of a government-commissioned evaluation of the ten authorities funded through round 2 of the Department for Education innovation programme to implement the approach with support from consultancy Munro Turnell & Murphy (MTM).

One of the researchers – the head of What Works for Children’s Social Care – said the findings should give councils considering adopting SofS pause for thought, however MTM said they reflected problems of implementation, rather than with the model itself.

The evaluation found that there was:

  1. No evidence from a staff survey or analysis of national data to suggest SofS had resulted in improvement in staff wellbeing or retention. Around half of social workers in the four pilots surveyed thought turnover was a problem in their authority.
  2. No evidence from case file reviews that a more detailed application of SofS led to more thorough assessments.
  3. No moderate or high-strength evidence that SofS decreased the probability of a child being re-referred within six months, or on the impact of SofS on the probability of a child being re-referred and their case escalating.
  4. Moderate strength that SofS reduced the probability of kinship care, compared with non-kinship care, contrary to the aims of the programme.
  5. No apparent impact from using SofS on the numbers of children in need or in care.While pilot sites had fewer children in need, on child protection plans and in care than similar authorities, this did not change over time.
  6. No difference between two of the pilots and two comparable non-SofS authorities from observations of practice in any of the indicators used.

‘Little evidence’ of impact

Researchers also pointed out that, of the eight authorities that took part in this pilot and a previous trial funded through round 1 of the the innovation programme, just two were rated ‘good’ or ‘outstanding’ by Ofsted by the end of 2019.

Of the model used in full by one-third of English councils and in part by another third, they concluded: “Adopting SofS may contribute to strengthening an agency, but it is just one part of what is required to improve outcomes for children, young people and their families. It may lead to more consistent recording of cases but there is no evidence that it leads to consistent and improved practice…

“In summary, we found little evidence to support the claim that SofS leads to better practice or reduced risk for children.”

What is Signs of Safety?

The evaluation report says that Signs of Safety is an approach to child protection underpinned by a commitment to work collaboratively with families and children, and to conduct risk assessments and draw up safety plans that focus on families’ strengths, networks and resources. These are based on an assessment of what is working well, what services are worried about and what needs to happen.

However, there was a lack of consensus across the pilots about the nature of Signs of Safety. Some viewed it as a practice framework, and others as part of a wider framework that may encompass different approaches, such as reflective, systemic or trauma-informed practice. Some practitioners viewed it as a value system and a way of working differently with families, others as an assessment tool and a third group as a tick-box exercise for navigating recording systems.

The lead researcher for the evaluation, King’s College London’s Mary Baginsky, stressed on Twitter that the study was not a verdict on SofS in all circumstances and that most of the data from the evaluation had come from just five of the pilots.

The chief executive of What Works for Children’s Social Care, Michael Sanders, who also took part in the research, said that the study did not find “very substantial harms” associated with SofS, and that it was “probably safe” for authorities using the model to continue.

However, he added that “anyone thinking of starting should probably pause for thought”.

‘Challenges of implementation’

MTM – which consists of former government children’s services adviser Professor Eileen Munro, SofS co-founder Andrew Turnell and Terry Murphy, who led SofS’s implementation in Western Australia in the 2000s – admitted that “the outcomes we
set ourselves — transformation of the organisations and their child protection case results — were not matched by our interventions”.

However, it said that the study and its own research, published last year, showed that “Signs of Safety was only partially and to varying degrees implemented in the ten local authorities and so the studies tell us about the challenges of implementation not the impact on families of receiving a Signs of Safety service”.

MTM said that the evidence showed that results depended on “leadership and the extent to which they aligned organisational systems with the practice”.

It added: “MTM takes this as a sobering reminder that a poor implementation of Signs of Safety could make things worse. This should be no surprise as introducing another set of demands into a busy and complex environment without clear commitment and making it fit and thus allowing space for the work, will produce as much contention as traction.”

How was the research conducted?

The evaluation sought to examine three issues:

  • how pilots had implemented SofS, and how faithfully they had followed the model, in relation to aligning their quality assurance systems with SofS practice and using safety planning at all stages of families’ contact with social workers;
  • whether it had improved outcomes for families, including the impact of SofS on the likelihood of families being re-referred, the use of kinship care, the quality of assessments and job satisfaction for social workers;
  • the costs of implementation and the cost-saving implications of changes in outcomes.

The fieldwork took place from April 2018 to March 2020, and included a staff survey, an examination of assessments, a contrast study examining practice in two SofS and two non-SofS children’s service departments, an examination of implementation plans and profiles, and an analysis of national outcomes data, including an analysis of how these have changed in SofS sites compared with similar authorities that had had not implemented SofS.

While some data was conducted from all nine pilots that completed the pilot (one council dropped out during the process), most was collected from five authorities.

‘No reason to stop using Signs of Safety’

In its response to the research, the DfE said there was no reason for councils to stop using SofS.

“When used alongside other elements of social work practice and tools, the Signs of Safety tool has proven to benefit councils, including bringing together agencies that work to protect vulnerable children, and there is no evidence that councils should stop using it,” a spokesperson said.

Association of Directors of Children’s Services president Jenny Coles said: “As the evaluation report states, of the local authorities piloted, the implementation of Signs of Safety varied markedly and many local authorities across the country are adapting it to meet their own needs or local context, allowing for improvement as they each learn what works best.

“Measuring the value and impact of these approaches is rarely easy but where this is done well both practitioners and families tell us this works for them. However, we also know that the effectiveness of any practice model depends on organisational values, leadership and the skills and confidence of practitioners.

“It is important that we continue to explore and evaluate ways of supporting children and families and to give local authorities the financial means to appropriately adapt these methods to meet local needs and context.”

12 Responses to ‘Little evidence’ Signs of Safety improves practice or children’s outcomes, finds pilot evaluation

  1. Kirsten Richardson February 9, 2021 at 5:00 am #

    EIP Signs of Safety Research: Statement by Munro, Turnell and Murphy in link below

  2. Ray Jones February 9, 2021 at 10:04 am #

    There have been two major innovations in children’s social services favoured and thrust forward by the Department for Education over the past ten years with the government making a considerable commitment of public funding. Each has been a means of generating income for those rolling out the innovations which they have captured and corralled as commercial business opportunities by promoting and protecting the brands they have created.
    ‘Signs of Safety’ gained traction as a means of shaping and structuring social work practice, but as now reported any possible efficacy is dependent on the stability, organisational culture, leadership and no doubt the adequacy or otherwise of resources such as social worker time.
    One big lesson might be that it is building a stable, experienced, confident and well supported and valued workforce with decent working conditions and which is embedded within communities, with continuity of relationships with children and families and with workers in other agencies, and with enough capacity and time to work well, is a more sensible script than novelty and change amidst cuts. In essence, innovation is not necessarily synonymous with improvement.
    The other multi-million pound publicly-funded innovation, championed in particular by the chief social worker in the DfE and which has been a significant income generator for the private company she had previously established, is the ‘Reclaiming Social Work’ model. It was based on very small pods of a consultant social worker who would be named as case-holder for all the pod’s children and families, and with maybe two or three social workers, a family therapist, and a children’s worker having the direct contact with the families, and supported by a pod administrator.
    It was heralded as allowing social workers to undertake more direct face-to-face work with children and families. But it was found to be unsustainable when the consultant social worker was accountable as the named worker for 60-80 children and families and they were often not available to supervise or support other workers in the pod as they might be immersed as the case holder in court proceedings and attendances at any one time for several of the families. It also had the consequence of social workers in the pods feeling disempowered by not being case holders and also by the direct work not being undertaken by the social workers but by the family therapist and the children’s worker. The most significant difficulty, however, was that the small pods were not large enough to be resilient as the absence or vacancy of one or two workers left the pods particularly vulnerable with not enough capacity to cover the workload. A corollary was that recruiting and retaining consultant social workers was a considerable challenge.
    Less has been heard about the ‘reclaiming social work’ model more recently. Have those local authorities who introduced the model in the early and mid 2010s, often with significant funding from the DfE’s Children’s Services Innovation Fund paid to Morning Lane Associates who promoted and rolled out the model, maintained and sustained the model as originally described by the chief social worker and MLA? Should there be a concern that the allocation of considerable public funding benefitted a small private company but created disruption not improvement? Or are there stories of success in keeping to and sustaining the model of small pods with consultant social workers as the case-holders which it would be good to hear?

  3. Jenny Green February 9, 2021 at 6:01 pm #

    I totally agree with Ray Jones. I feel vindicated to some degree because from the outset I said caution was necessary when the local authority I worked for at the time, was tripping over itself to introduce signs of safety as the social work panacea for all ills. I cited ‘reclaiming social work’ and the lack of research there which actually meant we didn’t know whether it was really as effective as claimed. The same can be said for signs of safety. I am not suggesting that it doesn’t have some merit in providing a framework for staff to structure what they were doing in assessments and for interventions. The premise of being strengths based and working more clearly and directly with families is admirable but what concerns me generally regarding social work, is this rush towards every new ‘trend’ which gets publicity as the next best practice miracle cure. In doing so many local authorities have spent a lot of tax payers money on training staff to use a particular approach…that is until the next ‘trend’ comes along. When staff are reliant on just these frameworks and the people who champion these approaches, they throw out the ‘baby with the bath water.’ So out goes theoretical understanding and/or the use of other approaches alongside it. This is a crazy way to develop strategy and leadership because staff are being told ‘now this is how we do it.’ At the time signs of safety was becoming all the rage, I was advocating the need to allow practitioners to use this approach alongside others in their tool bags but the response was nearly always, no, this is now how we practise in this authority.
    Why have experienced and often knowledgeable practitioners and then tie one of their hands behind their backs by insisting that there is only one approach that counts. Now although this latest research is saying that the approach causes no harm and could continue to be used, some local authorities will begin to ditch it as an approach in favour of the next big thing. I know that the authority I used to work for no longer favours signs of safety, they now favour a systems approach and the main champions have made their name and gone on to bigger and better things. When will this stop and senior leaders stop trying to make their mark by ensuring the whole workforce takes on this either/or way of ‘equipping’ social workers to do their jobs which are already hard enough. Then we layer that pressure with the additional pressure of having to learn a new framework or approach.
    This cannot just be the government’s fault, it’s a collective response of ‘leaders’as well and that is where the brakes need to be applied. What’s interesting is that by the time, it’s realised that the approach doesn’t work or has had minimal success, those leaders have been promoted or moved on, leaving the workforce to have to adapt to the next big thing, when the new leader arrives. This has been going on for years and must have it’s roots in social work still not being strong enough in its own foundations but the people who suffer the most are the families exposed to these ever changing approaches and the practitioners who constantly have to embrace whatever is next, whether they have the skills, knowledge and ability to work in the proposed way or not. This ongoing cycle has to stop. And yes, I too would like to know how many authorities have embraced wholesale (within systems that were not always compatible) both signs of safety and reclaiming social work and then have quietly dropped the approach.

  4. DC February 10, 2021 at 11:11 pm #

    I was instantly suspicious of signs of safety when it was indoctrinated literally overnight upon a struggling LA I was with at the time.

    My main critique was that whilst I am all for strengths based approaches the SOS model over relied on this which conflicted massively with my bread and butter CP work where talking honestly and openly about risk with families was integral.

    To introduce such a sweeping sea culture of change without warning on a critical domain of frontline social work seemed a nonsense and everyone around me knew it too.

    The complete hilarity of the SOS model was only compounded when we were told that it could also function as a model of structuring supervision!

    Like SOS was an absolute, decreed on high, and could be applied anywhere. I came away wanting to send out a true SOS and got out of that authority whilst there were still life jackets!

    • Elody MG February 20, 2021 at 8:11 am #

      Where do we start? I have been a frontline CP social worker for 15 years now and when I hear that the strength based approach conflicts with talking honestly about risks with families, in CP work, I really dont know where to start! One doesn’t prevent the other ….if you think it does maybe you should have a read at “reconceptualising parental non-engagement in child protection” Frontline briefing from Reseach in practice http://www.rip.org.uk

  5. Phil Sanderson February 12, 2021 at 4:38 pm #

    I worked for an LA that took up reclaiming social work and poured £250,000 into Morning Lane’s coffers. I well remember Trowler attending a staff conference and telling us that within 2-3 years we would see a reduction of 250 children in care. As Ray outlined the model failed and we could not recruit enough family therapists despite one of the Morning Lane leaders telling me that we would easily recruit such staff. The wheels soon came off as the numbers of children in care kept rising I think it was around 650 when we started reclaim and had hit 900 when I left last year

  6. Emily February 12, 2021 at 10:05 pm #

    My sense is that the implementation is the issue rather than the model itself which I think has a tremendous amount to offer. Often hear social workers talking about SoS like it is a process to follow rather than a framework that is helping you to analyse.

    SoS is not about making a list of positives and negatives… If you’re talking to a family using unspecific and institutionalised language (all the jargon) then this is not SoS either.

    SoS is a framework but social workers need the headspace / capacity as well as training / role modelling to use it properly – and it needs to be used in a way that stays true to its original spirit and values.

    There is no panacea for overstretched and highly stressed social work systems. SoS will never fix that.

  7. Louise February 16, 2021 at 1:28 pm #

    Any practice model will only be successfully implemented if you have the social work and leadership capacity to drive and embed this – wholescale change is difficult to implement when you have a workforce with huge turnover/churn/vacancy rates. Embedding any new ways of working at times of increasing caseloads for practitioners and first line managers to manage and restrictions on resources to support any interventions is never going to yield outstanding results for the workforce or families.

    The basis of Signs of Safety is good old fashioned social work practice – who doesn’t want to put the family at the centre of identifying what is not working for them and ask them what they are going to do to change things with support?

    Let’s not blame the model – the fault lies with a totally underfunded, under-resourced, unvalued public service. This report is another attempt at deflecting from governments failure to properly support children’s social care.

  8. carol February 18, 2021 at 11:33 am #

    Signs of Safety is just really good social work practice. The problem is that EVERYTHING prevents us from doing really good social work, including the culture of our organisation. SOS drives home that the organisation needs to embrace good old fashioned social work values – top to bottom. Throw out risk aversion, erosion of peoples rights and removal of children to a care system which is deeply flawed and dangerous. Do your job as a social worker. work with people realistically and positively. Protect children by working with families properly. And we can’t make that work ? DON’T BLAME THE MODEL. Also , sadly many social workers are insufficiently skilled to do their job properly. These are the reasons signs of safety “hasn’t worked”. I saw a group of experienced workers trying to do strengths based interviewing in a role play exercise. They struggled to do it. So far removed from what they normally do.

  9. Kate February 19, 2021 at 6:44 am #

    How is it that in the age of social work Masters graduates becoming routine, the astonishing claim can be legitimately made that many social workers are insufficiently skilled to do their job properly?

  10. Tom February 21, 2021 at 12:54 pm #

    Not much of a shock to hear WWCSC criticising SOS. They’ve got their own snake oil to sell and there’s not enough cash for all the profiteers.

    Children’s social work is sometimes difficult but it’s not complicated. If it’s done by well supported teams of good social workers – who focus on relationships, family, and community – in an organisation with a positive, learning culture; whatever “model” they use doesn’t matter.

  11. Inci February 23, 2021 at 6:15 pm #

    Careful Tom, you risk giving the myths a shove overboard. Where will the industry of self selected experts peddle their fads then?