Research realities: Assessing early intervention programmes

Research analyst Susannah Bowyer examines difficulties in assessing the effectiveness of early intervention schemes

Title: Issues in Earlier Intervention: Identifying and supporting children with additional needs

Authors: June Statham and Marjorie Smith

Published by the then Department for Children, Schools and Families, March 2010

Objectives

Statham and Smith’s review was commissioned to provide more detailed understanding of early intervention and it will be useful to anyone charged with ensuring that reduced resources are used effectively. Three main themes in relation to earlier intervention with children with additional needs are addressed: recognition and identification of need; cost effectiveness, and effectiveness of interventions. The review considers UK studies published after 1998. In unpacking the concept of earlier intervention, the authors identify three related aspects:

Recognition and identification

Effective early intervention relies on being able to identify who is most at risk of the poor outcomes we seek to prevent. Risk factors may be structural (poverty, environment) or more closely related to the child or family functioning (domestic violence, parental mental health).

Community level variables (such as local levels of employment or crime) are poor predictors of individual outcomes. Area-based targeting (such as building Sure Start centres in areas of deprivation) throws a wide net over a locality. Those most in need may not take up the support offered, while those less in need take advantage of provision, as was indicated by the first national evaluation of Sure Start.

Not every child identified as at risk goes on to become an offender, while others who were not exposed to that risk factor may go on to develop the problem in question. The earlier an attempt to identify risk, the more imprecise it is likely to be as a predictor of future outcomes. Consequently, if service strategies are based on risk factors with poor predictive validity there is strong likelihood that money will be wasted delivering interventions to people who will not develop the problem while failing to reach others. For people from all communities it is the risk factors that affect a child directly (such as maternal behaviour) that impact most profoundly on child outcomes.

Identification on the basis of multiple risk factors can give a high level of predictability for high cost and high harm outcomes in later life, but this should still not be seen as an inevitable causal relationship. As the authors point out, it is important to remember that people can and do break out of cycles of disadvantage where protective factors such as resilience, coping strategies, extended support, and help-seeking behaviour interact with well-targeted professional interventions to promote positive change.

Frontline practitioners are familiar with the reality that, for any negative outcome there will be a number of related risk factors. Both risk factors and negative outcomes tend to cluster, so that exposure to one increases the likelihood of exposure to others while one poor outcome (behavioural problems) increases the risk of others (poor educational attainment). Understanding the variety of factors that characterise multi-problem families is essential if such families are to be helped.

Families most in need are least likely to take up offers of support. Parenting programmes, for instance, have fairly low participation rates and high drop-out rates, and the more disadvantaged parents are the ones less likely to complete the course. Effective targeting needs to consider how best to market programmes that are based on white, middle class values to parents from other social or cultural groups.

Using risk factors to identify at-risk groups does not replace the need for good individual assessment of children. Early signs of problems for individual children and families can be picked up in universal settings, where the use of the common assessment framework processes by midwives, children’s centres and school staff is key. So is working locally to address the lack of confidence that research has identified among education and health professionals about making referrals to children’s social care (Gilbert et al 2009)? Where it works well, the Common Assessment Framework (CAF) process can contribute to lower rates of referral to social care and better early intervention through other means. CAF working is hindered if multi-agency practitioners feel that additional resources are unlikely to be available to support the needs uncovered.

Cost effectiveness

Everyone’s mind is sharply focused on cost effectiveness, but making these calculations requires robust data on outcomes as well as costs. Strong evaluation data on effectiveness is still scarce.

We are increasingly familiar with the names of the handful of parenting programmes that can demonstrate efficacy through rigorous evaluation, such as Triple P and Incredible Years, but across children’s services few earlier interventions have been evaluated as systematically as these. There is very little research that compares intervening earlier with later intervention.

Another issue is the timescale for measuring the outcomes of preventive work. There may be projected savings, such as reducing offending behaviour in later life, but very few evaluations have any kind of longitudinal follow up. Where savings might accrue to other agencies in the future, such as to the criminal justice system, distributing and accounting for costs and benefits becomes a particularly relevant issue. It is interesting to note the suggestion by Iain Duncan Smith and Graham Allen of an “early intervention bond”, a financial instrument to allow private investment in effective programmes (Allen and Smith 2010).

Effective interventions

The report looks in detail at three well evaluated interventions in relation to emotional and mental health and three aimed at child abuse and neglect. The six are designed to be delivered at three different levels: universal and preventive; targeted and selective and indicated (where a need has been identified).

A common thread that emerges strongly from a range of findings is that duration and intensity of interventions should be matched to levels of need, rather than “more” being seen as better in itself. There is certainly some evidence that interventions of a longer duration may be necessary to achieve changes in behaviour rather than changes in softer outcomes such as attitude or confidence. One example, from a meta-analysis of home visiting programmes that aim to reduce incidences of child maltreatment, is that these work better as longer, more intensive, programmes, with schemes of less than six months or with twelve or fewer visits showing the least effect.

There are critical periods at which intervention may be more welcomed by families, or at which the ‘window of opportunity’ to intervene is particularly significant. The growing body of evidence of the impact of neglect and maltreatment on early infant brain development provides an increasing impetus for working with high risk mothers in pregnancy and when babies are very young. Some projects have found that parents seem willing to engage with support at the point at which children start school, before major problems emerge (see case study).

The point at which a programme ends can also be crucial. The Reading Recovery programme provides intensive support to children in year one at school, usually 15-20 weeks of tailored work that finishes when the child is able to work to the class average level. Follow-up research has shown that once the programme has ended, reading level trajectories remain parallel, so if a child is not brought up to normal reading level in the course of the programme they are unlikely to get there in future. The programme also provides a good means of identifying children who are not helped by the programme, who need more sustained, long-term remedial help.

The overall conclusion of the review is that a continuum of services is essential and that it is, the report states, “unrealistic to think of earlier intervention as an alternative to later intervention when problems have become established: both are needed”.

Susannah Bowyer is a research analyst at Research in Practice

Further reading

Access the full report

Prompt topic briefing 4: Effective Targeting

Allen, G and Smith, ID (2010) “The cross party challenge: Early intervention for children and families”, Journal of Children’s Services, 5 (1), 4-8.

Gilbert, R et al (2009). “Recognising and responding to child maltreatment”, Lancet 373, 9658, 167-180

Scott S et al, “Randomized controlled trial of parent groups for child antisocial behavior targeting multiple risk factors: the SPOKES project” Journal of Child Psychology and Psychiatry 51, 1 pp 48-57

Case study

Spokes Project: Reading and parenting, London

When it’s done well, with rigorous research on outcomes built into the project design, population-based early intervention can produce strong results.

The Spokes project combined the Incredible Years parenting programme and a reading programme – Supporting Parents on Kids Education in Schools (Spokes) – in a school-based project that targeted four risk factors: ineffective parenting, conduct problems, attention deficit/hyperactivity disorder (ADHD) symptoms, and low reading ability.

A randomised controlled trial was carried out in eight primary schools in a deprived area of London. Nearly a thousand five and six year olds were screened for antisocial behaviour and the parents of 112 high scorers were randomly allocated to parenting groups held in schools or to a control group.

But those in parenting groups received a 26-week programme which included 12 weeks of Incredible Years and 10 weeks of Spokes, which helps parents read with their children. Emphasis was given to strong training and supervision of therapists to ensure the programmes were delivered as designed.

Those in the control group were given information about a telephone helpline advising them how to obtain regular services.

Progress of both groups was followed up a year later. Effectiveness was measured through parent interviews to assess conduct problems, direct observation of parenting skills and psychometric testing of children’s reading.

The behaviour of the children in the control group didn’t change. For the children receiving the interventions, conduct problems and ADHD symptoms were reduced, oppositional-defiant disorder was halved from 60% to 31% and reading age improved by six months.

Teacher-rated behaviour did not change. Parenting improved with parents using play, praise and time-out more often than those in the control group and resorting to harsh discipline less often. The costs were around £2,500 per child.

Source: bit.ly/4ZkU75

PRACTICE IMPLICATIONS

Health visitors

● Health visitors have been found to be one of the commonest sources of referral to children’s social services of cases involving domestic violence and substance misuse. In a Welsh study of the implementation of Incredible Years parenting programmes through Sure Start, health visitors were able to effectively identify three- and four-year-old children on their caseloads who were most at risk of developing conduct disorders and to recruit them to the parenting programme.

School-based social work

● The small-scale research studies looking at social workers based in schools is promising, suggesting that the stigma of contact with social care is reduced, difficulties are addressed earlier and that relationships are built with universal services staff.

Avoid stigma

● Work to avoid the stigma of being targeted for an intervention: use a non-judgemental approach in talking to people about attending; make programmes accessible and attractive (childcare, meals, transport, raffle prizes for attendance); think about cultural and social issues.

Case closure

● Make sure that “no further action decisions” by children’s social care include redirecting families to more appropriate sources of support, to avoid families bouncing repeatedly off the social care threshold until their problems intensify enough for them to receive help.

Revolving door syndrome

● For families with multiple and complex needs, intensive interventions should be backed up by low level ongoing support to mitigate the revolving door syndrome that characterises the relationship of many high need families with children’s services.

This article is published in the 29 July issue of Community Care magazine under the heading Assessing early intervention