The search for objectivity in measuring social work outcomes

Philosophy pervades policy and practice in social work but is usually hidden behind the implicit assumptions underpinning both.

Bringing these assumptions into the open makes it easier to understand why there are conflicting views on key issues and can bring clarity about the foundations of current policies.

One area in which this is a valuable exercise is examining the quest for “objectivity” in social work.

When twinned with the demands for transparency and accountability it has led to efforts to find objective ways of describing and measuring practice.

By why is this seen as desirable and is it feasible? Does it lead to adequate and useful accounts of social work?

Scientific knowledge is often taken as the paradigm example of objective knowledge. Yet there have also been significantly different notions of objectivity in science. Daston describes objectivity as having “more layers of meaning than a mille-feuille”:

“Sometimes objectivity refers to ontology: “an objective world of particulars independent of experience”. Sometimes it refers to epistemology: “beliefs, judgments, propositions or products of thought about what is really the case”. And sometimes it refers to character; “impartiality, detachment, disinterestedness, and a willingness to submit to evidence”.

Objectivity is usually defined relative to subjectivity which is often seen as dangerous in some way. So the various efforts to be objective are usually trying to allay some fear of how subjectivity may be distorting our reasoning. In social work I shall concentrate on just two specific instances of this: in making judgments about the quality of practice and in relating to families and understanding them.

Since the 1980s there has been a significant loss of trust in professionals generally, and in social workers in particular. Therefore, the autonomy and privacy they had previously enjoyed is now seen as risky. Their own judgments about the quality of their work are now seen as too subjective, potentially self-serving, not focused on the good of service users and therefore not reliable enough. A more stringent form of inspection, paying more attention to measurable aspects of practice, was introduced.

Social workers will be familiar with the “objective” measures now used to judge practice: timescales and aspects of procedures creating data that can be readily measured – an inspector or manager can easily count up the number of initial assessments completed within the prescribed 10 days with a high level of agreement between all who make the measurement.

However, the objectivity this provides is limited. The choice of variables to study and how to evidence them are both subjective. Other variables could produce a radically different picture of practice. A focus on the impact on service users more than the activities of professionals would also offer transparency but provide a very different picture.

In our current culture, numbers convey a possibly unwarranted degree of authority and power so that variables that can be easily measured by counting tend to be more highly valued than variables that require qualitative accounts. Children’s subjective accounts of how they experienced the help from professionals offer one form of transparency and evidence about how well the service has functioned, but it is hard to turn these into simple numbers.

Economic constraints also tend to encourage the choice of variables that are simple and cheap to measure. The political goal of being able to compare local authorities also places a high value on standardised data- ensuring the ‘naming and shaming’ of those who score badly.

However, the robustness of such comparisons is limited since there is variability in the practice behind the data, i.e. they can be achieved through poor practice as well as good. For instance, a low score on the indicator of how many children are on child protection plans more than two years can be achieved by removing children from a plan just to meet the indicator or because it is genuinely thought that home circumstances have improved sufficiently.

When deciding how or what to measure, we need to consider both the reliability and validity of the measure. They are often in conflict so that the most reliable measures – such as timescales – often have low validity. We are not really interested in time in isolation but in timeliness relative to the child’s needs (i.e. ten days may not be the right timeframe for some children) and in achieving a good quality assessment as the basis for action.

Determining what is a good quality assessment requires judgment but this need not lead to low reliability. In science, the limitations of simple measurement or observation have long been recognised so that, for some variables, trained judgment is needed. Although this does not remove the observer from the measurement (an ambition of some) it does lead to ratings that are consistent between observers.

Similar developments have occurred in social work when we see Ofsted inspectors, for example, trained to improve the consistency of judgments so they are not limited to just those aspects of practice that are easy to observe.

Thus subjectivity has a place in determining transparency and accountability.

In my second area – relating to families – subjectivity clearly has a place but also some potential dangers.

Social workers seek to treat service users as subjects not objects and that requires understanding how things look from the service user’s point of view, how they make sense of problems, what their values and preferences are. The scientific image of the disinterested and remote scientist is clearly inappropriate, and probably unattainable because our intuitive and empathetic responses mostly occur automatically and unconsiously.

Talking with a child who is crying in despair, a social worker will intuitively respond and feel some of the child’s distress. Indeed, when workers cease to have an emotional response, it is often a sign they are getting burnt out.

This closeness, this subjective response, is a rich source of understanding. The quality of the relationship contributes to therapeutic effectiveness. It can be an efficient source of accurate judgments but also a potential source of error.

As Gigerenzer notes: “Gut feelings are in fact neither impeccable or stupid they take advantage of the evolved capacities of the brain and are based on rules of thumb that enable us to act fast and with astounding accuracy.”

Kahneman’s work has identified the recurrent biases found in our intuitive reasoning and offers a way of checking for them that will be familiar to social workers: that of supervisors offering an outsider’s view of the assessment and the planning of a case and seeking to detect and remove the biases. There is considerable evidence to show that it is difficult to review one’s own judgments but defending them to another person can reveal omissions or preferences that may have distorted them.

Overall, current debates in social work tend to see ‘objectivity’ as a desirable attribute, a way of obtaining a more reliable or truer account of practice.

However, when objectivity is cited as the goal, it is important to find out what claim is being made on its behalf and what feature of subjectivity is being seen as dangerous. Within social work there is a place for both subjectivity and objectivity.

Eileen Munro is Professor of Social Policy at the London School of Economics and author of the Munro Review of Child Protection Systems in England.

Further Reading

Daston, L. & Galison, P. (2010) Objectivity, New York, Zone Books.

Porter, T. (1995) Trust in Numbers, The Pursuit of Objectivity in Science and Public Life, Princeton University Press.

Gigerenzer, G. (2007) Gut Feelings: The Intelligence of the Unconscious, London, Viking Penguin.

Kahneman, D. (2011) Thinking, Fast and Slow, London, Allen Lane.

What do you think? Join the debate on CareSpace

Keep up to date with the latest developments in social care Sign up to our daily and weekly emails

Related articles

The Munro Recommendations on Child Protection in England

‘Council social workers can’t practise ethically’

Inform subscribers

Guide to analytic and intuitive reasoning