In recent years, the charity and social enterprise sectors have made great improvements in the way we assess impact. Models such as Theory of Change have become commonplace, and it is now expected that organisations can demonstrate, rather than just assume, that their intervention has a positive impact. We are becoming increasingly skilled in using impact data to improve delivery processes, and ensuring that our programmes are as effective as possible in meeting the needs of our users – including women’s services.
But, as with all great tools, Theory of Change models and other methods of measuring impact can be misused and end up becoming a liability as well as a help. There are a number of ways that this can happen, but in this blog we want to focus on one which is often overlooked: the fact that Theories of Change often assume a homogenous client group.
Models like a Theory of Change are usually organised to show a number of things:
Often, however, models are not very good at distinguishing between the different experiences of groups accessing the service in question.
Why is this? Implicit in many Theory of Change models – and programme design – is the assumption that the same intervention will lead to the same outcomes for different people. It is often also assumed that participants will experience the intervention in the same way, i.e. that the mechanisms connecting activities and outputs with actual positive outcomes will work in a similar way for everyone. When we stop and think about it, this is clearly a big assumption to make and is rarely going to hold true. Is this a problem? Not necessarily – after all, Theories of Change are only theories and are intended to simplify reality. They are not supposed to accurately represent everything that changes as a result of an intervention which would quickly overwhelm us with data. Everyone is unique, but we can’t possibly design a different intervention for everyone, so we have to accept some level of homogenisation.
Nonetheless, Theories of Change with a one-size-fits-all approach can become a problem if they lead us to become blind to differences between groups. In effect, theoretical models assume that an intervention will work for the ‘average’ person, or for the ‘majority’ – but who is that majority? And what about other groups of people who may benefit from a similar intervention, but may not be reached, or may not experience the same positive outcomes as others? These questions are particularly pertinent for organisations that adopt a more ‘general’ or ‘open’ approach, rather than delivering programmes that are very targeted at particular groups in society.
In designing and evaluating services, we need to make sure that we’re not assuming a homogenous group of clients, a homogenous set of outcomes, or even that a service is effective just because it works for the majority. Of course, there are many different groups in society to consider – based on age, sexuality, language, life experience, etc. – but given that it’s International Women’s Day, we’d like to focus on women’s services.
Do you know how many women access your service? Do you know how many of them experience positive outcomes as a result? Do you even know what constitutes a positive outcome for them, taking into account their own experiences and perspectives? Did you include women in any decisions about what data you should collect about your intervention? Do you capture information about ‘soft’, emotional and interpersonal outcomes, which may impact women’s lives?
A Theory of Change that hasn’t considered the differences between your client groups can end up burdening your intervention with a narrow, ungendered lens. It can lead you to fail to recognise how women (or others) experience their circumstances and how they might respond to your programme. Gender socialisation is a significant force in our society and tends to impact on choices made by women – and men – in accessing services, as well as their priorities and needs. It is important that we account for these differences when designing women’s services and assessing their impact.
At a basic level, a good Theory of Change will prompt organisations to collect information about their service users that will give an insight into the differences between groups, and not leave us blind to important aspects of their lives. This International Women’s Day, we encourage you to reflect on how you deliver and monitor your women’s services. Ask yourself whether you are confident that you understand and cater for the needs of women of all backgrounds and life experiences, and your service has an appropriate system for responding to these issues. Don’t let a flawed impact measurement system leave you blind to issues that may be crucial to the lives of your service users, and ultimately to the success of your intervention.