Tag Archives: evaluation

The difference that makes the difference

By John Hitchin

I went to see a film by Nora Bateson last night about her father, Gregory – anthropologist, linguist, semiotician, ecologist, cyberneticist and all round polymath.

Whilst there were certain things about the film that I didn’t enjoy – Californians walking bare-foot around rock pools did not add to my thinking about double bind – it was, in the most part, a wonderfully accessible route into Bateson’s thought. Some of the archival footage of him talking about schismogenesis made the idea much clearer and more accessible than one could imagine from his books.  It made me realise what a wonderful teacher he would have been.

There are lots of ways into Bateson’s thought – and I am nothing more than a beginner – but the idea that struck me most last night was his definition of information as ‘the difference that makes the difference’. If my understanding is right, our way of defining the difference between x and y is one based on the thing that makes a difference to us in our usage of either x or y.

And since I am just finishing off a piece of work for the Princess Royal Trust for Carers, that is where my mind immediately turned. The work was an evaluation of a pilot project that was exploring whether supporting the whole family was a more effective route to better outcomes for young carers, than supporting the young carer in isolation.

We hope to publish that work soon, but my view was that it definitely can be.  And if you have any knowledge of Bateson, then you would know that his thinking would support that too. It’s all about relationships and ecologies, not ‘things’. In fact, for Bateson there is no ‘thing’, just our ideas and relationships that surround and define them.  And so just looking at one end of a relationship – the young carer – is always going to lead you down a cul-de-sac.

The challenge for any evaluation, however, is the ability to pull out and definite the ‘difference that makes the difference’.  There is a lot of interest at the moment in randomization in public policy evaluation. I think that is a good thing and to be commended. As I have written about before, however, there are risks to the push. It is the potential loss of the qualitative.  And for me the qualitative is what Bateson is talking about a lot of the time.

The difference that made the difference for many of the families that we worked with in that study, was the relationships between them and the work of the project staff in shifting those relationships.  We were only able to get into those relationships by purposively talking to families, young carers, staff, health workers and others: to develop our understanding of the ecologies of relationships around those young carers.  That was the only way we could see what was working and what was changing.

The challenge for this sort of work, however, comes also from Bateson. Because qualitative research leads us to create typologies, and if those typologies are too loose or too imprecise, then we define types or circumstances for ease rather than for their ability to identify the crucial characteristics.

The best thinkers encourage cognitive dissonance in others. And Bateson does for me. But from the perspective of public policy research and evaluation, I strongly believe that whilst we need more randomization, we also need more qualitative.  Because as there is a growing interest in randomised controlled trials, there is also a growing interest in the human elements in our services.  In his new book, David Boyle says that ‘The human element may be a source of error, but it is also the only source of genuine change’. I strongly agree with this, but I believe the only way to understand this human element is to get to grips with the relationship ecologies they create.  Because they are the differences, that make the difference.

Urban Academy – Evaluation

Renaisi was commissioned to carry out an external evaluation of Urban Academy – a lifestyle development programme for young people, delivered across the London Boroughs of Richmond and Hounslow.

The evaluation is to be outcome based and is being carried out in order to support the process of creating a forward strategy for the programme and to help generate new investment.

Urban Academy has been in operation since 2005 and is a partnership based programme funded by a combination of investment from its accountable body; Richmond Housing Partnership, and grant funding from the Football Foundation.  The objective of the project is to transform the lives of young people on a number of core social housing estates across the two boroughs, with a large focus on sport.

Quantitative primary data sources have been used to gather information, which will be collated in order to provide a coherent picture of Urban Academy’s history and development as well as the demographics of those participating in the project and some of the outcomes resulting from the project.  Once the data is all drawn together the evaluation will identify areas where the project is strong and where delivery can be improved.  It will also include practical methods to help Urban Academy make the suggested improvements.

Renaisi’s consultant Russell Spencer is working on the evaluation.  He says: “The Urban Academy is an exciting and innovative project that uses sport as a means of engaging young people. I am proud to be evaluating their work and believe that it will be vital in clearly demonstrating their good work and helping them secure further funding to continue their programme”

St Paul’s Way Transformational Project

Renaisi has been commissioned to work on the St Paul’s Way Transformational Project. This innovative partnership of London Borough of Tower Hamlets, Poplar Harca, and Tower Hamlets Primary Care Trust aims to transform both the physical and social environments for the residents and working population of this street in Tower Hamlets.

Renaisi’s role will be to provide support to the policy group who are managing the partnership, as well as evaluating the effectiveness of the current group’s structure to enable the project to run more efficiently. We have also been asked to deliver communications support for the project as a whole.

St Paul’s Way runs through the heart of Poplar, in the midst of Europe’s largest regeneration area, encompassing Canary Wharf, the Royal Docks, Stratford City and the site of the London 2012 Olympics. Along St Paul’s Way, within a few hundred yards of each other, are two housing estates, a secondary school, a GP practice, a primary school, a retail parade and several community facilities. Yet, far from bringing communities together, St Paul’s Way acts as a car-dominated boundary between the residential communities on either side of it.

The St Paul’s Way Transitional Project was initiated because a series of significant developments were planned along the street, but they were being developed in isolation to each other. The St Paul’s Way Partnership was created bring these developments together and pursue a coordinated vision for the future of the area. This provided a unique and exciting opportunity to deliver a legacy of sustainable regeneration.

The partnership aims to create a place where people want to live, work, visit and play. There are already committed proposals for mas­sive investment in local facilities and provision of 1500 new homes. These proposed projects include:

- St. Paul’s Way Community School- a new building for 1300 pupils

- Leopold Estate Regeneration – providing new and refurbished homes

- Cafe Reconnect and Community Centre – a brand new building

- A new Medical Centre catering for 11,000 patients

- Improvements to Furze Green

- New developments on the Limehouse Cut

- A new artist’s community

The overall aim is not only to transform the physical environment along St Paul’s Way in Tower Hamlets and to coordinate the various planned developments, but also to transform relationships between fragmented local communities.

Renaisi to Evaluate

Kings Norton Community Wardens

Renaisi has recently been commissioned by the Kings Norton 3 Estates Community Development Trust and Birmingham City Council to evaluate their Community Wardens project.  The Community Wardens project has been funded by the Kings Norton New Deal for Communities (NDC) programme since 2002 and aims to reduce crime, and fear of crime, within the NDC area, which comprises of three housing estates.

The evaluation will consist of a range of partner and stakeholder interviews, plus focus group discussions with residents from each of the three estates.  This will be complemented by an analysis of output and spend performance. The evaluation will outline the key successes and weaknesses of the project, the overall impact it has had on the NDC area and will give recommendations as to how the project can be improved and sustained past March 2009, when its current funding comes to an end. A final evaluation report will be delivered in March 2009

Doncaster NRF Evaluation

Renaisi has been commissioned by Doncaster Metropolitan Borough Council to carry out an evaluation of 50 projects funded by the Neighbourhood Renewal Fund (NRF) between 2006 and 2008. The evaluation results will be used to inform decision making for the next round of strategic commissioning in relation to NRF funding.

The 50 projects being evaluated are from across many different themes including health, education, crime, housing, income and worklessness.

Rather than looking at output figures, the evaluation will focus on the projects’ beneficiaries. Renaisi’s team of consultants has been carrying out interviews and liaising with focus groups to gather information on the impact the projects have had on the beneficiaries.

Doncaster are currently working on agreeing their Local Area Agreement (LAA) priorities. This evaluation will be used when deciding which projects should be commissioned next year as part of the LAA commissioning process.

Renaisi consultant Rose Baker is one of the team working on the evaluations. She says “We have been able to meet with a large number of projects and witness some really inspirational work with vulnerable people in Doncaster”.

For further information please contact Rose Baker on 0207 033 2600 or email r.baker@renaisi.com