As with all businesses, charities and funded services we use our outputs to measure efficiency, report on performance and to check how we are progressing towards our targets and end goals. But when it comes to measuring the difference your efforts make to an individual, therein lies the challenge.
We talk about soft outcomes and indeed we can ask a service user if they feel their mental health has improved, but the measurement of this is so absolutely and subjectively related to that one individual in that moment and so it is difficult to find a “one size fits all” standard monitoring tool. Extrapolate this over 10 different approaches, all working to improve mental health and wellbeing, and it is clear this cannot be a simple problem to solve.
All of the services available through Together For You have already been quality assured and have a strong evidence base to their efficacy. The partner organisations use a range of already available and evidenced monitoring systems to evaluate the impact of their interventions including the Core Outcome Measure and the Warwick Edinburgh Mental Wellbeing scale. However each of these systems suit specific types of intervention, for instance the Core Outcome Measure is widely used by counselling services but would not necessarily fit as a monitoring tool for a Carer attending a social activity with their loved ones.
To be assured we are delivering good quality services which improve the mental health and wellbeing of individuals and communities, and to evaluate the impact of the partnership, the Together For You partners have faced this challenge of developing an evaluation system. A system which enables the project evaluator to analyse beneficiary impact, evaluate the range of delivery methods which are used and measure the additional benefits of the partnership.
Whilst the simplest option would be for each of the partners to continue using their own monitoring processes and forward the information through for the evaluator, this would only support analysis of each individual organisations data, and could not provide an overview of impact of the project as a whole.
Once the project evaluator had been selected in January 2014, the need for a consolidated system became clear. An evaluation steering group was established to oversee the evaluation of the project and to support the evaluator and the partnership to develop a system for collection and analysis of data. It was agreed a number of common indicators were needed across all of the services and we began developing questionnaires for each of the partners to use with the individuals who accessed their services. This quickly became a very arduous and complex piece of work as each partner delivers a number of interventions to a range of individuals from large groups of young people accessing a Mental Health Awareness session to an individual participating in Cognitive Behavioural Therapy Counselling, and we needed to know not only the impact of the service for each individual but also that the services were reaching target groups.
The second piece of the jigsaw was finding a way to collate all of the information being collected without the resources to develop a sophisticated database which each partner could access and which could potentially track individuals across the services. Bearing in mind that the Together For You project will reach over 35,000 individuals with a need to collect equality monitoring as well as impact monitoring information from each individual, it was clear that a software solution would be the most effective and efficient way forward.
The agreed evaluation processes were implemented across the partnership in November 2014, the information collected was presented and analysed in the interim evaluation report. Although the information was limited it did provide a broad overview of those accessing the services in that month and supported the identification of recommendations. When the 2nd annual project report is published in August 2015 we will have a more substantive analysis and look forward to reviewing the outcomes of this. The interim report can be viewed here.
Enter your email address below to receive regular updates from Together For You.
Most browsers allow you to turn off cookies. To do this, look at the 'help' menu on your browser. Switching off cookies may restrict your use of the website and/or delay or affect the way in which it operates.