Final Evaluations – creating positive energy for continuation

Luís Costa, Holger Schmid, Marie-Madeleine Gomez, Verónica Troya

As our funding came to an end, MAVA convened our partners to evaluate progress and impact in the Outcome Action Plans. This blog provides a glimpse behind the scenes of how MAVA managed the challenge of measuring impact, gathering partners’ perspectives on the results of their work, and what the future of the work looks like. 

____

During our normal activity, we are used to evaluating the grants we give and the impact of our activity. The closure of MAVA brings a new challenge: how to evaluate the results and the impact of the Foundation after 28 years of activity?

We decided to do it at three levels. First, by evaluating the strategies of our final phase, when we developed 24 Outcome Action Plans (OAP) within the four programmes (Mediterranean, Sustainable Economy, Switzerland and West Africa) between 2016 and 2022.

Adding to the 24 OAP evaluations, we will also undertake a final evaluation of each programme for the full lifespan of MAVA since 1994, adding the Impact and Sustainability Unit to the 4 programmes above. And finally, we will perform a global final evaluation of the Foundation’s impact.

Whilst the programme and global final evaluations will be carried out early next year, we just finished the busy phase of OAP Final Evaluations by bringing together our large community of partners that worked hard during the last 7 years to deliver.

Developing a simple and inclusive methodology

For defining the methodology for this evaluation, we wanted to do something that would not create unnecessary work but would rather create added value for all – in the same mindset as the collaborations of each OAP, trusting the partnership and having them self-assessing their impact.

The methodology aimed to step back from the individual project and look at the bigger picture, to show outcomes and impacts in a succinct and objective report. We chose to do this instead of individual project reports, to show in a fast and effective way the collective work done during the last 7 years.

One workshop has been organized for each OAP, gathering all direct partners that collectively have been running the projects towards a common conservation goal by using the Open Standards for Conservation. As everyone was using monitoring scorecards for regularly assessing progress and effectiveness of their work, we analysed all indicators in those scorecards as a basis for discussing results and impact.

A collective exercise of writing allowed reports to be completed by the end of each workshop, in an inclusive and equitable manner that gives all partners the power of the pen. In the end, each report includes:

A general introduction to the OAP and its Theory of Change.

A chapter on Progress to describe how activities were carried out and how innovative the approach was, and to respond to the question ‘did we do what we were meant to do?’

A chapter on Effectiveness and Impact, responding to the question ‘did we achieve the impact we were meant to deliver?’ to show clear results and impacts of each OAP.

And finally, notes on Sustainability and Next Steps, showing the new challenges for the future and what will happen to the partnerships and their new projects.

In addition, some case studies were included to illustrate good results from projects that represent the collective impact of the OAP.

The final results

All workshops took place between September and December 2022. As the reports are finalized on a rolling basis (after final editing, translation and design) they will be posted on our website. The final versions can be viewed at Resources – MAVA Foundation (mava-foundation.org)

In general, we are amazed by the great work and the huge impact that this exercise of final evaluation was able to demonstrate. More than analysing individual projects within each OAP, this exercise of compilation of all projects into a single report allowed all partners and us to assess and realise the value of how well partners delivered on the commonly-defined strategy.

Evaluating the evaluation

The collective exercise of writing the reports was fun and apparently welcomed by all participants. It came with some questioning in the beginning about how feasible it could be to write a full report agreed by all partners in two or three days, but in the end all workshops delivered. The main method was to form groups of people to write each chapter of the report; after a first round of writing, groups rotated to review and complete the previous group’s work; and another rotation allowed the group to agree on a final version of each chapter before reviewing and agreeing on the global text in the end.

This exercise was the core of the workshop, but more than that, workshops provided a tremendous opportunity for partners to sit together after a couple of years of remote work, and to celebrate their successes in nice events and activities. A group photo identifies partners in each report and shows who is behind all this work.

In the end, the feedback from participants and facilitators was positive:

Everybody felt involved in this final evaluation process and played various roles that included a valuable help from some of them to facilitate the preparation work and facilitate the workshops. In general, people were surprised how productive it was, from the quality of discussions to the final product.

The collective writing exercise allowed everyone to write and removed the bias that usually comes when one person leads on the writing and may have a tendency to put more focus on her/his own ideas. The exercise led to a product with a high degree of buy-in and ownership, and this methodology is recommended not only for evaluation purposes, but also for strategy and programme development because it creates joint ownership and clarity.

This process allowed partners to know more about each other’s work and how projects could contribute altogether to a common goal and objectives. The final result shows a clear collective output of each full partnership.

Normally each project must write and submit a comprehensive final technical report. Replacing these by a single short and collective report was much better; it saved partners’ time to write their reports, but also allowed the MAVA team to analyse reports based on collective results and impact, rather than one piece at a time. Incidentally, it also made an efficient use of MAVA staff time, which we appreciated as well. Each project still had to submit final financial reports which were reviewed separately.

By meeting together, partners had a chance to celebrate great work. In addition to some emotional goodbyes, it was especially a great and crucial opportunity to discuss how to continue collaborating and delivering on OAP goals in the future and an occasion to reinforce bonds among partners.

The report shows collective work and impact, therefore it may be an important resource for fundraising so they can continue their work together.

The complete list of final reports will be available soon on our website (Resources – MAVA Foundation (mava-foundation.org) ), and will remain there for the next 10 years. In early 2023 we will start our final evaluation process at Programme and global Foundation-level. Watch this space for more news on those outcomes as the last evaluations are completed.

Previous post

Back

Next post