Introduction

The Civil Society and Testing Change project (CS&TC) is an initiative to develop and test new ways for civil society organizations and their partners to operate.  The project’s model is for global, multi-stakeholder working groups to focus on areas where operational changes may have the most effect on impact, and to have a lead group test these ideas and theories over a multi-year period.  An introductory blog post can be found here. Our target group are NGOs and civil society organizations that are 10+ years old, interested in considering operational changes to remain effective and relevant, and with enough leadership commitment and staff capacity to initiate some changes.

To represent the range of issues organizations need to be dealing with we plan to have 4-5 working groups.  We currently have two  – one on youth engagement (co-led by CIVICUS) and one on impact assessment (co-led by GlobalGiving).  The members come from 9 different countries. The youth engagement group has developed and recommended ideas to CIVICUS and one is now being tested.  The impact assessment group is in the process of developing ideas.  We are working primarily virtually and are using a modified design thinking process.

These are some of our lessons we have learned thus far:

Process lessons

  • Keeping the focus on paradigm change – We are looking for ideas that will make significant, paradigm change, not just incremental change.  Paradigm change needs on-going practice to change our mind-sets and normal behavior patterns. This may call for strong and creative facilitation to encourage people to not fall into accustomed tracks but continually step off of them.  We were somewhat surprised at how easily all of us (even those of us who consider ourselves ‘change agents’) fall into these patterns and how important it is to be intentional about changing some of the patterns we have become accustomed to. Venkat Ramakrishnan, Project Leader for Isha Education at GlobalGiving and a member of the impact assessment working group, points out that there are two methods in which the paradigm-shift ideas might be generated – (1) more in depth analysis and converting the outcomes of the analysis into actionable ideas and (2) updating existing programs to create the paradigm-shift in the mind-sets as well as in the organizations’ strategy and processes. He also notes that while ideas that are related to updating existing programs might be easier to identify because of strong expertise on what exists today, delving into theory may allow paradigm-shift ideas to evolve, but it does call for strong moderation to keep the focus on actionable (rather than just theoretical) discussion and outcomes.
  • Avoiding falling into organizational silos – The project was designed to deal with a wide range of challenges organizations are facing. But as organizational structures (and the broader ecosystem) is still organized into ‘issue areas’ that is also how we have organized our working groups – youth engagement, impact assessment, multi-stakeholder collaboration, etc… Although our goal is to keep the working groups and their experiences integrated at the highest leadership levels of organizations this can be a challenge.  We believe that the way most organizations are currently structured can lead to unintentionally siloed learning pattern. Thus we are trying to intentionally work on i) keeping senior staff engaged so the work is fully integrated into the organization’s operations and doesn’t get siloed into the staff working on the ‘issue area’ and ii) linking the work of the different working groups so they (and a broader audience in the social sector) can see the learning and synergy that comes from this linkage.  We are using the project’s Steering Group to help do this.
  • Keeping the focus on what is doable – Some of the subjects we have taken on (such as impact assessment) are quite broad and are being considered by a number of different groups around the world. Notes Alison Carlman, Director of Impact and Communication for GlobaGiving the co-lead on impact assessment: “In a field as broad as monitoring and evaluation (M&E), it is difficult to stay on top of all the research and perspectives. Rather than asking everyone to be responsible for every bit of background, we simply chose to invite well-versed participants and asked everyone to bring his or her own perspective to the table.”  Having a working group from different networks and different perspectives allows us to bring in a wide range of background information and issues but it can also pull us in many directions.  It is important for us to keep our focus on our target groups and lead testing groups to know where we can realistically make a difference.
  • Starting a new diverse, working group virtually is hard, but doable – Most of us are accustomed to starting new working groups in-person.  That gives us time for ‘orientation’ and some team building.  Doing this online can be more challenging. Activities to get to know each other need to be streamlined.  And without the benefit of being able to easily read some of the nonverbal communication among group participants, facilitators and attendees need to actively work at surfacing feedback that may happen more naturally in-person.  (Fairly reliable videoconference platforms – such as Zoom – are very helpful, but depending on where individual participants are connecting from, bandwith and local reliability can still be a challenge and may result in some switching from video to just audio, or even chat text during the call.  We’ve also used WebEx, a google site and google docs).  Alison notes: “One-on-one conversations (whether by phone or in person) outside the working group meetings also helped some participants build rapport and understanding that’s hard to do in a one-hour timeslot”.
  • Using a “modified” design thinking process – A modified design thinking process can work well for prototyping ideas and in a virtual setting. Although design thinking was created to brainstorm among a group working in-person, and to prototype physical products, we have found that brainstorming virtually and ‘prototyping’ ideas by having group members further develop them through text, graphs or storyboarding can work well to generate ideas and then find the ones that they would like to take forward.
  • Be patient and stay curious – We all are tempted to stop immediately when we arrive at some decisions or conclusions. Magda Mook, CEO of the International Coach Federation and a member of the youth engagement group notes that “Instead, it worked well for our group to stay in the question, to inquire about all different aspects of the proposed decisions and to push back a little in order to examine all the facets of the issue at hand. That resulted in a better end product and a solid recommendation. “

 

Content/Issue Lessons

In addition to our process learning, each of the working groups are also beginning to learn about how the broad issue areas they are looking at can be filtered to be especially useful to organizations.  Although we are just in the early stages of each group’s work, these are some initial learnings.

Youth

  • Based on initial discussions, the CIVICUS staff has made an effort to include more youth organizations as partners.
  • One partnership with a youth organization in Brazil resulted in over 2,000 people attending 10 events. The organizers believe that many of those who attended were ‘young people who wouldn’t normally engage in social action’.
  • An institutional protocol or guideline might help to reduce the gap in the efforts to consciously include youth perspectives in each programmatic level.

Impact Assessment

  • Evaluation can focus on accountability and/or learning. We are often moved in the direction of accountability by our donors and other stakeholders, but we can choose how to balance our focus.
  • We may be able to use concepts like ‘learning orientation’ as proxies for organizational effectiveness where it may be hard to evaluate the exact cause and effect of change.
  • Impact assessment should go beyond evaluating an organization’s outputs and outcomes and should also account for unintended outcomes from interventions. It’s important that organizations consider the ‘first, do no harm’ principle.
  • It takes more time, but moving towards qualitative approaches such as individual interviews and focus groups can help to see who may have changed some behavior based on engagement with the organization and its programs.
  • Pablo Rodriguez-Bilella, professor at the Universidad Nacional de San Juan in Argentina, and a member of the impact assessment group notes “The stress and emphasis of this project in paradigm change and learning around impact assessment matches some of the most vital and present concerns of the evaluation community.”

Finally, Elisa Novoa, youth engagement officer of CIVICUS and the co-lead for the youth engagement group, notes:  “So far, the project has contributed to strengthen partnerships and communication among a great diversity of civil society practitioners and thinkers across the globe in a pace adapted for each of the participants to reflect, learn and test.”