Melanie Redman tag:www.melanieredman.com,2009-02-03://4 2010-01-19T20:54:59Z social mission collaborator Movable Type 4.34-en Constellation Model in Support of Family Literacy tag:www.melanieredman.com,2010://4.153 2010-01-19T20:44:48Z 2010-01-19T20:54:59Z Melanie Redman http://melanieredman.com Ontario Literacy Coalition (OLC).  The goal?  To imagine, convene and implement a cross-sectoral, cross-ministerial strategy in support of intergenerational family literacy programs in Ontario.  (Wow, say that 10 times fast!)  Like many umbrella groups, the Ontario Literacy Coalition is a membership-based organization that works to not only increase the capacity of its member organizations, but also to act as a convening entity in the space between the public, private and charity/non-profit sectors.  This is where I was invited in.
]]>
For example, some of the most successful family literacy programs in Ontario operate out of the Toronto District School Board with consistent, on-going funding.  Clearly the funder in this case will determine the service delivery mechanisms and priorities for moving forward.  In another community, a critical family literacy program operates out of the basement of a library and attracts young families through free lunches and vouchers for transportation.  This program constantly has to re-invent itself, even going so far as re-naming the program, in order to secure new pockets of funding.  Programs across the spectrum and those who fund them have one thing in common - they believe that intergenerational family literacy programming is a key component of healthy, vibrant communities.

So how does this wide array of stakeholders begin to work together in an organized, comprehensive way WITHOUT creating yet ANOTHER organization or government body?  I found an interesting solution in my work and association with an imaginative leader and practitioner at the Centre for Social Innovation in Toronto, Tonya Surman.  Tonya had previously encountered this conundrum of how to bring various stakeholders together and create some real movement on an issue without creating another organization.  She started looking at how to distribute responsibilities and break projects into components and hand them off to smaller constellations of stakeholders that have a natural interest or affinity in that particular piece of the work.  You assign the role of convening the larger strategy and shepherding the individual project or research constellations in the direction of the shared vision to an organization or a hired individual that acts on behalf of the collective interest.  

What about funding, you ask?  Well, the Constellation Model asks organizations that lead various project or research constellations to act as the flow-through for funds and distribute them to the constellation as the work requires.  The key here is to identify which organization sitting at a constellation table is already in a position to lead their particular chunk of work with their constellation of stakeholders and remain accountable to the entire collective of constellations.

Sound interesting?  Read my full report on applying the Constellation Model to the interests of the intergenerational family literacy stakeholders.  Apparently, HRSDC/OLES thinks it's interesting too, as they recently funded the OLC to move forward with this strategy for the next two years, acting as a convening organization.  Stay tuned as we follow the implementation of this interesting approach to collaboration.

]]> MSC Stories for Evaluation AND Marketing? I think not. tag:www.melanieredman.com,2010://4.152 2010-01-19T15:50:17Z 2010-01-19T16:24:51Z Melanie Redman http://melanieredman.com Most Significant Change Evaluation and Monitoring Framework (MSC) has been insightful, interesting and rewarding.  Having just wrapped up a two-year stint working with Intercordia Canada to imagine, test and implement MSC in the context of their international programs, I'm now settling in for a little reflection on the process.  What I see as a primary challenge of implementing MSC is the tension between story collection for evaluation purposes and story collection for marketing purposes. 


]]> earlier post where I thoroughly lay out the implementation of MSC with Intercordia, the framework is extremely rich and versatile.  The heart of MSC is the collection of stories of change at the ground level, followed by various rounds of review and story selection by identified stakeholders.  What happens in this process is that many important and insightful stories find their way into the hands of those responsible for either procuring program funding or funding programs.  Anyone with a basic understanding of program marketing or public relations would jump at the chance to have authentic tales of positive program impact/change to share with the funding community and the broader community of supporters and stakeholders.  It's natural.  I get it.  And, I'm sure you can make a case for it.  However, I will not be the one making that particular case.

The purpose of an evaluation/monitoring framework like MSC is to get to the heart of what's happening on the ground.  How do our programs and good intentions play out in the lives of real folks?  With these people as the focus of the work, I've come to believe that the stories submitted for evaluation and monitoring should not be used for marketing.  Not only is there the potential for the MSC process and story selection to include bias in favor of choosing stories that "sell," but there is also the real possibility of over-exploiting the genuine participation and intention of the story teller.  (I should mention that I participate in a global forum on the implementation of MSC in a whole slew of contexts.  This question is the most prevalent concern of those implementing the framework - having experienced the tension first-hand, I completely understand why.)

Sure, if in the process of collecting and reviewing MSC stories you run across a person who tells their story in a way that fits perfectly with the marketing goals of the organization,  it makes sense on a case-by-case basis to request a separate interview for these purposes.  The real key is to keep the intention of evaluation/monitoring and the intention of marketing separate so your evaluation results are as authentic and unbiased as possible. 

If you're interested in discussing my experiences in implementing MSC, please feel free to contact me.  I really enjoy this conversation.]]> Intercordia Canada: Evaluating Transformation of the Heart tag:www.melanieredman.com,2009://4.56 2009-06-27T23:07:09Z 2010-01-19T16:27:15Z Melanie Redman http://melanieredman.com Questions

Over the last couple of years, I've been working with one of the most compelling organizations in Canada, Intercordia.  The organization is a thoughtful off-shoot of the international L'Arche movement, steeped in the philosophy and learnings of L'Arche founder, Jean Vanier.  Intercordia is a radical model of service learning/study abroad programming for Canadian university students.  Among many areas of engagement, the folks at Intercordia came to me to help them answer the question, "How do we evaluate transformation of the heart?" 

I must admit that I am not an evaluation guru, though I've been forced over the years to be thoroughly engaged in the practice of program outcome evaluation (thank you, United Way).  Sure, I took methods courses in graduate school and know my way around a randomized trial.  What I am known for is exploring and applying creative solutions in spaces with limited resources.  You might say I can turn an egg and a beer into a champagne brunch.
]]> Intercordia's Web site: "The goal of this innovative learning program is to encourage moral responsiveness, develop respect for diversity and a valuing of other cultures, religions and socio-economic backgrounds that will enable Canadian students to attain a well-educated solidarity with others who are different."  Through this type of inner-transformation, Intercordia enhances the capacity of young Canadians to thoughtfully engage in their own communities, now and in the future.  Sounds great, eh?  So what does the program actually look like?

Students enrolled in the program take an academic course designed by their university to provide them with critical tools and lenses to engage and begin to understand an unjust world.  Simultaneously, students participate in seminars that cover everything from the practicalities of travel in the majority world to Jean Vanier's philosophy concerning encounters with the "other."  Students then spend 3 months living and working in community with marginalized people in Ghana, Swaziland, Bosnia, Ukraine, Ecuador, Honduras, Nicaragua, Dominican Republic or Brazil.  Sounds like any other service learning/study abroad program, eh?  Well, the key difference is the radical formation and re-integration that the students experience.

Intercordia encourages students away from the need/desire to DO, which is a key characteristic of most service learning programs.  Intercordia helps students recognize that at this juncture in their lives, the only thing that they really have to give is their time, presence and openness to learning.  Sure, they have volunteer jobs while they are in placement, but the main focus is on BEING in community with the people.  Intercordia students live in often extreme circumstances to learn about people, community and, dare I say it? love.  The most important thing the students experience is the formation of honest relationships with people.  The breaking down of barriers and stereotypes in this way encourages future moral responsiveness in the face of widespread injustice.

The experience isn't controlled in the ways that most service learning/study abroad programs are.  Instead, Intercordia relies on the wisdom of partnering grassroots organizations to trouble-shoot, work with host families and community employers and set the terms by which the students are present in the communities.

Challenges

The key challenge Intercordia faces in evaluating the impact of their program is a common story for innovative organizations and projects - resources.  While a UN-funded development project may have the funds and personnel required to engage in a well-rounded, long-term evaluation strategy, Intercordia does not.  In addition, evaluation gets a little more tricky because Intercordia's partners differ in their approaches and offerings.  For example, student A may be coming from the Christianity and Culture program at one university with certain theoretical frameworks and approaches, while student B comes from the Social Justice program at another university.  Then we take student A and send her to rural Ghana to live in community and teach in schools where caning is sanctioned.  Maybe we send student B to Honduras to live in community and work with families who live and work in toxic dump sites.  While student B is in Honduras, the current president is ousted in a military coup and mass civil unrest ensues.  So clearly, we have different approaches, frameworks and emergent challenges to incorporate into an evaluation framework.

Assets

Intercordia has a true culture of innovation.  From the Board and executive leadership all the way to part-time program staff, there is a history of deep-listening and learning that translates into the ability to innovate, or to engage in what I call some honest-to-goodness trial and error. The program itself is small and nimble, with no more than 50 participants in a given program year.  An additional asset is one not to be underestimated - Intercordia has some pretty fantastic, innovative funders who believe in the uniqueness of Intercordia's model of transformation.  Finally, Intercordia is an organization that loves stories. I remember when they contacted me to have our first meeting. In that session I was asked repeatedly by the Director and Board members to "Tell us a story of a time when..." This last asset is the key to my evaluative approach.

Actions

In 2007, I had the privilege of joining a 2-year working group convened by the Millennium Scholarship Foundation and the Centre for Social Innovation to think about and document strategies for evaluating networks.  It was through this group that I met Jamie Gamble from Imprint, Inc.  Jamie had recently authored "A Development Evaluation Primer" with the support of the J.W. McConnell Family Foundation (also one of Intercordia's funders).  Prior to the working group, I was already smitten with the possibilities for Development Evaluation (DE). It was through my exploration of DE that I came to know an evaluation/monitoring framework often used in sustainable international development projects known as "Most Significant Change (MSC)."

According to the framework's primary authors, Rick Davies and Jess Dart,
 
"The MSC technique is a form of participatory monitoring and evaluation.  Essentially, the process involves the collection of significant change stories emanating from the field level, and the systematic selection of the most significant of these stories by groups of designed stakeholders or staff.  The designated staff and stakeholders are initially involved by "searching" for project impact.  Once changes have been captured, various people sit down together, read the stories aloud and have regular and often in-depth discussions about the value of these reported changes."

In the case of Intercordia we ask the participants a series of open-ended questions while they are in placement.  The questions are designed in collaboration with the university partners to encourage personal reflection, the telling of personal stories, within a critical framework.  Students respond to the questions in a password-protected "member zone" of the Intercordia Web site. The responses are monitored by Intercordia staff as an additional risk-management layer (looking for "red flags" in the experience of the student). The responses to the questions become the focus of the reintegration seminar - a way of highlighting the common threads of experience for all of the participants and deepening their understanding of their stories in the larger context of the placements. The students are the engaged stakeholders who review the stories with each other and staff, discussing the experiences and reported impacts. These selected stories are then passed on to the broader community of stakeholders (Board, funders, etc.) for review and discussion. The responses to the questions offer a way to assess the extent to which Intercordia's desired outcomes are met.  The MSC framework allows real learning of the positive, negative and unintended impacts of the Intercordia program on the lives of the participants, and as Intercordia has the resources to expand the framework, on the participating partners and communities.

I must be clear that due to limited resources and numerous areas of inquiry, the piloting of the MSC program evaluation/monitoring framework was not intended to be a comprehensive program evaluation strategy, but rather one framework that builds on the current culture and systems of the organization to engage in deep-listening concerning the program impacts.  It also enables the participants to deepen their reflections while in placement and beyond - which is a key component of inner transformation and action beyond the boundaries of the program.

Learnings

I will admit that what I find so amazing about this project are the MSC stories that emerge from the program participants.  It's hard to maintain objective distance when the stories make me laugh, cry and bring me closer to the struggles and joys the students experience in placement.  Below I've highlighted a handful of key learnings in the refinement of the MSC process.

  • Early in the implementation of MSC, I discovered a vibrant MSC community of practice.  This group of global practitioners allows me to challenge my assumptions and refine the process.
  • During the first year of implementation, it was clear that the MSC reflection process with the students would be greatly enhanced through the presence of a mentor.  In year two, we piloted a mentor program that placed mentors in 3 of the placements (Swaziland, Ghana and Dominican Republic) to live and work in community with the students in order to faciliate deeper reflection.  The program has proven to be incredibly valuable in supporting students in their MSC reflection. Intercordia plans to expand the pilot mentor program in the coming years.
  • It is clear that the MSC technique can be expanded to include the collection of stories from community organizations and family members in the international placements.  We are currently crafting a strategy for years 3 and 4 that captures stories from these important stakeholders.
  • Several MSC practitioners world-wide have had great success combining the MSC technique with Participatory Video.  We believe this marriage is appropriate for Intercordia moving forward as a way to facilitate additional MSC participation in placement communities.  We are currently exploring the implementation logistics.
  • MSC stories are fantastic marketing tools where appropriate. 

*Stay tuned for updates on the expansion of this compelling framework.  If you have any questions or would like to offer suggestions, please feel free to contact me.




 
]]>