Measuring Outcomes for Non-profit Orgs

Jane Reisman presented an overview workshop on outcome-based evaluation for non-profit organizations this week in Seattle.  Her methodology makes sense at the enterprise level and is different from the metrics we often use to measure individual, team, or business unit performance.  Both are important.

Jane asserts that to evaluate the success of a non-profit program an assessment must move away from a focus on activities, or what staff members do (e.g. number of counseling hours provided, number of workshops delivered, etc).  Now, outcome-based evaluation focuses on these key questions: "How has a program made a difference? and "How are the lives of program participants better as a result of the program?"  In other words, "So what?"

For me, a specific example comes to mind from the annual Microcredit Summit report where they make a healthy introspection of whether their aggressive goals to eliminate extreme poverty are appropriate.

There are two primary questions in measuring progress. One is whether it can be reliably said that change has occurred and the other is the more difficult question of causality. It might be determined that a certain number of families have moved above the US$1 a day threshold, but was microfinance a primary cause of this positive change? The idea of setting aside the question of causality was clearly articulated by New York University professor Jonathan Morduch who wrote the following in a longer message to the Campaign:

"In the perfect world, we could set goals where we could really nail causality and say that microcredit would be the cause of the improvements sought. Here, though, I think it would be enough to put causality aside in terms of setting goals. It would be a major step simply that 100 million households who are microfinance customers move from being under $1/day to over the line—even if microfinance is only 50% responsible or only 5% responsible. Setting the goal in terms of making progress (but not specifying that the progress would necessarily be due on net to microfinance) would still have the role of placing an emphasis on the very poor and on raising living standards. In short: your wording of Goal 2 makes sense. In the end, what matters is sustained poverty reduction, not the spread of microfinance per se or of particular microfinance strategies."

At first blush, accountability might seem to be sacrificed as causality is relaxed.  Afterall, the goals of outcome-based evaluation are:

• To provide accountability
• To improve program quality
• To support decision-making about resource allocation
• To help programs market themselves

This makes developing a "logic model" the essential foundation for any program.  A logic model ties together these relationships in explicit ways:

Resources -> Activities -> Outputs -> Outcomes -> Goals

Jane points out that non-profit agencies cannot do almost any outcome you might wish for!  They can influence a change.  They cannot control that change.  With the foundation of a logic model you can then move to identify simple "indicators" of success and then to the tools to measure them.

For more information about outcome-based evaluation, you can contact Jane here:
Jane Reisman, Ph.D., Organizational Research Services