In the Nonprofit World, Numbers Don’t Tell the Full Story

This post is a response to “How Might We Measure What’s Most Meaningful?” Read more of the conversation here.

The dashboard is the holy grail of many non-profit boards—a repository of brightly colored bubbles that provides a quick diagnostic on the health of an organization. Green dots signal “this is going really well—no need to put focus here” and red dots warn “focus an hour of your meeting here—deep well of trouble brewing.” Of course, dashboards are only as good as the goals on which they are predicated, the data on which they are based, and the intelligence of the people who read them. And more importantly, when you are driving on a stormy day (or even on a sunny one), I can’t imagine that many of us think it’s a good idea to stare mesmerized by our odometer or fuel gauge.

But, where to look for meaningful measurement and assessment in a noisy and complex world?

The Draper Richards Foundation selects, funds, and supports early-stage nonprofits with multi-year general operating grants and a range of other resources. We are agnostic with respect to the topics and geographies in which we fund, and instead focus our aggressive grant diligence on visionary entrepreneurial leaders, with distinctive and scalable models, and the promise of sustainable social impact.

We are not experts on the field of measurement or evaluation, but over the years we know that encouraging our portfolio members to focus on impact assessment leads to stronger programmatic outcomes, larger numbers of follow-on funding opportunities, and increased credibility in their fields of endeavor. As our portfolio has increased in size to more than 25 grantees, and portfolio members have scaled, grown their outcomes and impacts, and undertaken assessment in a more mature ways, there are trends that have emerged.

For example, single bellwether outputs can be very meaningful, especially when they reflect the values of the people being served. Agora Partnerships, a social enterprise dedicated to providing talented entrepreneurs in developing countries with tools, networks, and financing, counts the number of jobs created by its supported businesses as a significant output.  Through repeated interactions with people in the regions in which they work, employment has surfaced as a core value, making that number meaningful in that context. Mapendo International, a nonprofit that protects and cares for at-risk and forgotten refugees in Africa, tracks the number of refugees that they help be referred for resettlement to the United States or other locations; again, a strong stated desire of many of the people whom they serve. Similarly, VisionSpring, a social enterprise that reduces poverty and generates opportunity in the developing world through the sale of affordable reading glasses, tracks the number of glasses it sells. Purchasers might not know that they need glasses in the same way that a person hungers for employment, but each sale is a catalyst that improves the lives of the buyer and the seller. The three organizations track lots of things, but in the end, these numbers tell the story of their success.

The obvious point to be made here is that user-centric data is authentic and credible. It is intuitive and honors the perspectives of those receiving intervention head on. Another obvious point? In resource-constrained environments, there is no excuse not to measure—and great opportunities to measure incisively and strategically.

Another trend worth observing is that not all data that indicates change is structured, consistently numerical, or easy to weight. I think that this is especially true of data that signals the change of a system. For example, much of Agora’s work is about changing cultural perceptions of entrepreneurship in Latin America. Greater numbers of regional university course offerings on small business topics, increased press coverage of entrepreneurship in Latin America, and changes in banking policy that support small enterprise are all signals of system change not easily captured in a stock set of numbers, but worth monitoring and celebrating. For Mapendo International, Rose Mapendo, a Congolese refugee with a moving personal story of resettlement, received the 2009 UN Humanitarian of the Year Award and spoke at the White House. And recently, Mapendo has started to get calls from peer NGOs asking how they might get more involved with international refugee resettlement programs, signaling a new understanding of the needs of refugees and the importance of finding solutions to those needs in more collaborative, timely, and humane ways. Being aware that a network of players is looking at their system and considering changing the rules is important to observe, both to hasten a tipping point and bring to bear the wisdom, leverage, and options that a new equilibrium might require. And finally, for VisionSpring, watching a business and distribution landscape—populated with manufacturers, sellers, and buyers of products previously unavailable—unfold over time, signals a new economy and opportunity for many emerging consumers. It maybe be hard to attach numbers to, but nonetheless, it’s still meaningful.

So, what does this mean for the possibly-old-fashioned, yet reliable and so-very-tempting, dashboard? I’ll finish by describing a dashboard that I think works—because it is rich enough in detail to tell a story and skinny enough to hold in your head. One Acre Fund, a nonprofit that helps East African farmers to emerge from persistent poverty and hunger by increasing their farm-based incomes, has a dashboard focused on three concepts—scale, impact, and sustainability—across two focus areas—program and finances. A classic 2-by-3 matrix. Each segment of this impact grid contains a few metrics. On the program front, scale details current footprint as well as capacity for future growth, impact indicates changes in the lives of the farmers served, and sustainability measures cost and repayment figures required for the program to be cost-neutral over time. The financial dashboard has a similar construct, though with a focus on access to various types of capital. The dashboard uses red, yellow, and green dots, based on trajectory toward annual goals, but also contains textual content to add to and generate dialog (some pace-setting indicators, some system-change indicators, and a frame to provide meaning and opportunities for generative discussion).

Would I stare at it while careening down the highway? Perhaps not. But has this set of measurements signaled to our team what is meaningful for our work to be successful for the people we serve and the people we can serve in the future?  Absolutely.

Thanks for getting this far down the page. If you are still with me, I will leave you with a few questions to ponder.

• When measuring impact, who is the audience and what do they consider meaningful?
• Can meaning be derived in a number, a set of numbers, a narrative?  What is just enough and what is too much when seeking a set of indicators and measurements?
• What tools enable meaningful data to be shared, understood and compared with ease and limited cultural friction?

Anne Marie Burgoyne is the portfolio director of the Draper Richards Foundation.  The organizations described above are all DRF portfolio organizations.