Sustaining Innovation in Evaluation

This week marks the close of our blog series. For the past three months, this stimulating conversation has prompted a considerable amount of new thinking and pushed a number of sensitive buttons. On behalf of the IDEO team, I would like to thank GOOD for hosting this series and all of you who have contributed through blogging and commenting. We’ve certainly covered a great deal of ground—with contributions from design firms, foundations, social enterprises, and management consultancies. The interest in the topic has been truly remarkable—or so it seems, judging by our page views (around 14,000) and nearly constant twitter activity.

A typical tweet: The #IDEO crafted column “Innovation in Evaluation” featured on @GOOD makes an important topic sexier.

So what’s next?
The attention we’ve generated in this series is both energizing and challenging. We hope to find ways to keep this community of passionate people engaged and moving forward together.

IDEO and GOOD plan to co-host “Innovation in Evaluation” events, bringing the community together to share inspiration and maintain a positive momentum. Details are not yet finalized, but we’ve agreed to host the first of these get-togethers in California; the second will take place on the East Coast. Please let us know if you’d like to participate and watch this space for details.

But that’s not all. We’d also like to document the content we’ve generated throughout this discussion in a manner that’s befitting of GOOD and IDEO. So, watch for a shiny publication based on this series.

Key Themes
Some key themes that emerged from this conversation follow.

1. Innovation and evaluation are inseparable
The series started with this statement and the ensuing dialog has reinforced the value of intertwining innovation processes with evaluation mindsets. I have spoken with many measurement professionals, and they’ve each echoed their support for this call to action.

2. Put people at the center of evaluation
This concept was most frequently referenced because the principle is really the center point of the series. Incorporating empathy into the evaluation process ensures that our goal of impacting people doesn’t get lost in the data.

3. Emphasize learning
Overcoming the strong association that people make between “evaluation” and “validation” (after the fact) is one of our primary challenges. We need to emphasize the importance of learning (during an intervention). We also need to determine what is working well and what could be improved, and use this insight to change things for the better.

4. Navigate uncertainty
Uncertainty can be a huge challenge for social enterprises. Anyone engaging in new ideas faces unknown outcomes and risks losing funding opportunities by breaking away from the status quo. Our starting position was to encourage strategies that increase comfort under uncertain conditions, but as this conversation continued over the week, we were surprised that our readers encouraged us to embrace discomfort as an indicator that new ideas are in action.

5. Leverage informed intuition
We’ve hotly debated the issue of intuition: when and how to use it. And we’ve decided that it’s important to distinguish the “informed” intuition of experts from that of some guy with a “gut feeling.” When resources like time and money are scarce, decision-makers can become more efficient and strategic by appropriately leveraging intuition to supplement costly and time-consuming methods.

6. Take a systemic view
This blog consistently referenced the needs of stakeholders. While the constituents, or target audience, are ultimately the focus, it’s also critical to gauge the impact on all stakeholders—including the funders, society at large, and even the surrounding environment. Thinking broadly in this manner increases the likelihood that we can anticipate unintended consequences (both positive and negative).

7. Measure what’s meaningful
You get what you measure, so choose metrics that are meaningful and relevant to the impact you desire. Our readers used this topic as an opportunity to link many of the other blog themes together. We realized that “measuring what’s meaningful” could be a meta-theme for the overall series.

8. Visualize data
Perhaps the hottest topic in the series was on visualization (a topic dear to GOOD readers). It is critical to put information into a form your audience can absorb and use. Results are rendered meaningless if we’re unable to clearly communicate findings.

9. Evaluate in the appropriate time frame
Timing is everything—in car racing, fertility… and evaluation. Balancing short-term and long-term goals is a critical part of the evaluation process—and each requires a different approach.

10. Choose the right tools and methods for the task
We were sorry to report that there is no silver bullet when it comes to evaluation; there’s no one tool or method that is appropriate for every organization. Each team must design an approach for the specific questions they hope to answer.

11. Seek cost-effective tools
The issue of cost was lurking throughout the series—in fact, it underlies most conversations about evaluation. Using cost effective evaluation tools can yield better results with less effort, enabling innovators to do more good with a given amount of resources.

An invitation
In keeping with the style of the series, I will close by posing a few questions.  I invite you to continue the conversation both online and off:

• How might we stay connected as a community?
• How might we implement these themes in our daily work?
• What next steps would you like to see happen (in your organization and in our broader community)?