GOOD

How Might We Celebrate Learning through Evaluation?

\r\n\r\nWhy do we evaluate? Sometimes it's for reflective validation: qualifying the success of a program after it is complete....

Why do we evaluate? Sometimes it's for reflective validation: qualifying the success of a program after it is complete. Other times it's for active learning: seeing what is working well and what could be improved, and using this insight to change things for the better.


Evaluation for validation has an important role in comparing different approaches: Which approach has the most impact? Which gives the best value for money? How can this affect strategy moving forward? The downside of this type of evaluation is that it often doesn't produce conclusions until months or years after the actual project has ended-when the opportunity to change course or affect the project outcome is gone. Evaluation for active learning, on the other hand, allows you to take action as soon as a problem is identified. In design and innovation, evaluation for learning is a natural and essential part of the process.

In its most basic form, evaluation for learning is the intuitive thought process of trial and error that occurs within a designer's mind. But in a complex project, it's often helpful to put structure around evaluation since it would be unmanageable to have too many pieces of the design under consideration at once. Structure can come in the form of a specific set of hypotheses or prototypes to evaluate, and a schedule governing the times for evaluation and the times for incorporating learnings.

One way we use evaluation at IDEO is to test design ideas at a very early stage before we invest in further development. In a project about how nurses exchange information when they change shifts, an IDEO and Kaiser team tested rough prototypes in a hospital nursing unit. Three times a day, at each shift change, we gathered feedback and iterated on the concepts. By the end of the prototyping and field testing phase, we had a concept that felt right to the nurses; the early stage evaluation had led to a product that was highly intuitive to use. It has now spread to all hospitals in the network, and has been declared a best practice by the Institute for Healthcare Improvement.

Another way we use evaluation is to guide programs at a larger scale. Ripple Effect is a collaboration between IDEO, Acumen Fund, and local water providers in India and Kenya to improve access to drinking water for the poor. (Click here for a short slide presentation on evaluation and the Ripple Effect project.) The project started in India, where we visited organizations in the field, gathered them together for a collaborative design workshop, and provided design business support as they developed new concepts in areas such as water distribution, health awareness, and safe water storage. Throughout the process we were evaluating: collecting stories from the field to inform development, seeking direct feedback from the project's users on ideas, and later evaluating the success of the pilots themselves. The evaluation not only fed into the innovations, but also informed the process as a whole. We are now conducting the second phase of work in Kenya, which has been redesigned significantly to incorporate learning from India-for example, we are increasing the amount of time we spend in the field in preparation for the pilots, and we are emphasizing the parts of the design process that the Indian organizations found most helpful.

I'd love to pose these questions to our readers:

    \n
  • When are the best times to evaluate and incorporate learnings?

  • How do you ensure that your evaluation findings shape the way interventions or programs are being implemented?

  • What are ways to evaluate a project that provide both immediate learning and longer-term validation of results?




Guest blogger Sally Madsen is a designer and project leader at IDEO. On Friday, look for a response on this topic from Lakshmi Karan of the Skoll Foundation.

Articles
via Jason S Campbell / Twitter

Conservative radio host Dennis Prager defended his use of the word "ki*e," on his show Thursday by insisting that people should be able to use the word ni**er as well.

It all started when a caller asked why he felt comfortable using the term "ki*e" while discussing bigotry while using the term "N-word" when referring to a slur against African-Americans.

Prager used the discussion to make the point that people are allowed to use anti-Jewish slurs but cannot use the N-word because "the Left" controls American culture.

Keep Reading
Politics

Step by step. 8 million steps actually. That is how recent college graduate and 22-year-old Sam Bencheghib approached his historic run across the United States. That is also how he believes we can all individually and together make a big impact on ridding the world of plastic waste.

Keep Reading
The Planet

According to the FBI, the number of sexual assaults reported during commercial flights have increased "at an alarming rate." There was a 66% increase in sexual assault on airplanes between 2014 and 2017. During that period, the number of opened FBI investigations into sexual assault on airplanes jumped from 38 to 63. And flight attendants have it worse. A survey conducted by the Association of Flight Attendants-CWA found that 70% of flight attendants had been sexually harassed while on the job, while only 7% reported it.

Keep Reading
Travel