How Might We Celebrate Learning through Evaluation?

\r\n\r\nWhy do we evaluate? Sometimes it's for reflective validation: qualifying the success of a program after it is complete....

Why do we evaluate? Sometimes it's for reflective validation: qualifying the success of a program after it is complete. Other times it's for active learning: seeing what is working well and what could be improved, and using this insight to change things for the better.

Evaluation for validation has an important role in comparing different approaches: Which approach has the most impact? Which gives the best value for money? How can this affect strategy moving forward? The downside of this type of evaluation is that it often doesn't produce conclusions until months or years after the actual project has ended-when the opportunity to change course or affect the project outcome is gone. Evaluation for active learning, on the other hand, allows you to take action as soon as a problem is identified. In design and innovation, evaluation for learning is a natural and essential part of the process.

In its most basic form, evaluation for learning is the intuitive thought process of trial and error that occurs within a designer's mind. But in a complex project, it's often helpful to put structure around evaluation since it would be unmanageable to have too many pieces of the design under consideration at once. Structure can come in the form of a specific set of hypotheses or prototypes to evaluate, and a schedule governing the times for evaluation and the times for incorporating learnings.

One way we use evaluation at IDEO is to test design ideas at a very early stage before we invest in further development. In a project about how nurses exchange information when they change shifts, an IDEO and Kaiser team tested rough prototypes in a hospital nursing unit. Three times a day, at each shift change, we gathered feedback and iterated on the concepts. By the end of the prototyping and field testing phase, we had a concept that felt right to the nurses; the early stage evaluation had led to a product that was highly intuitive to use. It has now spread to all hospitals in the network, and has been declared a best practice by the Institute for Healthcare Improvement.

Another way we use evaluation is to guide programs at a larger scale. Ripple Effect is a collaboration between IDEO, Acumen Fund, and local water providers in India and Kenya to improve access to drinking water for the poor. (Click here for a short slide presentation on evaluation and the Ripple Effect project.) The project started in India, where we visited organizations in the field, gathered them together for a collaborative design workshop, and provided design business support as they developed new concepts in areas such as water distribution, health awareness, and safe water storage. Throughout the process we were evaluating: collecting stories from the field to inform development, seeking direct feedback from the project's users on ideas, and later evaluating the success of the pilots themselves. The evaluation not only fed into the innovations, but also informed the process as a whole. We are now conducting the second phase of work in Kenya, which has been redesigned significantly to incorporate learning from India-for example, we are increasing the amount of time we spend in the field in preparation for the pilots, and we are emphasizing the parts of the design process that the Indian organizations found most helpful.

I'd love to pose these questions to our readers:

  • When are the best times to evaluate and incorporate learnings?

  • How do you ensure that your evaluation findings shape the way interventions or programs are being implemented?

  • What are ways to evaluate a project that provide both immediate learning and longer-term validation of results?

Guest blogger Sally Madsen is a designer and project leader at IDEO. On Friday, look for a response on this topic from Lakshmi Karan of the Skoll Foundation.

Ottawa Humane Society / Flickr

The Trump Administration won't be remembered for being kind to animals.

In 2018, it launched a new effort to reinstate cruel hunting practices in Alaska that had been outlawed under Obama. Hunters will be able to shoot hibernating bear cubs, murder wolf and coyote cubs while in their dens, and use dogs to hunt black bears.

Efforts to end animal cruelty by the USDA have been curtailed as well. In 2016, under the Obama Administration, the USDA issued 4,944 animal welfare citations, in two years the numbers dropped to just 1,716.

Keep Reading Show less
via I love butter / Flickr

We often dismiss our dreams as nonsensical dispatches from the mind while we're deep asleep. But recent research proves that our dreams can definitely affect our waking lives.

People often dream about their significant others and studies show it actually affects how we behave towads them the next day.

"A lot of people don't pay attention to their dreams and are unaware of the impact they have on their state of mind," said Dylan Selterman, psychology lecturer at the University of Maryland, says according to The Huffington Post. "Now we have evidence that there is this association."

Keep Reading Show less
via Real Time with Bill Maher / YouTube and The Late Late Show with James Corden / YouTube

A controversial editorial on America's obesity epidemic and healthcare by comedian Bill Maher on his HBO show "Real Time" inspired a thoughtful, and funny, response by James Cordon. It also made for a great debate about healthcare that Americans are avoiding.

At the end of the September 6th episode of "Real Time, " Maher turned to the camera for his usual editorial and discussed how obesity is a huge part of the healthcare debate that no one is having.

"At Next Thursday's debate, one of the candidates has to say, 'The problem with our healthcare system is Americans eat shit and too much of it.' All the candidates will mention their health plans but no one will bring up the key factor: the citizens don't lift a finger to help," Maher said sternly.

Keep Reading Show less

There is no shortage of proposals from the, um, what's the word for it… huge, group of Democratic presidential candidates this year. But one may stand out from the pack as being not just bold but also necessary; during a CNN town hall about climate change Andrew Yang proposed a "green amendment" to the constitution.

Keep Reading Show less
Me Too Kit

The creator of the Me Too kit — an at home rape kit that has yet to hit the market — has come under fire as sexual assault advocates argue the kit is dangerous and misleading for women.

The kit is marketed as "the first ever at home kit for commercial use," according to the company's website. "Your experience. Your kit. Your story. Your life. Your choice. Every survivor has a story, every survivor has a voice." Customers will soon be able order one of the DIY kits in order to collect evidence "within the confines of the survivor's chosen place of safety" after an assault.

"With MeToo Kit, we are able to collect DNA samples and other tissues, which upon testing can provide the necessary time-sensitive evidence required in a court of law to identify a sexual predator's involvement with sexual assault," according to the website.

Keep Reading Show less