How Might We Celebrate Learning through Evaluation?

\r\n\r\nWhy do we evaluate? Sometimes it's for reflective validation: qualifying the success of a program after it is complete....

Why do we evaluate? Sometimes it's for reflective validation: qualifying the success of a program after it is complete. Other times it's for active learning: seeing what is working well and what could be improved, and using this insight to change things for the better.

Evaluation for validation has an important role in comparing different approaches: Which approach has the most impact? Which gives the best value for money? How can this affect strategy moving forward? The downside of this type of evaluation is that it often doesn't produce conclusions until months or years after the actual project has ended-when the opportunity to change course or affect the project outcome is gone. Evaluation for active learning, on the other hand, allows you to take action as soon as a problem is identified. In design and innovation, evaluation for learning is a natural and essential part of the process.

In its most basic form, evaluation for learning is the intuitive thought process of trial and error that occurs within a designer's mind. But in a complex project, it's often helpful to put structure around evaluation since it would be unmanageable to have too many pieces of the design under consideration at once. Structure can come in the form of a specific set of hypotheses or prototypes to evaluate, and a schedule governing the times for evaluation and the times for incorporating learnings.

One way we use evaluation at IDEO is to test design ideas at a very early stage before we invest in further development. In a project about how nurses exchange information when they change shifts, an IDEO and Kaiser team tested rough prototypes in a hospital nursing unit. Three times a day, at each shift change, we gathered feedback and iterated on the concepts. By the end of the prototyping and field testing phase, we had a concept that felt right to the nurses; the early stage evaluation had led to a product that was highly intuitive to use. It has now spread to all hospitals in the network, and has been declared a best practice by the Institute for Healthcare Improvement.

Another way we use evaluation is to guide programs at a larger scale. Ripple Effect is a collaboration between IDEO, Acumen Fund, and local water providers in India and Kenya to improve access to drinking water for the poor. (Click here for a short slide presentation on evaluation and the Ripple Effect project.) The project started in India, where we visited organizations in the field, gathered them together for a collaborative design workshop, and provided design business support as they developed new concepts in areas such as water distribution, health awareness, and safe water storage. Throughout the process we were evaluating: collecting stories from the field to inform development, seeking direct feedback from the project's users on ideas, and later evaluating the success of the pilots themselves. The evaluation not only fed into the innovations, but also informed the process as a whole. We are now conducting the second phase of work in Kenya, which has been redesigned significantly to incorporate learning from India-for example, we are increasing the amount of time we spend in the field in preparation for the pilots, and we are emphasizing the parts of the design process that the Indian organizations found most helpful.

I'd love to pose these questions to our readers:

  • When are the best times to evaluate and incorporate learnings?

  • How do you ensure that your evaluation findings shape the way interventions or programs are being implemented?

  • What are ways to evaluate a project that provide both immediate learning and longer-term validation of results?

Guest blogger Sally Madsen is a designer and project leader at IDEO. On Friday, look for a response on this topic from Lakshmi Karan of the Skoll Foundation.

AFP News Agency / Twitter

A study out of Belgium found that smart people are much less likely to be bigoted. The same study also found that people who are bigoted are more likely to overestimate their own intelligence.

A horrifying story out of Germany is a perfect example of this truth on full display: an anti-Semite was so dumb the was unable to open a door at the temple he tried to attack.

On Wednesday, October 9, congregants gathered at a synagogue in Humboldtstrasse, Germany for a Yom Kippur service, and an anti-Semite armed with explosives and carrying a rifle attempted to barge in through the door.

Keep Reading Show less
via Andi-Graf / Pixabay

The old saying goes something like, "Possessions don't make you happy." A more dire version is, "What you own, ends up owning you."

Are these old adages true or just the empty words of ancient party-poopers challenging you not to buy an iPhone 11? According to a new study of 968 young adults by the University of Arizona, being materialistic only brings us misery.

The study examined how engaging in pro-environmental behaviors affects the well-being of millenials. The study found two ways in which they modify their behaviors to help the environment: they either reduce what they consume or purchase green items.

Keep Reading Show less

One of the biggest obstacles to getting assault weapons banned in the United States is the amount of money they generate.

There were around 10 million guns manufactured in the U.S. in 2016 of which around 2 million were semiautomatic, assault-style weapons. According to the National Shooting Sports Foundation, the firearms industry's trade association, the U.S. industry's total economic impact in 2016 alone was $51 billion.

In 2016, the NRA gave over $50 million to buy support from lawmakers. When one considers the tens of millions of dollars spent on commerce and corruption, it's no wonder gun control advocates have an uphill battle.

That, of course, assumes that money can control just about anyone in the equation. However, there are a few brave souls who actually value human life over profit.

Keep Reading Show less
via Reddit and NASA / Wikimedia Commons

Trees give us a unique glimpse into our past. An examination of tree rings can show us what the climate was like in a given year. Was it a wet winter? Were there hurricanes in the summer? Did a forest fire ravage the area?

An ancient tree in New Zealand is the first to provide evidence of the near reversal of the Earth's magnetic field over 41,000 years ago.

Over the past 83 million years there have been 183 magnetic pole reversals, a process that takes about 7,000 years to complete.

Keep Reading Show less
The Planet
via Pixabay

The final episode of "The Sopranos" made a lot of people angry because it ends with mob boss Tony Soprano and his family eating at an ice cream parlor while "Don't Stop Believin'" by Journey plays in the background … and then, suddenly, the screen turns black.

Some thought the ending was a dirty trick, while others saw it as a stroke of brilliance. A popular theory is that Tony gets shot, but doesn't know it because, as his brother-in-law Bobby Baccala said, "You probably don't even hear it when it happens, right?"

So the show gives us all an idea of what it's like to die. We're here and then we're not.

Keep Reading Show less