GOOD
Articles

How Might We Emphasize Cost Effective Evaluation Tools?

Emphasizing cost effective evaluation tools can get us better results with less effort, enabling innovators to do more good...



Emphasizing cost effective evaluation tools can get us better results with less effort, enabling innovators to do more good with a given amount of resources. In this week's discussion, we will think broadly about the costs associated with evaluation throughout the innovation process, and suggest we rethink how we approach evaluation in order to get better results with less effort.

To understand what it means to emphasize cost effective evaluation tools, let's consider what types of costs are typically associated with evaluating the success of an innovation effort. These include (what did I miss?):
    \n
  • Costs of choosing the right methods of evaluation
  • \n
  • Costs of planning and conducting the evaluation
  • \n
  • Costs of processing the evaluation
  • \n
  • Costs of sharing and spreading the results
  • \n
  • Costs of misunderstanding consequences
  • \n
  • Costs of pilot studies and implementation
  • \n
  • Costs of not learning from our successes and failures
  • \n
  • Costs of acting (or not acting) on the evaluation results (opportunity cost of pursuing the wrong path)
  • \n
    \n









I'd like to suggest that it is the last item on this list-opportunity cost-that represents the biggest opportunity for improvement. What we can least afford is wasting everyone's time pursuing solutions that aren't working or that are less effective than other potential solutions that are on the table. This raises the question:

How do we minimize the amount of time being spent on solutions that aren't working or that should be tweaked to be more effective?

Building on this, here's an attempt to define what we mean by "cost effective" evaluation tools. Cost effective evaluation tools are (how would you define it?):
    \n
  • \nTimely: quick to deploy and minimize time wasted pursuing less effective solutions
  • \n
  • \nEfficient: use the minimum level of fidelity and rigor needed to inform decision making (sample size, refinement of prototypes, etc.)
  • \n
  • \nFocused: on the high priority / high uncertainty issues where more learning is needed to move forward (see fig. 1 below)
  • \n
  • \nShareable: results are collected, processed, and distributed in a way that tells a compelling story to the relevant stakeholders
  • \n
  • \nActionable: findings are distilled down to those which are most meaningful to quickly inform decision making to guide the ongoing innovation process
  • \n
    \n






To get this discussion started, I propose the following starter list of design principles for cost effective evaluation in innovation:

Evaluation is a mindset, not a step in the process. It should be applied throughout the innovation process to guide the work as it happens. Innovation and development are continuous processes where neat and tidy endpoints rarely exist. Evaluating the work is the work.

Begin with the end in mind. Don't start an innovation process without clear goals and priorities regarding what you are trying to accomplish and at least some ideas regarding how you will measure how well you've accomplished them. Track progress against goals and priorities.

Triage. During the innovation process, take a smart approach to evaluation, recognizing that some unresolved issues will be more important to your success than others. Try this framework for evaluation triage. Rank unresolved issues according to: low to high uncertainty, and low to high priority (see fig 1). Priority could mean importance to the success of the project or in terms of impact on decision making.


Keep Reading Show less
Trending Stories