Great article!  I think that the core of this issue is summed up in your first ‘question to ponder’:  when measuring impact, who is the audience and what do they consider meaningful? I’m really interested in helping social impact organizations define and develop their own metrics for success.  In doing so, my biggest question is always “what does success mean to YOU?” – an agencies’ metrics need to match that specific definition of success.  To borrow an example from Jim Collins, if a successful orchestra is one that plays technically complicated music very well, then ticket sales is a bad measure of success. Too many organizations improperly match metrics with success, and consequently end up with meaningless evaluation.   I wouldn’t write off the notion of dashboards entirely, but I think the problem with them is two-fold:  1.  most dashboards are much too complicated to be fully understood or practical to use.  2.  most dashboards mismatch the metric and the definition of success.  Spending the time to develop custom metrics makes a huge difference here.Ideally, I think that every organization should have its own set of mission-specific metrics that match their own unique definition of success.  However, I recognize the challenge that this poses for comparison across agencies.  If that comparison is necessary, (for granting agencies, etc), then secondary metrics can certainly be developed and used.  However, metrics designed for comparative purposes should never be the principal metrics used in an (non-comparison-focused) organization.

My biggest challenge in promoting change has always been convincing people that evaluation needs to be an ongoing process.  Too often, organizations or teams come up with an idea, execute it, and refuse to do any evaluation until “the end” (the end of the launch, the end of the program, etc).  They either commit to an idea 0 or 100%, with no middle ground.The problem with this is the incredible missed opportunity to refine their idea, make it stronger and increase its impact – and honestly, how often are ideas perfect the very first time?  When change becomes all or nothing, and there is no room for refinement during the process, it is too big and too risky for most people; this is why people and organizations are so resistant to change. Resistance to change is a direct consequence of evaluating only at the end.Constant evaluation provides organizations the ability to adjust and change course when needed.  It provides organizations with the chance to make change more impactful, and it also allows them to stop bad changes before suffering the full hurt of a completed mistake.  In effect, constant evaluation makes change less risky – since risk is exactly what we’re so worried about to begin with, this should be a no-brainer.

As an MBA student interested in helping nonprofit organizations define and measure success, I find this topic fascinating.  I can’t wait to hear more about the forum and IDEO’s contributions in this area!

Great article!  I think that the core of this issue is summed up in your first ‘question to ponder’:  when measuring impact, who is the audience and what do they consider meaningful? I’m really interested in helping social impact organizations define and develop their own metrics for success.  In doing so, my biggest question is always “what does success mean to YOU?” – an agencies’ metrics need to match that specific definition of success.  To borrow an example from Jim Collins, if a successful orchestra is one that plays technically complicated music very well, then ticket sales is a bad measure of success. Too many organizations improperly match metrics with success, and consequently end up with meaningless evaluation.   I wouldn’t write off the notion of dashboards entirely, but I think the problem with them is two-fold:  1.  most dashboards are much too complicated to be fully understood or practical to use.  2.  most dashboards mismatch the metric and the definition of success.  Spending the time to develop custom metrics makes a huge difference here.Ideally, I think that every organization should have its own set of mission-specific metrics that match their own unique definition of success.  However, I recognize the challenge that this poses for comparison across agencies.  If that comparison is necessary, (for granting agencies, etc), then secondary metrics can certainly be developed and used.  However, metrics designed for comparative purposes should never be the principal metrics used in an (non-comparison-focused) organization.

My biggest challenge in promoting change has always been convincing people that evaluation needs to be an ongoing process.  Too often, organizations or teams come up with an idea, execute it, and refuse to do any evaluation until “the end” (the end of the launch, the end of the program, etc).  They either commit to an idea 0 or 100%, with no middle ground.The problem with this is the incredible missed opportunity to refine their idea, make it stronger and increase its impact – and honestly, how often are ideas perfect the very first time?  When change becomes all or nothing, and there is no room for refinement during the process, it is too big and too risky for most people; this is why people and organizations are so resistant to change. Resistance to change is a direct consequence of evaluating only at the end.Constant evaluation provides organizations the ability to adjust and change course when needed.  It provides organizations with the chance to make change more impactful, and it also allows them to stop bad changes before suffering the full hurt of a completed mistake.  In effect, constant evaluation makes change less risky – since risk is exactly what we’re so worried about to begin with, this should be a no-brainer.

As an MBA student interested in helping nonprofit organizations define and measure success, I find this topic fascinating.  I can’t wait to hear more about the forum and IDEO’s contributions in this area!