So I served on the advisory board of the TechImpact project done by NTEN and NPower. That project confronted a key problem faced by Nonprofit Technology Assitance Providers (NTAP):
It’s common to hear examples of how technology has helped nonprofits achieve their missions. However there are few studies that demonstrate this impact in a measurable way.The project got off to a great start but never quite got to the point of generating performance metrics for NTAPs. Well, over the past year I've been developing the performance metrics for NetSuite.org. We are basically an NTAP, so I very much looked at all the research and evaluation data on NTAPs out there.
I got a headache.
Lots of data, lots of academic mumbo jumbo (which is fine unless all you are trying to do is measure outcomes), lots of ideas and no overall simple solution for building a measurement system.
So what did I end up with?
(1) Second order social impact ( the social impact of a charity attributable to an NTAP) is hard, so
- Don't bother with it
- Allow the charity to self report on a question like "What social impact was most enabled by working with us".
- Collect narrative data on the project and, if your can afford it, do a content analysis.
I personally use #3 becuase I suspect we'll be able to do the content analysis in the future.
(2) Use a simple proxy.
- I like the Net Promoter score. Adjust the question a little to "How likely would you be to recomend XYZ to somone that needs to use technology to expand their social impact" That will generate a simple metric you can manage to (read the details, linked below)
Net Promoter
TechImpact Project
Content Analysis
1 comment:
The problem is that we don't spend enough time considering the most important question that we want to answer before coming up with a list of data to collect. If the data can't be related back to a goal or decision or action, then it is useless. 100% agree with your thoughts here related to any data collected for evaluation.
Post a Comment