If you have ever looked at a nonprofit’s reported beneficiary feedback survey results, you would not be blamed for raising a skeptical eyebrow. Ninety-percent-plus satisfaction scores are the norm. When you look beneath the hood, you find that nonprofits get little value from these too-positive-to-be-true surveys, and as a prospective funder, you could reasonably see it as part the donor-nonprofit “dance of deceit.”

But it would be wrong to conclude that all such surveys are a waste of time. In fact, the opposite is the case. Done well, beneficiary surveys are the simplest and most reliable compass for navigating to better results for people. Like all other organizational performance tools, there are effective and ineffective methods to do beneficiary surveys.

Here are three ways that we teach our nonprofit clients to think about feedback. They are things that you can look out for if you are a nonprofit board member or donor.

Step One: Find a great survey question. In conversation, we naturally tack back and forth until we find the answer to the larger question that we want to answer. If a nonprofit wants to know if its beneficiaries are truly benefitting from its service, its staff can get there by rambling across topics near and far, like the weather and pop culture. In a survey, however, one is locked in and static. A direct survey question like “How much do you benefit?” or “Are you satisfied with our services?” are probably not that useful, particularly when they have no alternative service option.

The trick is to find another question that is likely to get a frank and honest response and can be used as a proxy for the question one is really asking.

My favorite example of this kind of question comes from, of all places, dog food. How do you find the dog owner who will pay twice as much for a twice-as-healthy dog food? It turns out that all the obvious questions – “Would you pay more to help your dog live longer?” or “Do you care about your dog enough to give it better food, even if it costs more?”  – don’t work. People say yes to these questions, but inevitably reach for the lower-priced product once they’re at the store. The question that should be asked is: “Where does your dog sleep?” If the answer is “on my bed,” that is a premium dog food buyer. “In the bedroom” is a probable buyer. If they reply “anywhere,” there’s no chance they’re picking up the more expensive food.

Any organization that is trying to meet a human need or solve an important problem can find a great survey question like “Where does your dog sleep?” Having worked with a few hundred organizations on their survey craft, we see some compelling patterns in the “great questions” that emerge – and that is the subject of future post.

Step Two: Analyze the survey data by the important distinguishing characteristics of the people served. Aggregates and averages disguise valuable information. Search for that hidden meaning. Say that 10 percent of people get no value from the organization. OK, not wonderful, but, hey, 90 percent do! These kinds of feedback scores tend to put nonprofits to sleep, rather than ignite a search for meaning. But, cut the data by age and sex, for example, to learn that almost all those who are not getting value are younger women. Pow! That is something the nonprofit needs to investigate, and certainly must improve on.

Step Three: Triangulate survey results against other evidence being collected. One of our clients learned that beneficiaries with high scores for having a dependable adult in their community were also making must faster progress on objective economic measures the organization was tracking. Only a very small percentage of its beneficiaries reported these adult relationships, but virtually all of those who did were at the very top of the heap when it came to economic progress. After careful investigation and confirmation that this was a consistent finding, the client developed a new program to help its beneficiaries get these reliable adult relationships.

For donors, the following survey-related questions can help when you’re evaluating nonprofits that solicit your support.

  • Can they share you their own “where does your dog sleep” question?
  • Can they tell you how beneficiary feedback varies by sub-groups of the people they serve?
  • Have they discovered any strong correlations between survey findings and other evidence that they collect?