Data, though seemingly an objective collection of facts, can reflect human bias. Depending on what data is being collected and how it is being analyzed, it may contribute to models that harm or help. For example: Looking at one piece of data, your zip code, your bank may give you a high interest rate on a loan, even if your credit is excellent. Hardworking teachers may lose their jobs if students’ test scores are the main criterion by which they are evaluated. Because of factors like confirmation bias and mixing up correlation with causation, data can be misread and misapplied.

From these examples, nonprofits can learn to be mindful of biases that may be harmful instead of helpful when collecting data about clients, donors, and impact.

Weapons of Math Destruction

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O’Neil makes the case that some mathematical models that are prevalent throughout society – models that rule out residents of certain zip codes for prime loan rates, personality tests that deny people jobs, predictive policing, and more – can cause great harm, especially to people who are already disadvantaged.

O’Neil plays on the term “weapons of mass destruction” to point out the danger of mathematical models that are built opaquely, implemented widely, and unchanging. Their inner workings are not revealed to the public, so people don’t know what data is being collected about them or how it is being used. These algorithms can perpetuate racism and “keep the poor poor” in a multitude of ways.

According to O’Neil, a defining characteristic of a WMD is that it causes toxic feedback loops that are essentially confirmation bias set in motion. Take predictive policing as an example. Predictive policing models send police officers to patrol neighborhoods that have had higher arrest rates than others, and greater police presence leads to more arrests, which means the neighborhood will continue to have high arrest rates and more police presence. The problem is that the data put into these models in the first place amounts to racial profiling. In many cases, black people are more likely than white people to be arrested for the same crimes, and while those higher arrest rates do not necessarily mean higher crime rates, they still lead to more policing and thus more arrests.

What can WMDs teach us about using nonprofit data?

You use data to target potential donors, to track who is giving and how much and when. Data can tell you how your year-end campaign is doing. You measure your nonprofit’s impact to prove to your funders that your nonprofit is worth funding. You keep a wide spectrum of information about your stakeholders in your database so you can continue to strengthen your relationship with them. You track all sorts of metrics in your nonprofit dashboard. Nonprofits can learn from the damage done by WMDs and use their data better!

  • Don’t mix up correlation and causation. Let’s say you discover that the more money your most generous donors give, the less likely they are to be regular users of Twitter. Do they give you more money because they’re not on Twitter? Probably not. Does this mean you should stop using Twitter and focus all of your fundraising attention elsewhere? Again, probably not; Twitter is invaluable for spreading the word about your organization to potential donors and clients, and even if you don’t have big donors following your tweets now, you can build toward that in the future. Facts may be correlated, but one does not necessarily cause the other.
  • Be flexible and willing to learn. If the data shows, for example, that contrary to your expectations, your nonprofit receives more donations from middle-class women than from upper-class men, use that data to strengthen your fundraising strategy. WMDs are mathematical models that do not learn. You, however, can learn and correct your course.
  • On that note: Confirmation bias is real. Without even being aware of it, people can interpret information in a way that confirms our existing beliefs, blinding ourselves to reality. In essence, we see what we are looking for. We are less likely to remember information that contradicts our beliefs, even though this information can be crucial. To avoid confirmation bias, we must remember that as humans we have a subjective viewpoint. Keeping aware of your subjectivity, and constantly challenging what you think you know, is crucial. It can also be a good idea to have colleagues or a third party review your research and check for bias. Read 5 tips for avoiding confirmation bias on the GutCheck blog.
  • Data can motivate not just funders, but employees too. Nonprofits get a lot of use out of donor data and impact measurement for external audiences, but what about using your data to show employees what a great job they are doing? Celebrate the small wins to show your team how they are helping the big picture. Have fun with it – even tracking the number of envelopes stuffed can be satisfying for employees to see!
  • Follow #npdata on Twitter for ongoing information on best practices for gathering and using nonprofit data.

Sources and Additional Reading

Weapons of Math Destruction by Cathy O’Neil (Penguin Random House)
Mathbabe.org (Cathy O’Neil’s website)
Is Your Big Data Project a “Weapon of Math Destruction”? (IEEE)
Predictive policing is ‘technological racism’ (Shaun King, New York Daily News)
Better Than Cake: Using Data to Celebrate Team Wins (Nonprofit Technology Network)
Research Best Practice: Avoiding Confirmation Bias (GutCheck)

  • Zach Shefska November 29, 2016, 10:34 am

    Great article! And thank you for sharing Cathy O’Neil’s book, I will have to check that out!