When Jara Dean-Coffey talks to colleagues about philanthropic evaluation, she makes sure to include a history lesson.

The founding director of Equitable Evaluation Initiative, which sees reshaping philanthropic evaluation practice and thinking to promote equity as a door to evolving our knowledge paradigm, often starts with early 20th century tycoons like Henry Ford.

She explains that the White male magnates who exerted disproportionate influence over modern American industry also changed ideas of science, economic value, and ways to measure labor. While Edsel Ford seeded the now US$16 billion foundation that bears his surname with an initial gift of US$25,000, his father, Henry, revolutionized workplaces by championing shorter work weeks, establishing hourly wages, and, importantly, setting standards of assembly-line productivity.

These leaders of U.S. business would use the same lenses — concentrating on what was important to them, what could be counted, and perfecting ways to track payments — to determine the purpose of and to assess the impact of their philanthropic funding. Their visions of efficiency undergird how many contemporary foundations require grantees to explain and sometimes overexplain their work, and ultimately, make a case for support with numbers. This has created a power dynamic that places the onus of accountability on grant recipients and communities with little such proof or reciprocity demanded from the funder.

Though funders are not a monolith, the idea that foundations know best took firm root, an extension of corporate paternalism perfected by the fathers of big business. The structures that industry titans such as the Fords and Robert Wood Johnson (of the multinational corporation Johnson & Johnson) forged also created excluding systems of knowledge and action: not merely just where they gave, but how they steered programming by their preferences; dictated evaluation questions; judged the proper evidence of progress; collected and analyzed data; and pronounced it sound using methods akin to those used in clinical trials.

“They were industrialists,” said Dean-Coffey. “They were focused on outputs because that’s what they understood: moving things through, not the ways those things were moved, who was affected, nor what the differential effects were.”

The corporate roots of much evaluation often shocks philanthropic workers, and yet make sense as they consider how their own foundations operate, said Dean-Coffey. She knows that with investment comes power to determine terms of engagement between funder, grantees, and community; the definitions of success; and indicators that can reflect and continually reinscribe inequality. The philanthropic evaluation process also reinforces the underlying tenets of capitalism: Those who pay get a say — or at least a bigger one. In such a system, those who receive money may have to operate as supplicants with little capacity to push back against assumptions that don’t align with their work or could be outright harmful.

Changing the Narrative

It doesn’t have to be that way. There are other ways of knowing, not all quantitative, and evaluators can play a role in productively disrupting current norms or “best practices” that block both better funder-grantee-consultant relationships and deter truly transformative work, said Dean-Coffey.

EEI has evolved into a community of organizations committed to catalyzing change within their industry. More than 40 “Equitable Evaluation Framework practice partners” from a diverse group of foundations have embarked on multiple-year commitments to exploring concepts in the Equitable Evaluation Framework, which Dean-Coffey developed in 2017.

Equitable Evaluation Framework is deceptively simple. It consists of three principles: 1) advancing equity should be a foundation of evaluation; 2) such research should interrogate historical and structural impact on measurement, including how given strategies work in varied cultural contexts and populations; and 3) evaluation design and method must reflect and uphold the drive for equity. It also includes a set of orthodoxies, mindsets, tensions, and sticking points.

“I’m ridiculously situational,” Dean-Coffey said with one of her frequent laughs. But she makes the serious point that specificity in evaluation matters. “When you talk about ‘rigor,’ ‘validity,’ and ‘objectivity,’ it depends on who needs the data, what degree of rigor is needed?”

A Boom in Charitable Giving

Dean-Coffey is a seasoned consultant who’s worked with senior foundation leadership but never held a staff position at a foundation. That fact gives her the simultaneous advantage of proximity and distance. She cut her professional teeth in 1990s California as the only Black person on a large evaluation team in the Bay Area. During that time, new foundations and new millionaires proliferated in the state due to a boom in nonprofit consulting, charitable giving, and technological change. In the decade between 1999 and 2009, according to a 2012 University of Southern California study, the state’s philanthropic donations rose from US$2.8 billion to US$6 billion despite multiple economic downturns, and the number of foundations there jumped approximately 70% in the same time.

Practically speaking, that also meant rising demand for evaluation staff or consultants — and a similarly rising need for evaluators to discuss how they can effect change as their colleagues distribute funds through grant-making.

Dean-Coffey tries to help others question and sometimes unlearn tried-and-true practices of the evaluation field. Paramount among her concerns are how funders and their evaluation consultants are rarely held accountable if their approach — often imposed on grantees — doesn’t fit. Their hidden assumptions shape inquiry and often reinforce frames of deficit and dysfunction in underresourced communities. And those unspoken beliefs can obscure complexity that, if unearthed, could lead to deeper insights, understanding, and true change in service of equity.

Framing Outcomes from the Start

“Evaluations, even still, very rarely say anything unkind about the effectiveness or the thoughtfulness of the strategy of a foundation,” Dean-Coffey said. “It tends to focus primarily on the grantees, as opposed to how well thought out was [the foundation’s] understanding of the issue.”

A funder can cut checks without grasping, for example, whether it has the right grantee to achieve its goals, and those misunderstandings can prove disastrous for a nonprofit that faces unreasonable expectations for reporting or performance.

Furthermore, evaluators typically train to have professional faith that quantitative data — seen as rigorous science — is the favored way to answer funders’ questions about progress. And those questions are often generated with minimal input from beneficiaries in organizations or communities. Rarely are the beliefs and experiences of the evaluators made visible. Why did they make the choices they did?

Dean-Coffey urges her peers to step back and ask why they do what they do.

“There’s this idea that data just floats in the air, that it just exists absent of a human heart, head, and hand,” she said. “But as long as there’s a human heart, head, and hand, data is influenced by that. So there is no objectivity; there’s different degrees of subjectivity. And what I would appreciate is a greater honesty about that subjectivity.”

It’s why she likes to identify herself in professional settings to set the context: She’s a Black woman, “serial social entrepreneur,” evaluation/strategy professional, cisgender and hetero normative, Pennsylvania native, daughter of still-married retired parents, sister, wife, Sagittarian (Capricorn rising and Scorpio moon), a highly educated thinker whom White professionals often find relatable but different. And she’s clear: “I’m objective but not neutral.”

Likewise, foundations have subjectivity and history. For example, ideas about race and poverty powered early 20th century hookworm eradication programs among low-income Whites in the South, while neglecting Black health. And this work, spearheaded by what became the Rockefeller Foundation, paved the way for public health departments and initiatives in rural areas.

Recognizing Unconscious Bias

Dr. Tracy Costigan, a clinical psychologist and senior director in the office of the executive vice president at the Robert Wood Johnson Foundation (the largest U.S. foundation dedicated to health and health care), counts herself among the people who needed just such history lessons.

“I came to the foundation, appreciating the rigor [in evaluation for which the Robert Wood Johnson Foundation is known] but had stepped into philanthropy, not appreciating that it is part of our White-dominant, supremacist culture,” Costigan said. “I thought about great missions and that developing research programs would be cool.”

Her entry into foundation work in 2014 coincided with what she calls a productive “bubbling up” for the field, in which she saw more foundations beginning to earnestly grapple with equity in their evaluations.

At about the same time, Costigan met Dean-Coffey.

The Robert Wood Johnson Foundation has since become an Equitable Evaluation Framework practice partner as well as an investment partner. The depth of this relationship has evolved thinking and practices broadly across the foundation, says Costigan. She returns to EEI’s framing paper, which lists the givens that govern philanthropic logic and practice often.

“One of the orthodoxies is about evaluation being very top down,” which trickles down into how foundations select evaluators and incorporate feedback, Costigan said. “So we have to change the questions in requests for proposals to figure out how we find new evaluators with expertise in equity as a concept and as a process. How do they measure equity? … [There’s also] the frame and design of the evaluation. Does it answer our questions about impact, or does it answer questions for grantees or the beneficiaries the grantees represent?”

But she added that “there are some overarching questions that are bigger than the foundation: What are we doing to achieve a culture of health? What progress are we making? … Things like strategy, evaluation, and research are always going to exist to one extent or another, and they should. But if we start to pay attention to different things, then things move differently. What gets named and explored gets changed.”‍

Challenging Orthodoxy and Paradigm

In EEI, the individual is an important unit of change, just as important as institutions. And Dean-Coffey isn’t daunted by upending “we’ve always done it this way” objections because purposeful and compassionately challenging conversation can be a game changer. The people who come to EEI are often “innovators and early adopters” she says, referring to the vanguards of change in Everett Rogers’ diffusion of innovation theory. They are that slim percentage of people who generate ideas and move forward to make them reality when others watch, wait, and size up the risk or are waiting for internal and external conditions to support their move forward.

“What is unique and effective about the Initiative is that it’s an invitation. Folks in relationship with EEI are folks who have had some ‘Aha … oh no’ moments.” Dean-Coffey has come to expect existential crises among her collaborators, usually “a ‘dammit, I’m part of the problem’ moment.” And this is the moment when possibility appears.

She wants Equitable Evaluation Framework’s practice partners to process that guilt and move toward “humility and a lightness” that can come with shedding old paradigms and getting someplace new, where they can’t “un-see” the ways in which their work is laced with structural inequality. That lightness comes through in Dean-Coffey’s hearty laughs, the occasional blip of profanity (in French for fun), and her conviction that redirecting the professional energies of her colleagues (not just evaluators) need not be onerous work. And it’s not impossible. She’s worked with foundations that have belatedly recognized the heavy-handedness with which they’ve treated grantees and have since tried to make amends, to be greeted in return with predictable suspicion. However, she said it’s also incredibly joyful to be in creative fellowship with colleagues and to jointly rethink received wisdom.

I tell them, ‘We were all part of a belief system. And now that you know [there are other options] and use the looking glass [to see the other side], can you walk through the looking glass with us?’”

Costigan notes that Dean-Coffey’s skill and dynamism helps people take those transformative walks.

“She’s one of the smartest people I know, so thoughtful and provocative,” Costigan said. “She gets people in the room to think and helps them unearth things that are just under the surface. She inspires reflection … and then steps aside to let it happen.”

For her part, Dean-Coffey often assesses change by how far the Equitable Evaluation Framework practice partners travel in their own journeys. In her experience, women evaluators sometimes begin to see how they give away their own power without anyone trying to take it, by preemptively saying, “We can’t do that.” She asks, “Why not?” and “Did you try?” She sees colleagues who realize they don’t have to wait for board approval to experiment but can find a measure of freedom in projects in which leadership takes no interest. The lesson she likes to impart — and perhaps the lesson of her career — is that “there’s a lot more wiggle room to play and push on the orthodoxies than we think.”

Those peers will continue to organize within their institutions to move Equitable Evaluation Framework as far as they can move it, Dean-Coffey said. “We’re not expecting miracles. What we’re looking for is deep commitment and an understanding that our ways of knowing limit all of us. We are simply not as smart as we think we are.”