Photo: Unsplash
Evaluators are trained and incentivized to strive for impartiality, rigor, and generalizability in their work. When practiced effectively, these skills are often assumed to ensure an evaluator’s neutrality toward the policy, intervention, or stakeholders of interest. This assumption is neither accurate nor helpful as it prevents us from considering: how can we apply values in ethical evaluative practice?
Variations of this question came up in conversations among global experts at recent convenings of the European Evaluation Society (EES) and the Evidence in Governance and Politics (EGAP) network.
Why bring values into evaluation?
For those who may balk at the combination of values and evaluation, I encourage you to read recent remarks from Ruth Levine, director of Global Development and Population at the William and Flora Hewlett Foundation and TAI member, on the moral case for evidence in policymaking and in impact evaluation. She challenged her audiences to examine and strengthen the application of “truth, distributive justice, and human progress” in daily evaluative practices.
Here is what I heard and learned in my recent travels that might help us continue to step into this challenge.
What does equity have to do with funding and producing evaluation and evidence?
Current and past leaders from the African Evaluation Association, the Community of Evaluators of South Asia, and the Latin American and Caribbean Monitoring, Evaluation, and Systematization Network put values front and center in the title of their session, “There is no Resilience without Equity: When will our Profession Finally Act to Reverse Asymmetries in Global Evaluation?”
Among other points raised, the speakers noted that the political economy of commissioning and conducting evaluations does not facilitate equity. Evaluation funding sources, teams, and even designs are mostly driven by global north frameworks and education systems. This conversation expanded the framing of “equity” to include not only assuring opportunity for evaluators of the global south, but also facilitating knowledge exchange from and within the global south.
Whose truth matters in an evaluation?
At EGAP, we discussed ethics in evaluation of governance and politics interventions, thinking beyond institutional review board requirements and the group’s current statement of research principles. EGAP is not alone in this effort with the American Evaluation Associate (AEA) having updated its guiding principles for evaluators in 2018.
Among other points, the EGAP conversation touched on both honoring the agency and dignity of the person or group being studied by the evaluation as well as the responsibility of the evaluator to report accurate truths. As Ruth notes, evidence generated can often reveal “truths [that] are specific, not universal.” While this certainly applies to those groups being studied, we can also consider the truths that those conducting and even commissioning the work bring to an evaluation.
What might this look like in practice?
Examples I observed and heard in these recent conversations include:
- Undertake a critical reflection on organizational practice and document the resulting common practices.
This can result in common principles that members of an evaluation or research community can use to translate values into the design, implementation, and dissemination of evaluations. As noted above, AEA recently updated its guiding principles with input from its global membership, and EGAP has started to do this with its cross-disciplinary and multi-sector membership. The network of academics and researchers of the Governance Lab at MIT have documented their approach to equity and inclusion.
Translating values to principles to practice resonates with ongoing efforts in philanthropy to integrate diversity, equity, and inclusion into grantmaking. This can also be reflected in funder grantmaking practice, as noted by the Ford Foundation, the Hewlett Foundation, or Luminate, for example.
- Support and engage in cross-cultural and -sectoral relationships.
There is a range of tactics and strategies from which to draw inspiration, such as exchanges incubated through EGAP Learning Days, the partnership approach of the Accountability Research Center, or the emerging global campaign of the South to South Evaluation Initiative. Relationships fostered through such approaches can also help to elevate the global south actors’ knowledge and evaluative capacity needs noted by the EES global evaluation association leaders.
- When funding or commissioning an evaluation, design a longer tail to the end of project timeline.
This practice is often cited around increasing dissemination and uptake of evaluation findings, more generally. In the context of values-driven and ethical evaluation practice, this time can be used for knowledge documentation among a more diverse set of evaluation stakeholders, and for dissemination, particularly with those populations most involved in or affected by the program and/or evaluation.
- Support knowledge and evidence consolidation beyond a given evaluation project period.
This is particularly valuable for evaluators and researchers who are typically excluded from or otherwise outsiders to global north publication platforms and outlets. And beyond formal or academic publications, there is also untold knowledge lost in unpublished null findings, rich content for future thematic or policy-relevant white papers, and instructive experiences to be shared through articles, blogs, or stories.
Why does this matter?
Keryn Hassall, an Australia-based social scientist, and presenter at the EES conference offered the premise that evaluation “is a process for valuing” what matters to people. And whether we like it or not, values are played out in and around evaluation processes and decision making.
Rather than assume away the presence of values in evaluative work, making values explicit and putting them into practice can strengthen the quality of evidence and evaluation processes. Keryn offered an excerpt from The Edge of Reason, that said paying attention to marginalized perspectives “is not to give up objectivity …, but to help achieve greater objectivity by getting a clearer, more expansive and fuller view of our shared reality.” This is surely a truth that matters.
This blog was written by Alison Miranda (Senior Learning Officer at TAI).
