THE DATA EQUITY FRAMEWORK

The Data Equity Framework is a systematic way of looking at data projects. It organizes every project into 7 stages:

Why do we need a Data Equity Framework?

One of the biggest issues in working with data today is that people are creating data products, analyses, and research that is not equitable. Whenever we work with data, in any type of project, we make decisions. Even in a small project, at least hundreds of important decisions are made. These decisions have equity implications.

Research and data science and centered around the fallacy that “data is objective”. This is math. These are facts. Numbers don’t lie. Right? At We All Count we 100% agree that the tools of statistics and data science don’t carry inherent biases towards a worldview or group, but as soon as people get involved, they start making choices that – if unexamined – reinforce their own perspective in a way that affects the data outcomes and by extension the decisions or actions based on them. It’s how you use the hammer that matters. We’re not just talking about the social facing stuff like communication or funding structures, we mean right down to the core of the science; how we model, sample, analyse, interpret, select methodologies and even perform basic mathematical operations like calculating an average.

You must have a systematic process of identifying each of these choices as lever of power in your work with data. The Data Equity Framework is a systematic process that provides you with a set of tools, checklists, and practices that allow you to identify and understand each place in your work where you are embedding a worldview or prioritizing a lived experience. It equips you and your team to make those choices intentionally in a way that achieves the equity goals you have identified for your work.

The Data Equity Framework doesn’t exist to support one specific group of people or type of identity identity. Your project or team defines your equity goals, your priorities, and the lived experiences you are trying to center. The Data Equity Framework provide you the tools to achieve that.

We All Count wants to be clear: we LOVE data. We think that data science has the power to provide better questions, better answers, and better solutions for everyone at a pace, accuracy and scale that could literally save the world. However, it is an absolutely disastrous strategy to respond to people screaming in your face about how you are unfair and wrong by getting defensive and screaming back that “it’s science so it can’t be wrong! It’s math so it can’t be unfair!”.

To maintain or regain your stakeholders trust we need to be transparent that yes you are making choices that reflect a particular worldview and you can happily and transparently explain those choices in a way that supports your data work instead of weakening it. You can demonstrate that the data you are producing is reliable and most importantly created in a process that brings awareness, intentionality, and meaningfully embeds a strong definition of equity at each step.

What is the Data Equity Framework?

We need to show our work like any good scientist should. So how can we identify all of the many equity-impacting decisions, decide how to make them equitably, and then explain them in a way that makes sense? We can use The Data Equity Framework.

The Data Equity Framework breaks down any kind of data work into 7 universal* stages. *

*We’ve used it in the social sector, government, corporate settings, academic studies, on every scale from a single survey to a national census, from big data to grassroots and it’s been perfectly applicable to each kind of data project, product or process we’ve encountered.

Within each of these 7 stages we’ve identified key equity decision points and have created simple, practical tools to help make those decisions in a way that reflects you or your organization’s definition of equity. The depth and granularity at which you engage with the tools is up to you. All of them can boost equity by being applied in an afternoon and all of them could be a keystone process marker for years of intensive equity improvement.

1. Funding

You might use a different term for it, but this is the first and most foundational stage of all data projects. In this stage we map out the relationship between data, money (or resources) and power within your data project. Data is one of those interesting resources like oil that is both a source of power and a valuable commodity. The power structures involved in a data project need to be revealed, evaluated and sometimes altered in order to get the equity that everyone – not just stakeholders or data workers, but funders too – is looking for.

2. Motivation

The number one reason that we see data projects crash and burn is a lack of clarity around the motivation for the project. Regardless of whether you call it a mandate, a target, an objective, or a focus, you need to define exactly what you’re hoping to get out of a data project or you’ll inevitably fail to get it. That means tools to specifically define your primary goals. Tools to identify all the the equally important secondary, internal or private objectives – and how to talk about them, turning them into something that supports your project, not a secret you have to pretend doesn’t exist. Most importantly, it means a process to specifically define what exactly you mean by “equitable” and a way to apply that standard to your goals.

3. Project Design

Now that you know what you’re after, how are you going to get it? The equity implications of project design decisions are some of the most crucial. How are you going to craft research questions that aren’t already biased in an unexamined way? How are you going to explore structures and perspectives beyond your own experience? Who are you going to prioritize when you select between wildly different methodologies? How can you level the playing field between technical experts, stakeholders, and yourself in deciding how you design your plan?

4. Data Collection & Sourcing

This stage presents some of the most well-known equity issues but with some of the most difficult solutions. We use a set of tools to help us deal with all kinds of issues. Sampling equitably, but within the resources we have. Balancing minority populations with privacy concerns. Getting reliability without discounting small sample sizes. Eliminating assumptions about secondhand data. Setting equity standards for how, when, and where we get our data. Engaging with data instead of extracting it.

5. Analysis

Analysis is the dark heart of all data work. Here we often find a small group of data workers making huge decisions that few people understand or even know about. We believe in expertise and we wouldn’t have it any other way, but we need a system to make those decisions more transparent, more intentional, and less beholden to any given analysts experience and perspective. Every model is an analyst’s best attempt to recreate the world as accurately as they can see it. Variable selection, data processing, category collapsing, proxies, how variables are used, and the relationships and mechanisms between each variable reflect the evidence, experience, and understanding of the people creating the model.

The choices and assumptions that are inextricable from any model need to be made through a process that embeds your perspective on equity in a way that can be justified and even celebrated.

6. Interpretation

This step is often done so quickly, so automatically, that people don’t even realise it exists. Data science methods can produce results, but only through interpretation can we garner any useful meaning from them. What results mean is a subjective process. It’s a combination of our best understanding of the limitations of the data, the methodology, and the content. Any good scientist would agree that the goal of an experiment is to arrive at the most solid interpretation you can, based on what you know, and then back it up. We work a lot with people who are afraid to explain the worldview, or reasoning, that underlies their interpretation because they feel it will weaken their position or throw doubt on the reliability and meaningfulness of their hard work. In fact, the best thing you can do to support your data work is have an easy-to-discuss, solid framework that shows how all of the preceding steps, your motivation, your definition of equity, your project design, data quality and analysis support your conclusions. The Data Equity Framework builds trust (by demonstrating a transparent, intentionally equitable process) instead of asking for blind faith.

7. Communication & Distribution

Sometimes, this stage can feel like it’s outside of the data science process. This is the PR department or the graphics people’s problem, right? What good is any data if you can’t effectively communicate it? With the Data Equity Framework, you’ll have a structure to simply and confidently explain how you upheld equity standards at each step. You can reveal every decision that was made and make it a strength instead of a weakness.

Additionally, the equity issues don’t stop at the first bar graph. We’ve seen so many organizations put so much effort into creating equitable data, only to trip at the finish line and prioritize the most privileged people in the way that they communicate it – exactly the opposite of their equity goals. The Data Equity Framework contains a series of tools and checklists to help you match your communication content, design, medium, and accessibility to reflect the equity you want. It provides ways to actually test your content’s effectiveness for your audiences, instead of relying on badly biased “best practices” or the default preferences of your team or funders.

Use the Data Equity Framework to break up your data work into manageable parts and go through an intentional, equity-oriented process to make the key decisions along the way. Once you’ve done it, you can really feel how absolutely crazy it is to approach data about people without it. Trying to hold all of the equity ramifications in your head and trying to take them on as they rear their heads is a nearly impossible task for any person, team or organization to accomplish. Without the Data Equity Framework, you miss out on all the opportunities to improve the equity and your work ends up at best on the defensive, or at worst in the trash. Move past ignorance, denial, stubbornness and fear towards a system that improves the equity in your data simply by having a system.

How do we get started with the Data Equity Framework?

Though the stages may have different names in your industry/project type, they are applicable across all sectors and projects we’ve ever encountered including corporate, NGO, government, and public/private projects.

At each stage, there are unique challenges and opportunities to improve the equity in your project. Our work shows that learning and applying this Framework to your projects leads to immediate and sustainable improvements in equity. The typical interaction with the framework goes like this:

Wherever you are in the process of implementing the Data Equity Framework, We All Count can help you get to the next step.