Using racial discourse communities to audit personalization algorithms

Abstract

Personalization algorithms are the information undercurrent of the digital age. They learn users’ behaviors and tailor content to individual interests and predicted tastes. These algorithms, in turn, categorize and represent these users back to society—culturally, politically, and racially. Researchers audit personalization algorithms to critique the ways bias is perpetuated within these systems. Yet, research examining the relationship between personalization algorithms and racial bias has not yet contended with the complexities of conceptualizing race. This article argues for the use of racialized discourse communities within algorithm audits, providing a way to audit algorithms that accounts for both the historical and cultural influences of race and its measurement online.

Personalization algorithms are the information undercurrent of the digital age. They shape large pieces of contemporary societies by setting public agendas through functions such as trending topics, curated news feeds, and personalized search results. They learn users’ behaviors and tailor content to individual interests and predicted tastes. These users, in turn, are categorized and represented back to society—culturally, politically, and racially—through the design and implementation of personalization algorithms.

Because the actual code behind personalization algorithms is rarely accessible or legible, researchers employ algorithm audits to make sense of how bias occurs within algorithms’ calculations. Algorithm audit studies employ an experimental design in which researchers manipulate input variables that run through an algorithm and then analyze output data to interrogate the inner logics of computational calculations (Sandvig et al., 2014). Because of the lack of democratic oversight on algorithms, Benjamin (2019, p. 190) writes that algorithm audits “scrutinize how systems operate in practice” and serve as a form of accountability for the information systems that shape our daily lives. Algorithm audits therefore offer an essential way to study algorithmic discrimination of race, gender, and any number of identity categories.

The operationalization of identity categories as variables in algorithm audits is rife with epistemological and ontological assumptions. Researchers must decide how to construct data in a way that makes each identity category “legible” for algorithms. So, auditing an algorithm for racial bias requires researchers to conceptualize how an algorithm may understand racial identity. In this article, we offer a framework for conceptualizing race as a discursive construct within the existing method of algorithm audits.

We propose using racialized discourse communities—social formations that share a set of symbolic practices, assumptions about the world, and political values that arise from and create a shared sense of racial identity—as input variables in algorithm audits of racial bias. We argue that racialized discourse communities offer a more fruitful way to study racial bias in algorithm audits than relying on the demographic and biological definitions of race commonly used in algorithm audits. Instead, racialized discourse communities afford researchers the ability to research personalization algorithms while recognizing the complexity of race as a social construct that is constantly (re-)constituted.

The utility of racialized discourse communities emerges from our own struggles in studying the relationship between race and personalization algorithms within an interdisciplinary research group of social science, humanistic, and computer science scholars (Le et al., 2019; Peterson et al., 2022). In sharing research from our respective fields, we saw epistemological differences in how algorithms are understood and studied. Computational scientists discussed algorithms as mathematical utilities that can be “optimized” and made “smart” (e.g., Blondel et al., 2008; Waltman & van Eck, 2013), while the humanists viewed algorithms as reflective of cultural change in which hegemonic power is reinscribed (e.g., Striphas, 2015). These differences led to our current focus on conceptualizing algorithmic research in a way that reflects algorithms as technologies that can be altered to function more equitably while also acknowledging the complexities of sociocultural phenomena. Therefore, we offer this article to the growing number of interdisciplinary humanistic, social scientific, and computational science research groups in hopes that its contribution will help in the crossing of epistemological boundaries and the creation of more nuanced research.

Our conception of racialized discourse communities provides researchers interested in the intersections of race and algorithms with a methodological approach with set epistemological boundaries for its conceptualization of race. Our goal is not to place limits on what constitutes a racialized discourse community but instead to offer a conceptual framework for studying race as a social construct within algorithm audits. We encourage others to think creatively about the types of online social interactions that occur in the process of racialization through the production of digitally traceable discourse.

Before demonstrating racialized discourse communities’ use in algorithm audits, we conceptualize racialized discourse communities, tracing how discourse socially constructs race. We then examine how the dominant ways of conceptualizing race in algorithmic audits generally fail to account for the social production of race. Finally, we examine how incorporating racialized discourse communities as input variables in algorithm audits allows scholars to examine racial bias within systems while maintaining the sociocultural commitment to studying race as more than a physiological or demographic construct.

Racialized discourse communities in digital media studies

Our construct of racialized discourse communities draws from social-constructivist theories that conceptualize identity categories, like race, as constantly (re-)negotiated through culture. Race in this conceptualization is fundamentally a set of contested cultural discourses that give meaning to a range of traits associated with race, including skin color (Goldberg, 1993). In other words, race connotes certain physical, psychological, and social traits, and those connotations are produced by racialized discourses. Most theorists of racialized discourse (Fiske, 1994; Gray, 1994; hooks, 1990; Stoler, 1995) see race as a contested category, where institutions and individuals with varying degrees of social power seek to shape wider perceptions and definitions of race. In this way, race is the product of “racialization,” or “the making of race, the ‘othering’ of social groups (…) the extension of racial meaning to a previously racially unclassified relationship, social practice or group” (Omi & Winant, 2015, p. 111). This process of racialization is similar to Fields and Fields’ (2022, p. 18) conception of racecraft, or the social imaginaries that inform how people give meaning to race as a social category, often with the purpose of exerting power over groups of people. The ways in which traits become associated with racial identity ties directly into the concept of discourse.

Discourse, as we use it, owes to Foucauldian approaches to analyzing culture, which differ from how the term “discourse” is used in communication and linguistics. While other approaches focus on the way that meaning and ideology work at the level of a single text or a handful of texts, Foucauldian approaches seek to understand these things on the level of society (Gill, 2018). As one of its main theorists in race and media explains, Foucauldian discourse analysis focuses “on what statements are made, rather than how they are made (…), who made them and who did not, and (…) the role of (…) media by which they were circulated” (Fiske, 1994, p. 3). Therefore, Foucauldian approaches to discourse are concerned with who speaks, what is said, and how social power functions through culture. “Speaking” here is a metaphor for a range of communicative practices. Online, these practices include posts, memes, videos, comments, and links, which are processed, filtered, and amplified by algorithms. The ability for a discourse to spread and be heard relates directly to Foucauldian definitions of power. Power, in this approach, is understood as conflicted and multivocal, with different groups vying to exert power through cultural discourses. “To make sense of the world is to exert power over it,” writes Fiske (1994, p. 3), “and to circulate that sense socially is to exert power over those who use that sense as a way of coping with their daily lives.'' Discourses, then, seek to normalize competing groups’ views of the world to further their community’s access to social resources.

Discourses emanate from discourse communities, although there seems to be little consistency in the term that Foucauldian discourse analysts use for these communities: bloc, social formation, and discourse community are all common. We prefer “discourse community” because it emphasizes the centrality of discourse to the production of social groups. Of course, everyone participates in discourse communities, as Raible and Irizarry (2007) explain in their work on Whiteness and racial identity. “[All] identities,” they write, “even racial ones, are enacted discursively and in dialogic relationships within the various discourse communities in which we participate” (p. 179). Most of us move between multiple discourse communities, only some of which would be defined as racialized discourse communities.

To summarize, then, our concept of racialized discourse communities understands racial identity as produced through discourse that is created by contesting forces that range from powerful institutions to disempowered groups. As such, racialized discourse communities share many characteristics with other conceptualizations of online communities in digital Blackness studies. Jackson et al. (2020) conceptualize “hashtag activism” as a type of counterpublic and use large-data network analysis to pinpoint central nodes in particular counterpublics. Brock (2018, 2020) studies online communities through “Critical Technocultural Discourse Analysis,” which conceptualizes online communities as produced through the interaction between the technological affordances of platforms and the forms of communication that those platforms enable. His approach focuses on digital technologies and their meanings for nondominant social groups, the specific messages that people share, and individuals’ online practices. He makes the important distinction between digital communities of practice, which share superficial commonalities such as using the same social media, and digital communities of identity, where people find community, understanding, and purpose. For our purposes of auditing algorithms for racial bias, we prefer the term “racialized discourse communities.” Unlike the term “digital community of practice,” the term racialized discourse communities signals the conflictual and politicized processes by which racial identities and communities are formed, while also applying to a wider range of digital communities than the term “counterpublic,” which primarily connotes communities that form around political issues.

Delineating racialized communities along the lines of discourse rather than, say, responses on a census form, includes and excludes different people. Discussions about the politics of these anti-essentialist definitions of race abound in anti-racist intellectual circles (hooks, 1990; Spivak, 2008) and are largely outside the scope of our analysis. Nevertheless, it is important to point out that we conceptualize the discourses themselves as racialized in such communities, not the participants. Which is to say that the specific discursive practices within a racialized community are largely recognized by those within and outside the community as racialized. An analogy would be the yearly meeting of the academic association, the National Council of Black Studies. While most participants at the meeting identify as Black, some participants do not. But everyone recognizes that the main focus of the meeting’s discourse revolves around Blackness. Similarly, particularly in its early days, the Twitter forum #BlackLivesMatter drew together mostly Black-identified participants, but differently identified allies also participated. The forum drew in non-Black participants while also excluding some Black individuals who did not share the group’s politics. Our conception of racialized discourse communities similarly seeks to employ anti-essentialist definitions of race within the method of algorithm audits.

Auditing personalization algorithms for racial bias

Technologies, such as personalization algorithms, contribute to the production and circulation of discourse by functioning as part of larger social structures that construct raced and racialized users (Feagin & Elias, 2013), often through White racial frames (Brock, 2020). In this article, we view personalization algorithms as “programmed codes” that operate within and (re)create racialized systems which users exist within and act against. For instance, research has consistently shown that the outputs algorithms produce, regardless of the type of algorithm, unequally affect poor and non-White citizens. This disproportionate effect has been shown in systems that use algorithms to determine who becomes a suspect in a criminal investigation, who gets medical attention, and how prices discriminate (Angwin et al., 2016; Obermeyer et al., 2019; Valentino-Devries et al., 2012). As such, algorithms should not be understood as neutral arbiters of racial discourse but as participants in the construction of racializing discourse that contain biases (Brock, 2020; Nakamura & Chow-White, 2012; Noble, 2018; Van Dijck, 2014).

Algorithm audits are one of the primary methods used to test for fairness¹ and discrimination within algorithms (Sandvig et al., 2014). Algorithm audits function when a researcher manipulates data, what we call input variables, that are fed into an algorithm. Once the algorithm has processed the input variables into output data, the researcher compares the output data to account for differences. An algorithm audit examining racial bias therefore employs racial cues as input variables that run through an algorithm to produce outputs that are compared for bias.

Many of the conceptualizations of race used in algorithm audits draw from audit studies, which are field experiments (Cook et al., 1979) that aim to identify discrimination within non-digital systems, such as economic markets, through a researcher’s manipulation of the system (Sandvig et al., 2014). For instance, in their audit study of racial discrimination in the labor market, Bertrand and Mullainathan (2004) sent out nearly identical fictitious resumes, which varied only by the “racial soundingness” of the names on the resumes. Using names like Allison and Brad to signal White jobseekers and Ebony and Rasheed to signal Black jobseekers, the authors found that jobseekers with “White-sounding” names were 50% more likely to receive callbacks for interviews. Similar audits using racialized names provided evidence of racial discrimination in mortgage lending, rental housing, and travel (Edelman et al., 2017; Hanson & Hawley, 2011; Hanson et al., 2016). While these studies do not audit algorithms, they provide a methodological analogue for the conceptualization of race within algorithm audits focused on racial discrimination.

Drawing from these earlier audit studies, many algorithm audits test how algorithms respond to an explicitly racial cue. Modeled after Bertrand and Mullainathan’s (2004) audit study, Sweeney (2013) found that searches for Black-sounding names returned more online ads suggestive of arrest records than searches for White-sounding names. Similarly, Noble (2018) found that Google’s search algorithms returned sexist and racist results through racially coded searches like “why are Black people so” and “Asian girls.” Baker and Potts (2013) similarly found that autocomplete results of racial groups returned negative stereotypes. These algorithm audits use racialized search terms—racialized names or words explicitly referencing racial categories—as input variables to compare differences in search results.

Beyond testing how algorithms respond to racial cues, algorithm audits also examine how algorithms respond to cues that are not explicitly racialized. For example, Noble (2018) conducted searches using ostensibly race-absent terms to demonstrate how Google Images results were racially coded. Whereas searches for “nurse” produced mostly White faces, searches for “unprofessional hairstyles” produced mostly Black faces. Similarly, Metaxa et al. (2021) collected images returned when searching for occupations, finding that people of color are underrepresented in image results. The algorithm audits use race-absent search terms² as input variables to compare racialized difference—visual markers indicative of racial groups—in search results.

These studies of algorithmic bias are undoubtedly valuable; however, they raise methodological questions about race’s operationalization in algorithm audits. For one, are racialized names a suitable proxy for race? The early audit studies discussed above used “racial-soundingness” of names to cue race, but we do not know if this cue is always legible for algorithms. Similarly, can race be deciphered by visual presentation alone? One’s presentation and physical features are associated with one’s race, but racial operationalizations based on visual presentation alone liken race to a biological variable rather than a social and cultural one. Name and visual presentation are but two of many traits associated with one’s racial identity despite neither defining racial identity. This begs the question: is there a different variable that can serve as a legible cue for race in experimental algorithm audits that reflects the discursive construction of racial identity? We argue that racialized discourse communities provide an answer.

Racialized discourse communities and algorithm audits

Race can only be experimentally studied through the cues connotatively associated with race at any given historical moment (Goar et al., 2013). For these reasons, Sen and Wasow (2016) argued that elements of race relevant to a particular study need to be identified and used as signals or cues for race. Such cues may range from names to neighborhoods to dialects to consumption patterns, which then can be used as input variables to test outcomes in experiments. Although these signals should be carefully selected for their association with a particular race (Sen & Wasow, 2016), it is important to recognize that racialized traits are not denotatively representative of an individual’s race (Hughes, 1993). Indeed, as mentioned in our conceptualization of race, traits are only connotatively associated with race through the dispersion of contested discourses about race (Fiske, 1994; Gray, 1994; Goldberg, 1993; hooks, 1990; Stoler, 1995).

We propose the use of digital traces of racialized discourse communities as input variables within algorithm audits. Digital traces are data that serve as “digital footprints which are left behind by the use of digital media” (Breiter & Hepp, 2018, p. 389). The digital traces of racialized discourse communities include but are not limited to the URLs, hashtags, language, etc. shared within online communities engaged in the process of racialization. The use of racialized discourse communities in algorithm audits highlights how racialization occurs within platforms themselves, something that much research that employs digital traces fails to account for, instead seeing discourse as occurring on platforms that function as “neutral facilitators” (Van Dijck, 2014).

Many forms of digital traces are publicly available, making the data both valuable and easily accessible. For example, “Subtle Asian Traits” (SAT) is a Facebook group composed of members who celebrate “similarities and differences within the subtle traits of Asian culture and sub-cultures” (https://www.facebook.com/groups/1343933772408499/). Alternatively, 4chan’s/pol/board is a discursive community where members routinely attack immigrants and racial minorities, thus reinforcing Whiteness (Ludemann, 2018). There are countless other examples of racialized discourse communities (e.g., #BLM, “Latinos for Trump,” etc.) online where group solidarity is predicated on racial (dis-)identification.

As such, racialized discourse communities are particularly valuable in sock puppet algorithm audits (Sandvig et al., 2014). This type of audit uses digital trace data when training profiles to emulate “users” of algorithmic systems and then compares how the algorithm personalizes each profile’s unique “experience.” The differences in the digital trace data used to construct unique profiles function as input variables in the auditing of algorithms, and differences in profiles’ personalized output function as comparable output variables necessary for uncovering potential bias. While sock puppet algorithm audits examining racial bias could train profiles based on demographic data, usernames, and profile pictures, we propose training racialized personas by having them engage with content shared within and about racialized discourse communities (e.g., joining communities, visiting URLs, producing content with relevant hashtags, etc.), racializing the profiles through participation in discourses.

Racialized discourse communities are not direct proxies for race; rather, we view racial identity formation as a communal practice made meaningful through contestations between competing discourses. For example, #BlackLivesMatter is not a direct proxy for Black racial identity, yet the hashtag serves as an articulation of anti-racist identity formation that decries the injustice of violence perpetrated against Black bodies. Similarly, #AllLivesMatter is not a direct proxy for White racial identity, instead asserting a neoliberal articulation of colorblind racism that ultimately privileges White racial identity. These two discourse communities function dialectically against each other, and therefore shape how the other groups are understood and how race functions culturally at any given moment. In algorithmic audit design, then, the use of racialized discourse communities allows for researchers to audit for racial bias by comparing results across competing racializing discourses. Each discourse highlights how participation within racialized discourse communities, either as a contributor or a passive consumer, results in personalized differences across groups.

Because the preceding discussion occurs at a high level of abstraction, we illustrate how racialized discourse communities function in algorithm audits by way of example. We employed racialized discourse communities in our work on political personalization on Google News (Le et al., 2019). Our research identified Twitter profiles that represented opposing views on the topic of immigration and collected the accounts’ shared URLs as articulations of racialized discourse communities. The anti-immigration account distributed links dedicated to “fighting the crime of White genocide,” while the pro-immigration account shared links aiming to “shift the conversation about immigrants, identity, and citizenship in a changing America.” The two profiles were considered “opposing” given not only their ideological differences but also written statements in each profile denouncing the other ideological perspective. The anti-immigration account functioned as racialized discourse advancing the hegemonic power of W(as a structural category) in America (Dyer, 1997; Lipsitz, 2006), whereas the pro-immigration account functioned as a discourse community seeking to deconstruct racial hierarchies. Again, it is important to note here that the racialized discourse communities did not function as direct proxies of race; instead, each profile was treated as an ongoing discursive project that contained competing racializing discourses. We then trained distinct user profiles to visit 50 URLs of one racialized discourse community, so that different user profiles’ digital traces embodied opposing racialized discourses. For example, the anti-immigration profile visited the Breitbart article “Somali Asylum Seeker Accused of Raping Helpless Pensioners and Killing an Elderly Woman” (Tomlinson, 2017), while the pro-immigration profile visited a Washington Post essay titled “I’m a ‘Dreamer,’ But Immigration Agents Detained Me Anyway” (Medina, 2017). After training the profiles to visit URLs reflective of anti- and pro-immigration users, the profiles performed a series of Google News searches. We tested the resulting stories for partisanship, ultimately finding evidence of personalization that reinforced the presumed partisanship of the profiles. Thus, our algorithm audit approached the topic of immigration as a social and cultural enactment of a racialized discourse as articulated through digital trace data.

Similarly, Asplund et al.’s (2020) study of racial and gender discrimination in online housing markets constructed race through engagement in specific discourses. The authors trained sock-puppet profiles to visit a “playlist” of websites that reflected the digitally traced behaviors of different racial groups. They created faux Caucasian, Black, Hispanic, and Asian profiles that each visited websites that were more likely to be visited by their racial/ethnic group than any others. In doing so, they used the varied discourses that different racial groups engaged in to represent groups of people. They found that housing ads delivered to these profiles varied based only on these racialized browsing histories. Thus, the use of racialized discourse communities has applicability in a variety of contexts.

When conducting an algorithm audit using racialized discourse communities, it is important to use controls and validation techniques to ensure the algorithm recognizes each racialized discourse community as meaningfully distinct. Researchers must validate discrimination through the operationalization of differences they will examine in the output data of algorithms. Such work involves the creation of control variables and firm definitions of differentiation within algorithmic outputs. For instance, Asplund et al. (2020) used the frequency of Spanish language ads to validate their Hispanic profiles. While their Hispanic profiles received 63 Spanish-language ads, their non-Hispanic profiles received 0. In our Google News study (2019), we employed several techniques to ensure our results would be attributable to each profile’s web browsing history. These techniques included starting with completely blank profiles running on separate Amazon EC2 cloud instances; programming our profiles to visit URLs and conduct searches at exactly the same time; waiting 11 minutes between consecutive searches to eliminate any carry-over effect; creating four copies of each profile to eliminate the chance that measurable differences could be a result of A/B testing; and adding a control profile that did not visit any websites before conducting searches. These methodological controls ensured the main factor affecting our search results was the set of URLs each profile visited before conducting their searches. Similar steps should be taken in future studies to ensure the amount and type of data used to embody racialized discourse communities is sufficient to be interpretable by the algorithm.

These recent studies demonstrate how racialized discourse communities can be used as input variables when studying racial bias through sock-puppet algorithm audits. In the next section, we demonstrate how our approach differs from other ways of studying race and personalization algorithms.

Approaches to studying racial bias in personalization algorithms

To denote how racialized discourse communities differ from existing conceptions of studying race and algorithms, we illustrate how three different approaches might be applied to a shared object of study—the relationship between race and targeted advertisements on Facebook. Below we sketch out how each approach might conceptualize race in such a study.

A computational social science approach

Research utilizing a computational social science approach to studying race and targeted ads on Facebook would treat race as a demographic or biological category that is assigned to individual users. User profiles would be constructed to align with particular racial groups through individualized identity characteristics, such as a username based on census data on names and racial identification probabilities (e.g., Chakravartty et al., 2018), profile pictures with specific skin colors (e.g., Ryu et al., 2017), or categorical data offered by users (e.g., Obermeyer et al., 2019) like their geographic location (Valentino-Devries et al., 2012). These demographic profile characteristics would be used to signal to Facebook’s algorithm that the user profiles should be interpreted as, for example, a Black user, an Asian user, and so on. Once these user profiles were created, computational social scientists could track and then compare the targeted ads received by each of these profiles, noting differences across racial groups. Although this approach would allow researchers to examine technological bias at scale, it would essentialize race as an individual demographic or biological category and fail to address race as more than a “race-inflected social situation” (Goldberg, 2009, p. 67). In other words, a common research design for computational social scientists would yield large, generalizable data about a superficial operationalization of race.

A humanistic critical approach

Unlike computational social scientists, humanistic scholars likely would employ one or more qualitative methods to study how race is constructed socially, culturally, politically, and technologically in relation to Facebook ads. This could include a historical account of racial discrimination in advertisements with a focus on how technological instruments of capitalism reinforce racial and economic inequalities. It could also include interviews with a small number of users with an interest in how they imagine and experience racialized personalization in their targeted ads. Humanistic scholars may also be interested in communal articulations of race and consumer capitalism on Facebook, similar to previous studies of particular Facebook groups that come together, share information, discuss events, or break apart based on specific technological affordances or encodings (Frederick et al., 2017; Allington, 2018). Alternatively, critical race scholarship on Facebook ads might examine how they are encoded with specific understandings of race as an embodied practice within the platform (e.g., Brock, 2018). Such an approach would offer a nuanced analysis of racial identification and contestation on Facebook but would be limited to specific users, pages, or posts, purposefully lacking the causal evidence afforded by algorithm audits. It would instead speak to scale by showcasing how representations signify social structures while they are also shaped by social structures.

A racialized discourse community approach

Our proposed approach blends the nuanced understanding of race as socially constituted from humanistic critical research with the ability to capture how racialization occurs through algorithms from computational social science. In an algorithm audit of targeted ads on Facebook using the concept of racialized discourse communities, we would train a set of user profiles that are racially distinct from each other. But rather than attempting to construct, say, Black, Asian, and White user profiles through images and names, like the computational science approach, we would construct profiles that are engaged in opposing racialized discourse communities. For example, one profile might be trained to join the Facebook group “I Love Being Black,” a racialized discourse community organized to share positive messages “that uplift the Black community” (https://www.facebook.com/lovebeingblack/). A second user profile might be trained to join the Facebook group “Sons of Confederate Veterans,” a racialized discourse community seeking “vindication of the Cause for which [the Confederacy] fought” (https://www.facebook.com/SCVOfficialPage/). Once these user profiles were created, we would track and compare targeted ads received by each of these profiles. Whereas the computational social science approach would approximate race-identified users through names, photos, and profile information intended to align with the demographics of racial groups, our approach would instead train automated profiles to join, visit links, and interact with posts on Facebook groups that represent distinct racialized discourse communities. Our approach, thus, treats race as a social, behavioral variable captured via digital trace data rather than an individual, biological variable captured via demographic information. Alternatively, whereas a critical race study might provide analysis of how Whiteness is articulated in the Facebook group “Sons of Confederate Veterans,” our approach would use these racialized discourses in test profiles to study how Facebook’s advertising algorithm interprets and responds to members of this discourse community. Our approach, thus, takes complex constructions of race as a starting point for research that is replicable³ and at scale while maintaining the goal of reducing bias shared across approaches.

To help clarify and organize the differences between computational social science, humanistic critical, and our proposed approach of using racialized discourse communities in algorithm audits, we present Table 1. In this table, we delineate how studying racialized discourse communities differs from other common approaches to studying race and algorithms. Notably, our approach uses constructivist insights in its operationalization of race to address (post-)positivist questions about how to observe and resolve bias in a system. This operationalization allows researchers to maintain the historical specificity of race’s sociocultural production within the operationalization of variables while simultaneously allowing the research to produce quantifiable results of bias in systems that the people who work directly on algorithms can easily interpret. Such an approach does not offer the specificity of critical race and humanistic approaches and requires an epistemological switch to be flipped for humanistic scholars. It asks critical scholars to view racial bias as a problem to be addressed and reduced within a specific platform (Blondel et al., 2008; Waltman & van Eck, 2013) while simultaneously recognizing that reducing bias in one platform does not resolve race as a larger conceptual category that is employed to (re)create systems of oppressions (Fields & Fields, 2022). Our approach also fails to consider how the affordances and business models of specific platforms shape which racialized discourse communities form and how they form (Brock, 2020), leaving these important threads open for extrapolation as researchers select platforms to audit. An approach based on racialized discourse communities offers a way for researchers to critique algorithmic bias while recognizing the social construction of race as a category, adding to an existing range of “abolitionist tools” scholars can draw on to reduce bias (Benjamin, 2019).

Methodological differences in researching race and algorithms

Even with the justifiable critiques of essentializing race into “ones and zeros” within digital positivist and post-positivist research, there is value in producing large-scale and replicable studies when researching algorithms and race online insomuch as they allow articulations of hierarchy to be studied quantitatively—that is, in and at the level at which these systems operate. Different methodological approaches conceptualize their scope differently. Social scientists study scope through statistically generalizable, quantifiable findings while critical scholars more frequently make claims about scope through thorough, evidence and theory-backed critiques of systemic issues and structural inequities. Issues of scope are value-laden and baked into different epistemological approaches to research and delineating how scope is conceptualized within each algorithmic research falls beyond the focus of this current article. Our approach aims to bridge these traditionally disparate areas of research, disrupt the arbitrary binaries in the broader field of communication (Scharp & Thomas, 2019), and (hopefully) allow the field to grow by both providing computations scholars with a more nuanced conception of race and allowing humanistic researchers to speak in the quantitative language of the workers who manage algorithms.

Because computer scientists work within (post-)positivist epistemologies, where algorithms are viewed as “optimizable” (Blondel et al., 2008; Waltman & van Eck, 2013), producing research that quantifiably shows issues of racial bias within their epistemological wheelhouse makes critiques of algorithms more legible to the people who create and maintain them. While we do not believe racism is “fixable” through resolving bias within individual platforms, we also recognize that algorithm audits of individual platforms can cause change within the platforms, reducing instances of that bias and encouraging more equitable computational processes and, therefore, representations. We recognize this as a limitation of our approach. We also recognize that humanistic critiques that look at representations speak to issues of scale within systems as well. Across epistemological lines, those same recognitions are not as common, however. Within our interdisciplinary research group, overcoming these epistemological boundaries was the first step in moving toward research that we could all agree upon and that we hoped could impact how bias occurs within systems in practice.

Conclusion

Beyond seeing the value of conducting research in quantitative terms, we see it as essential that computer scientists draw on humanistic research to better understand the complexities of social identity categories. Algorithms should be tested through socially responsible data selection that does not reify social categories, like race and gender, into essentialist cues, and the only way to accomplish that is through the recognition of these categories as social constructions. Our concept of racialized discourse communities moves away from the essentialized biological and demographic racial categories often used by computational researchers.

Throughout this article, we have laid out the difficulties in studying race computationally and proposed that theoretical insights from critical digital race and post-positivist scholarship can help alleviate some of the epistemological challenges of studying personalization algorithms and race. We hope this intervention provides ways for critical digital race scholars to research race and technology at a scope that is sensitive to algorithmic power while continuing to make their findings legible to the workers who shape the ways platforms represent the world to vast numbers of users. Further, we hope that (post-)positivist scholars will move toward more complex ways of understanding race as a sociocultural construct.

Data Availability

No new data were generated or analyzed in support of this research.

Funding

The research was supported by the Obermann Center for Advanced Studies at the University of Iowa and the Minerva Research Initiative (award number FA9550-20-1-0346).

Conflicts of interest: All authors declare that they have no conflicts of interest.

Endnotes

References