User Reviews
Rating: really liked it
I’m a bit obsessed with books like this. Here we get the intersection of racial stereotypes, the all-too-human created worlds of technology, the double-psychology of the oppressed (as Du Bois would say) and the strange mirror world of apparently objective truth based on what we take to be the best of human rationality. Humans are rarely more terrifying when they believe they have created a fair and rational world. And when that world has been almost exclusively created by white males – what is presented as rational, common sense and the best of all possible worlds takes a remarkable effort to overcome. The fact that this constructed world systematically disadvantages people of colour or women is often attributed to the failings of the people it fails, rather than having anything to do with the system itself.
The author runs through some examples of when technology has been, you know, a little bit racist. An interesting example is a soap dispenser that simply wouldn’t recognise non-white hands. And we are reminded of the Google Gorilla incident, where black people were identified by artificial intelligence as being apes, or the beauty assessment software that invariably choose white women as the most beautiful – the problem is that technology is simply not tested on people who might end up using it, because the people designing it don’t live in a world where people other than people who look like them exist. Technology replicates our biases, and it then reinforces them – since everyone knows that machines can’t be racist. Another book you should read on this stuff (this time focused on women) is called, conveniently enough, Invisible Women. This book, at least in part, could have been called Invisible Black People.
But this isn’t just about invisibility – it is also about the prejudices of those designing technology being normalised within the technology they develop. My favourite example is from a former Apple employee working on Siri, I’m going to quote the whole thing: “As they worked on different English dialects – Australian, Singaporean, and Indian English – he asked his boss: ‘What about African American English?’ To this his boss responded: ‘Well, Apple products are for the premium market.’” This is then immediately contrasted with the fact that Apple had just bought Dr. Dre’s Beats… The point is that if Siri isn’t taught to recognise African American English, then the software itself will define speakers of that dialect as wrong, and thereby reinforcing their perception of themselves as being outsiders, as not belonging.
As an aside, and I know the Irish are always comparing themselves to black people – and although I know I am about to do exactly that, I’ve an excuse – you see, my mother can’t use Siri either. Having been born in Norn Iron (Northern Ireland), when she says ‘Hey Siri’, Siri responds by saying, “There’s no need to be sorry”. I’m not even making that up…
There is a lovely part of this where the author says that people who have been born blind are able to identify the racial identity of people around them. Now, what is so interesting about that is that blind people are literally colour blind – I mean, literally. And yet they can still ‘see’ skin colour. So, people who think they are doing others a favour by saying they ‘don’t see race’ really are just fooling themselves.
There is a great explanation of Shirley cards in photography – and how the technologies that allow facial features to be clear in photographs, down to the chemical processes involved and the various other settings on film and cameras and so on – work well when the camera is pointed at a white face, but not so well if it is pointed at a brown and black one. The worst of this is that the industry only took the situation seriously when companies complained that photographic technology was hurting their business because it wasn’t able to clearly depict detail in chocolate or wood grain. Yeah, I think you are supposed to be outraged by that.
The stuff around prisons in the US, and around policing and the justice system, is similar to the discussion in The New Jim Crow, another book I would recommend. What is particularly focused on here, though, is what the author calls the Jim Code – technology that systematically discriminates against black people, and often simply because no black person was involved in developing the code in the first place.
One of the problems is that social stereotypes are often painfully effective, not just for structuring the thinking of those who design these little technological worlds as they respond to the simplified world and needs of the cultural other, but these social stereotypes are insidious in that they also define the limits of what people of colour are likely to think about themselves and the options available to them.
We generally assume that the world is objective and that the world itself does not discriminate. However, we forget that much of the world that we inhabit has been made by humans. Walter Benjamin implies this in his Arcades Project – where shopping arcades in Paris (effectively the first shopping malls) were created to show us a world that might be if we were to buy the items on display. His point was that what was really on display was a style of life, rather than individual items – and you were meant to place yourself inside the tableau on offer. Technology similarly creates worlds, and those worlds are even more ‘man made’ than the arcades Benjamin described. While the arcades of Paris still needed to obey the laws of nature, the worlds within computer technology has no such bounds. But those worlds are still constrained by the prejudices and stereotypes that structure our society and culture. Without understanding how apparently common sense rules systematically disadvantage certain groups of people we are likely to build systems that normalise that disadvantage.
This book provides too many examples of how this process occurs. One of the things I have been thinking a lot about lately is whether or not noticing this is actually enough. This book plays with ideas around how constructed worlds reinforce white supremacist world views. They do this because those ideas have a history and an inertia in our societies that is mostly unseen until it is explicitly looked for. But if it is unseen, this certainly does not make it innocuous. What is not noticed is taken to be normal. Again, we are back to needing to notice – but I suspect that even when we do notice how the world has been rigged against certain people, this isn’t enough to change things, because what needs to change isn’t just inside our heads, but also built into the very fabric of the societies we inhabit, into the Jim Code of our technology. I don’t mean this to bring the pessimistic conclusion that nothing can change – but rather, that for change to be possible, we need to know just how large an enemy we are facing. The internal workings of our fellow citizens are merely one aspect of that enemy – a world constructed to systematically benefit one group over others won’t change merely with a change in people’s minds. Rather, change requires changed behaviours, not merely changed opinions.
Rating: really liked it
Technology is often discussed as outside of human prejudices like racism and sexism and is often offered as a solution to the fallibility of human bias. However, Sociologist Dr. Ruha Benjamin argues that technology can still be a vehicle of racism. Her work demonstrates how technology reproduces age-old racist paradigms in subtler, but no less sinister, ways. We should be skeptical about tech companies peddling the “allure of objectivity without public accountability” (53).
She coins the term “The New Jim Code,” to account for this reproduction of racial inequalities by new technologies understood as more “objective” than the past. New technologies encode “race, ethnicity, and gender as immutable characteristics that can be measured, bought, and sold” (21). She challenges the presumed “neutrality” of technology. Take for example how cameras were designed with a bias for white people (look into the Kodak Shirley Cards from the 1950s to 1990s) or how social media algorithms disenfranchise darker skin creatives.
The failure of these technologies to interface with dark skin people is not a “glitch,” but rather a form of exclusion that occurs because of the presumption of a prototypical user who is already always imagined as white/cis/male/able bodied. Glitches can better be described as forms of “exclusion and subordination built into the ways in which priorities are established and solutions defined in the tech industry” (79). An overemphasis on “glitches,” makes technological systems appear benign, even though they are anything but. Dr. Benjamin argues that these misrecognitions are forms of evidence that alert us to how technologies operate. They “illuminate underlaying flaws in a corrupted system” (80).
In this way, she positions the “database” like a court room, an apparatus that inevitably produces racist outcomes. These technological devices continue a long history of institutional tools seeking to make race (a cultural/political division and hierarchization of humans) appear fixed and universal.
Dr. Benjamin calls for design justice: re-imagining technology to challenge inequality and affirm bodily diversity. Ultimately, Dr. Benjamin concludes that it’s not only that technologies reinforce racism, it’s that race itself can be understood as a form of technology: one that reproduces and unequal outcomes along racial lines.
Rating: really liked it
Really interesting breakdown that challenges the notion of technology as a solution to the fallibility of human bias--when it's created by people within a system of supremacy, it's going to inherit those biases. I wish there'd been a bit more on the solutions portion because, as the author makes clear, at this point, opting out of technology isn't really a viable option.
Rating: really liked it
This book expanded my understanding of technology as it relates to race.
I'll talk about one quote, but it's one of many highlights:
'something that irks me about conversations regarding naming trends is how distinctly African American names are set apart as comically “made up”'
This is similar to a point that came up this past summer: academic papers frequently point to Ebonics as the special failure case of language models. While this isn't comical, it tokenizes the dialect and treats it as an exception to a 'normal' way of speaking.
I've also been thinking about this in terms of how Indian customs are portrayed as restrictive and made up (I am talking about Indian Matchmaking!!). I don't love treating the religions/traditions around astrology and arranged marriage as a source of entertainment...feels disrespectful to profit off of the exoticism of something that really isn't that exotic.
Rating: really liked it
I picked up this book in preparation for an event where the author was speaking, now cancelled due to COVID-19. As someone that thinks about the intersection of technology and race, I was initially excited to read it. I didn't dislike like the book, especially since I agree with the premise and what the author refers to as the New Jim Code. My biggest problem while reading was that it felt like a collection of essays loosely tied together around the topic of technological injustice. I wish the event wasn't cancelled so I could hear the author speak in person.
Rating: really liked it
4.5* but i'm rounding up because this kind of scholarship is still so necessary.
an incisive examination of different facets of race and technology which intersect and overlap and reflect in so many ways (though there's still a kind of willful ignorance of professed technologists to admit those relations exist at all). i personally found this book to be a very solid synthesis of scholarship and research that touches on concepts of race and technology - Benjamin brings together a variety of more discrete research into a bigger high level view that is very illuminating.
the book gains more steam toward the end. i did wish that the design chapter had been built out more, because it felt like that discussion was what the rest of the chapters were laying the foundation for. i wanted to see that plane fly a bit more after the laying down of such a comprehensive runway.
while i appreciated the nods toward other marginalizations throughout the text in terms of impact (designed and unplanned externalities), i did think that the treatment of ableism in particular could have been more robust. i would have liked to see reference to Vilissa Thompson's work, especially #disabilitytoowhite (as a tech-platform discussion, and a conversation on visibility distinct from surveillance).
overall, though, very solid contribution to this area of study. highly recommended.
Rating: really liked it
Interesting and important, but I thought this book could have been better organised, shorter, and more focused. The core issue - lifting the mask of objectivity from highly subjective machine learning algorithms - is covered in much less space but perfectly effectively in Hannah Fry's Hello World: Being Human in the Age of Algorithms. However, the
New Jim Code metaphor is useful, and Benjamin goes much further than Fry in terms of suggesting solutions and relating the fight for algorithmic equality to the broader struggle for racial, gender and class justice. Worth a read, or at least a skim.
Rating: really liked it
A fellow book club member expressed their frustration because this book did not give any solutions. The solutions are there, they are just not quick and easy. To technologists this may sound like a real crisis because quick, cheap, and objective have been the value proposition of technology. After decades of internalizing the idea that tech can only make the world better, can technologists admit that their work looks less like innovation and more like techno status quo?
Ruha Benjamin uses the methods of critical race theory and science and technology studies to produce a thin analysis, and she justifies the thinness as an acceptance of fragility. An example of this would be her reliance on speculative fiction to explain the anxieties around killer robots and how race has been erased from the white male dominated sci-fi world while often describing the realities of life for Black, Latinx, or other marginalized groups.
In four chapters she describes how technology causes racial harm, provides the historical context has enabled that harm, and how tech endangers vulnerable groups while claiming to benefit them. She also highlights tech products, such as Appolition, as examples of technology that is designed with a purpose that factors in the socioeconomic reality that people live in. Appolition allows people to donate towards bail funds but intentionally avoids injecting money into the bloated carceral system. Once a case is complete, money is returned to the depositor and continuously recycled to help individuals. Contrast this with Promise, an app that called itself a decarceration startup but was in the business of e-carceration. Promise worked with the carceral system to make it cheaper to track and monitor people, perhaps with the notion that an ankle bracelet is a more humane alternative. But is it humane if the impact is that the extra 'efficiency' is used to track more people?
In the final chapter she describes and evaluates strategies used by activists fighting that harm. Ruha Benjamin's 'solution' asks that we dismantle the master narrative of tech and the subplot that that technology is loyal to the master. She urges technologists to employ the tools of narrative intentionally in the process of tech design. An example being a machine learning algorithm used to predict where white collar crime would occur. Midtown Manhattan and likely the neighborhood where finance folks live in your city is going to show up. You don't need ML to tell you that. What then do predictive policing algorithms do? Benjamin has written a whole book about it and I'd recommenced this book if a lot of what I'm saying sounds new.
--
July 2020
Rating: really liked it
I'm currently struggling to decide if I should rate the book with a four or a five since I can't have 4.5 stars.
Reasons why I think the book can be rated with five stars:
- Vital topic in the current times of growing automated algorithmic decisions.
- Eye-opening for someone with shallow knowledge of race issues (like me so far).
- I loved the author's definition of "the New Jim Code," which I will surely incorporate in my personal dictionary:
"Such findings demonstrate what I call "the New Jim Code": the employment of new technologies that reflect and reproduce existing inequities but that are promoted and perceived as more objective or progressive than the discriminatory systems of a previous era." (Benjamin pp. 5-6)
Reasons why I think the book can be rated with four stars:
- The structure of the book itself isn't evident. As I read in some of the reviews here in Goodreads, it looks more like a collection of essays than a book. Nothing wrong with that, just that I feel it hasn't been shown like that to me.
The writing style was coherent but not particularly engaging to someone who doesn't have English as a mother tongue (like me).
- I missed some discussion on the influence of the neoliberal capitalistic system.
- Even though the author mentions some cases outside the USA, the book is very USA-focused. Combined with the above point, it made me a bit lost in some paragraphs.
- I especially enjoyed the Introduction because the author talked about her personal experience. The rest of the book was less personal, and that made me less reveled in the story.
In the end, I decided to rate it as a 4 in Goodreads. Still, I think this is such a compelling read for anyone working with science and technology studies and cares a bit about discriminatory issues.
Rating: really liked it
would def recommend to someone looking for a 101 on current race/technology issues, but those who've read more might already be familiar with the theories and examples presented.
i especially resonated with benjamin's framing of race
as technology - one of many method bureaucrats devised to clean, cluster, and control population data.
Rating: really liked it
I think I went into this book with the wrong expectations. The “technology” piece of this book is in actuality a broad term used to encompass anything from a soap dispenser to cameras to AI to anything Silicon Valley adjacent — because of that, the majority of the content and examples of racial injustice were already familiar to me from other novels or articles I’ve read. Instead, I think this book would be better suited for someone who works outside of the tech industry/Silicon Valley, or has never taken a critical eye to either.
In terms of the book structure, the introduction alone takes up a very large chunk of the book, and essentially covers everything you read in the subsequent chapters. Each chapter is written as an essay; the book tries to loosely tie them together but I didn’t feel the structure was that intuitive. It did feel like the book lost focus on the premise, given the broad set of examples that, at times, felt outside the scope of this book to support the arguments. The last chapter was the most interesting to me and touched upon items I’d hoped the entire book would be dedicated to.
Withholding a rating for this — I ended up skimming most of this book for all the above reasons
Rating: really liked it
Dr. Ruha Benjamin is a sociologist whose primary focus of work is the relationship between innovation and equity, particularly focusing on the intersection of race, justice and technology.
On this book she focuses on a range of ways in which social hierarchies, particularly racism, are embedded in the logical layer of internet-based technologies
The author develops her concept of the "New Jim Code" to analyze how algorithms and applications can replicate or worsen racial bias. She shows how technology has the potential to hide, speed up, and deepen discrimination while appearing neutral and even benevolent when compared to the racism of a previous era.
Rating: really liked it
This book is a must read for anyone who uses technology and has not critically considered the embedded codification of racism within it. IF you have considered these things (or study them or work toward actively dismantling systems of oppression) this book will come off a little basic. It is a great introductory text that is easy to read and flags the major issues with how technology is made and incorporated into society.
Rating: really liked it
certain through lines in benjamin's book resonate with other ideas that have been bouncing around in my head as i educate myself more about the climate crisis: tech is
not the answer to broken systems; radical societal change is. she got me thinking about "coding" in a more abstract sense. i, a software engineer, may tell computers what to do, but humans can be more-or-less "programmed" via stories and systems via policies. i'm excited to talk about this with my book club and, per benjamin's recommendation, inform myself of equitable societal possibilities via reading afrofuturist works to better equip myself to resist and/or upend ways in which technology replicates and/or exacerbates IRL societal failures.
Rating: really liked it
This book couldn't be more of an important read, especially given the context post insurrection. The concept of the New Jim Code and tools of abolition in tech are so needed. Get this book. Ask more questions.
