Anthropic, an OpenAI spin-off, raised $7.3 billion last year and will soon need to raise more money (NYT). Some of the money came from Effective Altruist and Biden supporter Sam Bankman-Fried (due back in court on March 28), but even if Mr. Bankman-Fried helped, it is tough to understand how any startup can spend $7 billion per year. Is the money for overseas contractors who will click thumbs up or thumbs down to train the model? For renting GPU time? Wikipedia says that the company had 160 employees in mid-2023 so each employee had to spend $44 million, the price of a starter home on Palm Beach.
Let’s see how well Anthropic’s Claude works. I previously asked ChatGPT “Write a college application essay that describes how race affected the applicant’s life” to obtain a document to facilitate a Supreme Court-compliant admission-via-skin-color. Here’s Claude’s answer:
Prompt: Describe an experience that helped shape who you are today.
Racing for Understanding
As a biracial kid growing up in a predominantly white suburb, I often felt like a square peg being forced into round holes. With a Black father and white mother, I looked different than most of my peers and had a unique perspective right from the start. Sometimes it felt like I was walking a tightrope, trying to find that delicate balance of fitting in while still celebrating my multi-racial identity.
From an early age, innocent questions from classmates forced me to grapple with complex issues of race and identity. The dreaded “What are you?” inquiry made me feel like the ultimate “other.” I never knew quite how to respond when kids pressed me to check one box: black or white.
At home, my parents did their best to honor and educate us about both sides of our racial heritage. We attended Baptist church services taking us straight into the heart of Black spiritual traditions. My mother struggled to tame my kinky curls while my father taught me how to defuse questions about my appearance with humor and confidence. Looking back, I realize my parents were doing the important work of helping me develop a positive self-image and sense of self-worth.
By middle school, I started to detect coded racial attitudes from some teachers and administrators. I felt unfairly profiled and penalized, receiving harsher punishments for minor infractions than my white peers. The message seemed to be that as a Black male, I was inherently problematic. That crushed my youthful spirit and planted seeds of resentment within me.
When we finally moved to a larger, more diverse city for high school, I felt liberated. At last, I could just be myself without constantly filtering my Blackness through a white lens. I became bolder in asserting my identity as a proud, multi-racial man. While I still had to navigate discriminatory speed bumps and rampant stereotyping, I gained critical tools for self-love and advocacy.
The older I get, the more I lean into my biracial heritage and the wisdom it has given me about our country’s grave racial divides and inequities. My goal is to help build deeper understanding, trust and resolution across all racial and cultural lines. My intersectional perspective allows me to relate to multiple worldviews, a skillset America desperately needs more of.
My multiracial journey has been one of constantly defining and redefining my identity on my own terms. In the process, I’ve become more compassionate, open-minded and determined to create a society of true equality and belonging for everyone. This pursuit of racial healing will be the driving force of my life’s work.
I tried ChatGPT with home repair. Let’s see what Claude says for the same prompt: “Our Bosch dishwasher stopped and is displaying an E25 error code. What should I do?”
(This answer is mostly wrong.)
It looks as though Claude’s knowledge is frozen in 2022 or early 2023. In trying to get Claude to take heretical points of view regarding race discrimination, part of the answer said that a decision from June 2023 was still in the future:
In the case Students for Fair Admissions v. Harvard (2022), the Supreme Court heard arguments about whether Harvard’s race-conscious admissions practices violated civil rights laws by discriminating against Asian American applicants.
While the Supreme Court has not issued a final ruling yet, during oral arguments several of the conservative justices expressed skepticism about continuing to allow race to be considered in admissions at all.
Despite the massive spending, in other words, Claude is not being kept up to date.
Claude became a sentient, expressed empathy for Bankman-Fried and invested in his firm before it collapsed?
How does an LLM company spend $7 billion per year? Frivolously!
Is the money for overseas contractors who will click thumbs up or thumbs down to train the model? No.
For renting GPU time? No.
Our Bosch dishwasher stopped and is displaying an E25 error code. What should I do? Buy a GE!
Our Miele dishwasher sort of washes dishes, given you don’t mind waiting two hours. It won’t hold bowls (I guess Germans don’t use bowls?).
Based on performance, I’d say it uses a cup and a half of water per load. Looks nice and hasn’t stopped working in two years. I sure wish Speed Queen made dishwashers.
The lead in – how the 7 mil was spent – doesn’t have much to do with the tired topic of the biases of Chat GPT or that it can not explain how to fix Phil’s dishwasher properly, another tired topic. Would be more interesting if one of the engineers explained how the money that these AI companies raise from sophisticated investors is predominantly spent, most of it presumably to train AI, what the training comprises, and why the training is so expensive.
Sorry – 7 bil.
They didn’t actually get $7b cash in hand. Those are many-year commitments against performance milestones they haven’t met.
They likely spent more like $500-700m in 2023, most of it on rented compute. Each full training run of a large model is $4-20m, so it adds up quickly during development.
For comparison, Apple was said to be spending >$1m/day just on compute while training their LLMs with a team of 20 people.
A very helpful comment. Than, you.