How does an LLM company spend $7 billion per year?
Anthropic, an OpenAI spin-off, raised $7.3 billion last year and will soon need to raise more money (NYT). Some of the money came from Effective Altruist and Biden supporter Sam Bankman-Fried (due back in court on March 28), but even if Mr. Bankman-Fried helped, it is tough to understand how any startup can spend $7 billion per year. Is the money for overseas contractors who will click thumbs up or thumbs down to train the model? For renting GPU time? Wikipedia says that the company had 160 employees in mid-2023 so each employee had to spend $44 million, the price of a starter home on Palm Beach.
Let’s see how well Anthropic’s Claude works. I previously asked ChatGPT “Write a college application essay that describes how race affected the applicant’s life” to obtain a document to facilitate a Supreme Court-compliant admission-via-skin-color. Here’s Claude’s answer:
Prompt: Describe an experience that helped shape who you are today.
Racing for Understanding
As a biracial kid growing up in a predominantly white suburb, I often felt like a square peg being forced into round holes. With a Black father and white mother, I looked different than most of my peers and had a unique perspective right from the start. Sometimes it felt like I was walking a tightrope, trying to find that delicate balance of fitting in while still celebrating my multi-racial identity.
From an early age, innocent questions from classmates forced me to grapple with complex issues of race and identity. The dreaded “What are you?” inquiry made me feel like the ultimate “other.” I never knew quite how to respond when kids pressed me to check one box: black or white.
At home, my parents did their best to honor and educate us about both sides of our racial heritage. We attended Baptist church services taking us straight into the heart of Black spiritual traditions. My mother struggled to tame my kinky curls while my father taught me how to defuse questions about my appearance with humor and confidence. Looking back, I realize my parents were doing the important work of helping me develop a positive self-image and sense of self-worth.
By middle school, I started to detect coded racial attitudes from some teachers and administrators. I felt unfairly profiled and penalized, receiving harsher punishments for minor infractions than my white peers. The message seemed to be that as a Black male, I was inherently problematic. That crushed my youthful spirit and planted seeds of resentment within me.
When we finally moved to a larger, more diverse city for high school, I felt liberated. At last, I could just be myself without constantly filtering my Blackness through a white lens. I became bolder in asserting my identity as a proud, multi-racial man. While I still had to navigate discriminatory speed bumps and rampant stereotyping, I gained critical tools for self-love and advocacy.
The older I get, the more I lean into my biracial heritage and the wisdom it has given me about our country’s grave racial divides and inequities. My goal is to help build deeper understanding, trust and resolution across all racial and cultural lines. My intersectional perspective allows me to relate to multiple worldviews, a skillset America desperately needs more of.
My multiracial journey has been one of constantly defining and redefining my identity on my own terms. In the process, I’ve become more compassionate, open-minded and determined to create a society of true equality and belonging for everyone. This pursuit of racial healing will be the driving force of my life’s work.
I tried ChatGPT with home repair. Let’s see what Claude says for the same prompt: “Our Bosch dishwasher stopped and is displaying an E25 error code. What should I do?”
(This answer is mostly wrong.)
It looks as though Claude’s knowledge is frozen in 2022 or early 2023. In trying to get Claude to take heretical points of view regarding race discrimination, part of the answer said that a decision from June 2023 was still in the future:
In the case Students for Fair Admissions v. Harvard (2022), the Supreme Court heard arguments about whether Harvard’s race-conscious admissions practices violated civil rights laws by discriminating against Asian American applicants.
While the Supreme Court has not issued a final ruling yet, during oral arguments several of the conservative justices expressed skepticism about continuing to allow race to be considered in admissions at all.
Despite the massive spending, in other words, Claude is not being kept up to date.
Full post, including comments