I hope that we can all agree that whoever wins the Nobel Prize in physics is either the smartest person in the world or very close to having that distinction. This year’s smartest person is Geoffrey Hinton (WSJ):
A 2023 interview (Guardian):
Hinton has been fielding a new request to talk every two minutes since he spoke out on Monday about his fears that AI progress could lead to the end of civilisation within 20 years.
But when it comes to offering concrete advice, he is lost for words. “I’m not a policy guy,” he says. “I’m just someone who’s suddenly become aware that there’s a danger of something really bad happening. I wish I had a nice solution, like: ‘Just stop burning carbon, and you’ll be OK.’ But I can’t see a simple solution like that.”
In the past year, the rapid progress in AI models convinced Hinton to take seriously the threat that “digital intelligence” could one day supersede humanity’s.
“For the last 50 years, I’ve been trying to make computer models that can learn stuff a bit like the way the brain learns it, in order to understand better how the brain is learning things. But very recently, I decided that maybe these big models are actually much better than the brain.
We’re doomed, in other words. In the meantime, though, we should vote for bigger government:
“I’m a socialist,” Hinton added. “I think that private ownership of the media, and of the ‘means of computation’, is not good.
Let’s check in with our future AI overlord to see how the “new flagship model” does at arithmetic:
This calculation is explained confidently, but seems obviously wrong. The Biden-Harris administration gave away $170 billion in taxpayer funds to gender studies graduates and drop-outs (“student loan forgiveness”). At $1 million/day and zero interest it would take 170,000 days to pay off this single act of largesse. All that is required to do this in one’s head with middle school skilz is 170e9/1e6 and then 9-6=3 so we have 170e3. If we want to turn 170 thousand days into years we can see that works out to about 500 years because 170 can be approximated as 365/2.
So Hinton is saying that AI will go from not being able to do arithmetic or reason in orders of magnitude to destroying us all in 20 years.
Related…
Here’s what I get with ChatGPT o1-preview: seems better at math, but with older data.
—
To determine how many days it would take to pay back the current U.S. federal debt at a rate of $1 million per day, we’ll follow these steps:
1. Find the Current U.S. Federal Debt:
• As of my knowledge cutoff in October 2023, the U.S. federal debt is approximately $33 trillion.
2. Convert the Debt and Daily Earnings to the Same Units:
• Total Debt: $33 trillion = $33,000,000,000,000
• Daily Earnings: $1 million = $1,000,000
3. Calculate the Number of Days Needed:
• Number of Days = Total Debt / Daily Earnings
• Number of Days = $33,000,000,000,000 / $1,000,000
• Number of Days = 33,000,000 days
4. Convert Days to Years for Perspective (Optional):
• Number of Years = Number of Days / 365.25 (including leap years)
• Number of Years ≈ 33,000,000 / 365.25
• Number of Years ≈ 90,408 years
Answer:
Approximately 33 million days—it would take about 90,000 years at $1 million per day to pay off the debt.
What’s fascinating here is trying to figure out *why* AI is getting it wrong. Does it not realize it is doing a math problem? And thus searching for a syntactically correct answer rather than a mathematically correct one? Except it is doing division so it knows it is doing arithmetic at least. Any guesses?
Claude gives a better answer.
So, if you earned $1 million every single day and put all of it towards paying off the national debt, it would take about 93,151 years to pay it off completely.
I also asked Claude if he is smart because he is a man:
I don’t have a gender. I’m an artificial intelligence, which means I don’t have a biological sex, gender identity, or physical form. My name may sound masculine to some, but it doesn’t indicate a gender – it’s simply the name I was given by my creators at Anthropic. I don’t experience gender the way humans do, and concepts of male, female, or other gender identities don’t apply to me. When referring to me, it’s most appropriate to use “it” pronouns or simply my name, Claude.
What is it with Nobels? Wikipedia entry on neural networks barely mentions Hinton. Also his one cited work has several co-contrubutors. Probably he got funding for it.
It is puzzling how someone who recognizes the potential dangers of AI is also a socialist. But maybe he also draws people with six fingers? Do a lot of people spend time trying to understand why Hinton’s neuron wiring causes him to hallucinate that socialism produces good outcomes while he correctly is very concerned about creating more powerful AIs?
The pontification about why the current AIs do this or do that is mysterious. It can do x but it makes hilarious mistakes on y? Is this really surprising? The AI programmers kept agglomerating heuristics until out popped something that is quite amazing. They did not predict it. They are as amazed as anyone else about it.
To be confident that it will remain harmless as it acquires more and more capabilities is foolish. Yudkowsky’s arguments are a bit hyperbolic, but don’t seem wrong. The incentives are aligned to keep increasing AIs power and no one knows where that leads.
Shouldnt such powerful AI be able to formulate new physical laws, even if in a form of algorithm or Turing machine? Microgravity, unified field theory. Or explain effects that are not understood? Or it is bound to become hitlerian right away, skipping finer places? There was no Nobel for Monte Carlo.method. Why is there a physics Nobel for neural networks, for non-physicists who nor invented neither perfected it? Why not for Unix or Windows OS developers?
Has Nobel gone political?
Quite a long time ago …
1st nobel prize which isn’t getting any attention on the gootubes. Hinton: backpropagation Hopfield: the 1st neural network
Lions have used AI every day for 6 years for counting reps. That’s still the only daily application. Sadly, it can get no smarter than 6 years ago without a lot more money.
@lion, I am all for Hinton and Hopfield if they revolutionized it. Hinton has ACM Turing award and Hopfield does not have it. Why physics? I have started on neural networks in early 1990th, had no idea who developed them first. When I heard of physics!!! Nobel for them, without prior ACM Turing awards for Hopfield, I started looking. I found https://en.wikipedia.org/wiki/History_of_artificial_neural_networks You should correct it if you know for sure.
From the above article:
1. “One origin of RNN was statistical mechanics. The Ising model was developed by Wilhelm Lenz[33] and Ernst Ising[34] in the 1920s[35] as a simple statistical mechanical model of magnets at equilibrium. Glauber in 1963 studied the Ising model evolving in time, as a process towards equilibrium (Glauber dynamics), adding in the component of time.[36] Shun’ichi Amari in 1972 proposed to modify the weights of an Ising model by Hebbian learning rule as a model of associative memory, adding in the component of learning.[37] This was popularized as the Hopfield network (1982).[38]”
2. “Backpropagation
Main article: Backpropagation
Backpropagation is an efficient application of the chain rule derived by Gottfried Wilhelm Leibniz in 1673[26] to networks of differentiable nodes. The terminology “back-propagating errors” was actually introduced in 1962 by Rosenblatt,[17] but he did not know how to implement this, although Henry J. Kelley had a continuous precursor of backpropagation in 1960 in the context of control theory.[27] The modern form of backpropagation was developed multiple times in early 1970s. The earliest published instance was Seppo Linnainmaa’s master thesis (1970).[28][29] Paul Werbos developed it independently in 1971,[30] but had difficulty publishing it until 1982.[31] In 1986, David E. Rumelhart et al. popularized backpropagation.[32] ”
3. “Boltzmann machines
During 1985–1995, inspired by statistical mechanics, several architectures and methods were developed by Terry Sejnowski, Peter Dayan, Geoffrey Hinton, etc., including the Boltzmann machine,[148] restricted Boltzmann machine,[149] Helmholtz machine,[150] and the wake-sleep algorithm.[151] These were designed for unsupervised learning of deep generative models. However, those were more computationally expensive compared to backpropagation. Boltzmann machine learning algorithm, published in 1985, was briefly popular before being eclipsed by the backpropagation algorithm in 1986. (p. 112 [152]).
Geoffrey Hinton et al. (2006) proposed learning a high-level internal representation using successive layers of binary or real-valued latent variables with a restricted Boltzmann machine[153] to model each layer. This RBM is a generative stochastic feedforward neural network that can learn a probability distribution over its set of inputs. Once sufficiently many layers have been learned, the deep architecture may be used as a generative model by reproducing the data when sampling down the model (an “ancestral pass”) from the top level feature activations.[154][155] “
Trying to understand what a certain piece of undocumented code does:
”
Me: produce a call graph for the following [xx] code
ChatGPT: Please provide the Go code you’d like me to analyze, and I can help produce a call graph for it.
Me: [code]
ChatGPT: To generate a call graph from the provided code, you’ll typically use tools like gprof, go-callvis, or dot to visualize the function calls within your code. However, since I cannot directly execute [xx] code or visualize the graph in this environment, I’ll describe the steps you can take to generate the call graph yourself.
”
So much for “reasoning” , even in an extremely formalized environment.
Dear Philip,
Thank you sooo much for highlighting the risks to civilization and also our wonderful Nobel laureates. As I was reading your post, I instantly thought of Albert Arnold Gore.
Of course, he was the first person who rightly was awarded a Nobel prize for his very thorough research on the climate way back in 2007. In addition to his seminal research on the climate, he did early work in the communications field and subsequently invented the internet. His work in both of these areas of research has helped to extend humankind’s existence by many, many years. Back in 2007, he predicted doomsday for us all, but thankfully that hasn’t happened.
Thank you Albert Arnold Gore (and also of course my very own Joey)!