Category: Artificial Intelligence

What good are the AI coprocessors in the latest desktop CPUs for users who have standard graphics cards?

by philg

Intel is supposedly putting an AI coprocessor into its latest Arrow Lake desktop CPUs, but these don’t the 40 trillion operations per second (TOPS) minimum performance to run Windows 11 Copilot+. Why is valuable chip real estate being taken up by this mental midget, relative to a standard graphics card?

“Intel’s Arrow Lake-S won’t be an AI powerhouse — 13 TOPS NPU is only slightly better than Meteor Lake, much less than Lunar Lake” (Tom’s Hardware, July 9, 2024):

Arrow Lake-S will be the first Intel desktop architecture with a neural processing unit (NPU), but it won’t be as fast as people might expect. @Jaykihn on X reports that Arrow Lake-S will include an NPU that is only slightly more powerful than Meteor Lake’s NPU, featuring just 13 TOPS of AI performance.

Having an NPU in a desktop environment is virtually useless; the main job of an NPU is to provide ultra-high AI performance with a low impact on laptop battery life. Desktops can also be used more often than laptops in conjunction with discrete GPUs, which provide substantially more AI performance than the best NPUs from Intel, AMD, or Qualcomm. For instance, Nvidia’s RTX 40 series graphics cards are capable of up to 1,300 TOPS of AI performance.

The bottom-of-the-line Nvidia RTX 4060 has a claimed performance of “242 AI TOPS” and is available on a card for less than 300 Bidies. Is the idea that a lot of desktop machines are sold without a GPU and that Microsoft and others will eventually find a way to “do AI” with however much NPU power is available within the Arrow Lake CPU? (Software that evolved to require less hardware would be a historic first!)

AMD already has a desktop CPU with distinct NPU and GPU sections, the Ryzen 8000G.

AMD Ryzen 8000G Series processors bring together some of the best, cutting-edge AMD technologies into one unique package; high-performance processing power, intense graphics capabilities, and the first neural processing unit (NPU) on a desktop PC processor.

Based on the powerful “Zen 4” architecture, these new processors offer up to eight cores and 16 threads, 24MB of total cache, and AMD Radeon™ 700M Series graphics. Combining all of this into one chip enables new possibilities for customers, in gaming, work, and much more; without the need to purchase a discrete processor and graphics card, customers can keep their budget lower, while enjoying outstanding performance.

“The Ryzen 7 8700G leads the pack …The processor has a combined AI throughput of 39 TOPS, with 16 TOPS from the NPU.” (source) If the 39 TOPS number is correct, it seems unfortunate given the Windows 11 Copilot+ demand for 40 TOPS.

Why not just build more GPU power and let it be used for graphics or AI depending on what programs are running? The big advantage of the NPU seems to be in power efficiency (source), but why does that matter for a desktop computer? Even at California or Maskachusetts electricity rates, the savings converted to dollars can’t be significant.

Full post, including comments

If we’re on the cusp of the AI golden age, why can’t web browsers fill out forms for us?

by philg

We are informed that AI is going to transform our daily lives. Web browsers are made by companies that are supposedly at the forefront of AI/LLM research and development. Why isn’t a browser smart enough to fill out the entire form below? It has seen fields with similar labels filled in hundreds or thousands of times. Why doesn’t the browser fill it out automatically and then invite the user to edit or choose “fill it out with my office address instead”?

Google Chrome, at least, will suggest values for individual fields. Why won’t it take the next step? Even the least competent human assistant should be able to fill in the above form on behalf of a boss. Why can’t AIs in which $billions have been invested do it?

Full post, including comments

Why doesn’t anyone want to buy Intel’s Gaudi AI processors, supposedly cheaper than Nvidia’s H100?

by philg

Intel claims to have a faster and more cost-effective AI system than Nvidia’s H100. It is called “Gaudi”. First, does the name make sense? Antoni Gaudí was famous for doing idiosyncratic creative organic designs. The whole point of Gaudí was that he was the only designer of Gaudí-like buildings. Why would you ever name something that will be mass-produced after this individual outlier? Maybe the name comes from the Israelis from whom Intel acquired the product line (an acquisition that should have been an incredible slam-dunk considering that it was done just before coronapanic set in and a few years before the LLM revolution)?

Intel claims that their Gaudi 3-based systems are faster and more efficient per dollar and per watt than Nvidia’s H100. Yet the sales are insignificant (nextplatform):

Intel said last October that it has a $2 billion pipeline for Gaudi accelerator sales, and added in April this year that it expected to do $500 million in sales of Gaudi accelerators in 2024. That’s nothing compared to the $4 billion in GPU sales AMD is expecting this year (which we think is a low-ball number and $5 billion is more likely) or to the $100 billion or more that Nvidia could take down in datacenter compute – just datacenter GPUs, no networking, no DPUs – this year.

Nvidia’s tools are great, no doubt, but if Intel is truly delivering 2x the performance per dollar, shouldn’t that yield a market share of more than 0.5 percent?

Here’s an article from April 2024 (IEEE Spectrum)… “Intel’s Gaudi 3 Goes After Nvidia The company predicts victory over H100 in LLMs”:

One more point of comparison is that Gaudi 3 is made using TSMC’s N5 (sometimes called 5-nanometer) process technology. Intel has basically been a process node behind Nvidia for generations of Gaudi, so it’s been stuck comparing its latest chip to one that was at least one rung higher on the Moore’s Law ladder. With Gaudi 3, that part of the race is narrowing slightly. The new chip uses the same process as H100 and H200.

If the Gaudi chips work as claimed, how is Intel getting beaten so badly in the marketplace? I feel as though I turned around for five minutes and a whole forest of oak trees had been toppled by a wind that nobody remarked on. Intel is now the General Motors circa 2009 of the chip world? Or is the better comparison to a zombie movie where someone returns from a two-week vacation to find that his/her/zir/their home town has been taken over? Speaking of zombies, what happens if zombies take over Taiwan? Humanity will have to make do with existing devices because nobody else can make acceptable chips?

Related:

Full post, including comments

How will NVIDIA avoid a Google-style Vesting in Peace syndrome?

by philg

NVIDIA is the world’s most valuable company (P/E ratio of 75; compare to less than 40 for Microsoft), which also means that nearly every NVIDIA employee is rich. A lot of people slack off when they become rich. Google ended up with quite a few “vesting in peace” workers who didn’t contribute much. It didn’t matter because it was too challenging for anyone else to break into the search and advertising businesses. But suppose that another tech company assembles a group of not-yet-rich hardware and software people. Hungry for success, these people build some competitive GPUs and the biggest NVIDIA customers merely have to recompile their software in order to use the alternative GPUs that are marketed at a much lower price.

How can NVIDIA’s spectacular success not lead to marketplace slippage due to an excessively rich and complacent workforce? Is the secret that NVIDIA can get money at such a low cost compared to competitors that it can afford to spend 2-3X as much on the next GPU and still make crazy profits? I find it tough to understand how Intel, which for years has made GPUs inside its CPUs, can’t develop something that AI companies want to buy. Intel has a nice web page explaining how great their data center GPUs are for AI:

Why can’t Intel sell these? Are the designs so bad that they couldn’t compete with NVIDIA even if sold at Intel’s cost?

Full post, including comments

Bachelor’s in AI Gold Rush degree program

by philg

A college degree is purportedly important preparation for multiple aspects of life. Universities, therefore, require students to take classes that are far beyond their major. Extracurricular activities are encouraged, such as sports, pro-Hamas demonstrations, drinking alcohol (how is that supposed to make immigrants from Gaza feel welcome?), casual sex, theater, etc. Students are forced to take about half the year off because the faculty and staff don’t want to work summers (defined as May through early September), January, or anywhere near various holidays. There is no urgency to earning a degree so why not stretch it out for four years?

What if there were urgency to getting into the workforce? Here’s the company that sold shovels to the crypto miners and now sells shovels to the AI miners (May 23):

It was a lot better to start work at NVIDA in June 2022 than in June 2024. Consider a Stanford graduate who could have finished in 2022, but instead didn’t finish until 2024. He/she/ze/they took Gender and Gender Inequality, Intersectionality: Theory, Methods & Research, and Race and Ethnicity Around the World from Professor Saperstein to round out his/her/zir/their engineering education. Was that worth the $5 million that would have been earned by starting work at NVIDIA in 2022 rather than in 2024 (two years of salary, stock options at $175 instead of at $1000, etc.)?

How about a “Bachelor’s in AI Gold Rush” degree program that would prepare students to build and use LLMs? It would be a 2-year program with no breaks so that people could graduate and start their jobs at OpenAI. There would be no requirement to take comparative victimhood classes (i.e., humanities). There would be no foundational math or science unless directly related to LLM construction (a lot of linear algebra?). There would be no pretense of preparing students for anything other than working at OpenAI or a similar enterprise.

Students will graduate at age 20. What if the AI gold rush is over when they turn 28? (Maybe not because AI turns out to be useless or even over-hyped, but only because the industry matures or the LLMs start building new LLMs all by themselves.) They can go back to college and take all of that “might be useful” foundational stuff that they missed, e.g., back to Harvard to study Queering the South:

(A friend’s daughter actually took the above class; she was most recently living in Harvard’s pro-Hamas encampment.) As a follow-on:

If the 28-year-old made so much money in the AI gold rush that he/she/ze/they wants to “give back” by becoming a school teacher, he/she/ze/they can get a Master’s in Education at Harvard and take “Queering Education”:

By the end of the module, students should be able to: (1) Talk comfortably about queer theory and how it can inform our understanding of schools and schooling; (2) identify specific strategies that educators at various levels might use to support students in negotiating gender and sexuality norms; (3) identify tools that schools can use to build positive, nurturing environments, which open up possibilities for complex gender and sexual identity development; and (4) analyze and evaluate a variety of school practices, curricula, programs, and policies that seek to support healthy gender and sexual identity development for U.S. children and adolescents.

Related:

May 31, 2024 update:

Full post, including comments

Where’s the AI customer service dividend?

by philg

ChatGPT (launched November 2022) and similar LLMs were supposed to make customer service agents more efficient. Has this happened? From what I can tell, the opposite has occurred. If I call a company that is supposed to be providing service the inevitable greeting is “we are experiencing higher than normal call volume” (i.e., demand for service exceeds agent capacity, despite the agents now being augmented with AI). When an agent does pick up, he/she/ze/they immediately asks, “What is your phone number?” In other words, the smartest computer systems ever devised cannot use caller ID.

(If Trump gets elected this fall and then, as predicted by the New York Times and CNN, ends American democracy, I hope that he will issue a decree that companies aren’t allowed to announce “we are experiencing higher than normal call volume” more than 5 percent of the time.)

My favorite company for customer service is Hertz. They recently hit my credit card for $262.41 for a 24-hour 29-mile rental of a compact Ford Edge in El Paso. I never signed anything agreeing to pay $262 and their app was quoting $76 including all fees (I picked up the car at an FBO so there wasn’t the fully array of Hertz computer systems on site). When I called Hertz to try to figure out why they charged so much I learned that they’ve eliminated the option of talking to a human regarding any bill. A human will be happy to make a reservation, but not to answer questions about what could be a substantial credit card charge. Hertz funnels all questions about past rentals to a web form, which they say they will respond to within a few days. Of course, my first inquiry about the bill yielded no response. My second inquiry, a week later, yielded a “everything was done correctly” response. I finally pinged them on Twitter private message. They admitted that they had no signed paperwork with an agreement to pay $262 and issued a refund of about half the money.

Circling back to AI… if LLMs make customer service agents more efficient, why has Hertz needed to shut down phone customer service? And if LLMs are brilliant at handling text why isn’t Hertz able to respond to contact form inquiries quickly?

Here’s an example pitch from the AI hucksters:

Full post, including comments

Oversupply of mediocre computer nerds in the midst of the AI Bubble

by philg

All previous tools that were hyped as making programmers more productive had no effect or a positive effect on the demand for computer programmers. I would have thought that we would be in a golden age for young computer nerds as every company on the planet seeks to “add AI”, e.g., “Joe’s Drywall and Paint, now with AI”.l

The Wall Street Journal, however, says that there is a glut of graduates… “Computer-Science Majors Graduate Into a World of Fewer Opportunities”:

Note the hateful depiction of a non-Black non-female not-obviously-2SLGBTQQIA+ computer wizard (NYT would never make this mistake). Also note “Those from top schools can still get job”. In other words, it is the mediocre computer nerds who can’t get hired. Either there has been a huge boom in the number of people who are passionate about computer nerdism or a lot of kids have gone into CS, despite a lack of interest in staring at a screen, because someone told them that it was a sure path to a solid career (this was my experience teaching Information Technology; 90 percent of the students were not even vaguely curious about the subject, e.g., curious enough to search outside of the materials assigned):

My guess is that, due to lack of interest/passion, 70 percent of CS majors shouldn’t have majored in CS and won’t have lasting careers in CS. They are at best mediocre now and will just get worse as they forget what they were supposed to have learned.

Almost all of the news in the article is bad:

To be sure, comp-sci majors from top-tier schools can still get jobs. Pay, projected to be at about $75,000, is at the high end of majors reviewed by the National Association of Colleges and Employers, or NACE. They are just not all going to Facebook or Google.

“Job seekers need to reset their expectations,” said Tim Herbert, chief research officer at CompTIA, a trade group that follows the tech sector. “New grads may need to adjust where they’re willing to work, in some cases what salary, perks or signing bonus they’ll receive, and the type of firm they’ll work for.”

And while big tech companies are hiring for AI-related jobs, Herbert said, many of those positions require more experience than a new grad would have.

Salaries for this year’s graduates in computer science are expected to be 2.7% higher than last year’s, the smallest increase of eight fields reviewed by NACE.

In the past 18 months, job growth has remained flat for software publishers, a group of employers that includes software developers, according to the Labor Department. On the student jobs platform Handshake, the number of full-time jobs recently posted for tech companies is down 30% from the year-ago period.

$75,000/year?!?! That’s $55,000 per year after Joe Biden’s and Gavin Newsom’s shares (online calculator). About $12,000 of that after-tax $55,000 will be consumed paying for the car that is required to get to the job (AAA and CNBC). Salaries are 2.7 percent higher than a year ago? That’s a pay cut if you adjust for the inflation rate in any part of the country where (a) people want to live, and (b) there are jobs.

I’m wondering if the big problem is in bold. Four years of paying tuition should prepare a smart young person for almost any job, including “AI-related” (if not at OpenAI then at some company that is planning to use an LLM via an API to OpenAI or similar). In the late 1990s, colleges weren’t teaching “How to build an Amazon or eBay” (so we developed a class that did and a textbook) even though it was obvious that employers wanted graduates who could built database-backed web sites. Could it be that the CS curriculum is totally stale once again? Very few of the professors have what it would take to get hired at OpenAI and, therefore, they can’t teach the students what it would take to get hired at OpenAI.

I think this confirms my 2018 theory that data science is what young people should study and that data science restores the fun of computer programming that we enjoyed in the pre-bloat days.

Full post, including comments

Maybe cars can teach themselves to drive in the more structured states (the MANIAC book)

by philg

I recently finished The MANIAC, a concise novelized biography of John Von Neumann bizarrely bolted onto a history of computer programs that dominate chess and go. Somehow the combination works! What I hadn’t realized was how quickly programs that play chess and go can evolve when entirely freed from human guidance. Apparently, in a matter of just a few hours, a program can go from knowing almost nothing about chess other than the basic rules to being able to beat a grandmaster.

This kind of success has famously eluded those who promised us self-driving cars. We’ve gone from failing via humans encoding rules to failing via AI-style training sets of good driving and bad driving (coded by people in India? if you’ve ever been to Delhi or Mumbai maybe that explains the failure). Benjamin Labatut (the MANIAC author) reminds us that when the situation is sufficiently structured computers can learn very fast indeed.

Returning from a helicopter trip from Los Angeles to Great Barrington, Maskachusetts, my copilot commented on the chaos of road markings as we entered Cambridge. “Are there three lanes here or two?” he asked. This is a question that wouldn’t be posed in most parts of Texas or Florida, I’m pretty sure, and certainly not on the main roads of the Netherlands or Germany. Instead of the computer promising to handle all situations, I wonder if “full self-driving” should be targeted to the states where roads are clearly structured and marked. Instead of the computer telling the human to be ready to take over at any time for any reason, the computer could promise to notify in advance (via reference to a database, updated via crowd sourcing from all of the smart cars) that the road wasn’t sufficiently structured/marked and tell the human “I won’t be able to help starting in 30 seconds because your route goes through an unstructured zone.” The idea that a human will be vigilant for a few months or even years waiting for a self-driving disconnect that occurs randomly seems impractical. The MANIAC suggests that if we shift gears (so to speak) to redefining the problem to self-driving within a highly structured environment a computer could become a better driver than a human in a matter of weeks (it takes longer to look at videos than to look at a chess or go board, so it would be weeks and not hours). We might not be able to predict when there will be enough structure and enough of a data set and enough computer power for this breakthrough to occur, but maybe we can predict that it will be sudden and the self-driving program will work far better than we had dreamed. The AI-trained chess and go systems didn’t spend years working their way into being better than the best humans, but got there from scratch in just a few hours by playing games against themselves.

Regardless of your best estimate as to when we’ll get useful assistance from our AI overlords, I recommend The MANIAC (note that the author gives Von Neumann a little too much credit for the stored program computers that make the debate regarding self-driving possible).

Separately, based on a visit to the Harvard Book Store here’s what’s on the minds of the world’s smartest people (according to Harvard University research)..

Full post, including comments

Why doesn’t ChatGPT tell us where to find items in our houses?

by philg

American houses have gotten larger:

Thanks to the hard-working folks in China and at Walmart, stuff has gotten cheaper. The result is that we live in large environments crammed with stuff. This makes it tough to find one’s keys, the cup of coffee that one recently set down, etc.

Instead of AI taking over the creative jobs of writing poetry and making art, why not have AI watch everything that happens in the apartment or house (video and inferences from the video stored locally for privacy) and then we can say “Yo, ChatGPT, where did I leave my keys?” If we get in the habit of showing documents that arrive in the mail to a camera we can also ask the AI to remind us when it is time to pay a property tax bill or ask where we left an important document.

This could be rolled into some of the boxes that Unifi makes. They already make sensors for the house:

They claim to have “AI” in their $2500 “DSLR” PoE camera (only 18 watts):

Their basic cameras are $120 each. If the basic cameras are good enough, this should be doable on top of the Unifi infrastructure for perhaps $300 per room plus whatever the central AI brain costs.

Speaking of Unifi, I’m wondering why they don’t sell a combined access point/camera. If the customer has just a single CAT 5/6 wire to the back yard, wouldn’t it make sense to have the same PoE-powered device handle both security and WiFi? As far as I know, there isn’t any combined camera/AP.

(I’m still using the TP-Link Omada system that I bought because Unifi’s products were all out of stock. See TP-Link Omada: like a mesh network, except that it works (alternative to UniFi). Everything works, but they don’t seem to be trying to expand beyond networking as Unifi has. Maybe when WiFi 8 comes out it will be time to trash all of the working WiFi 6 Omada gear and get with the Unifi/Ubiquiti program.)

Speaking of houses, here’s a recent New York Times article informing us regarding what a typical American “couple” looks like (the word is used 11 times in the article)…

Full post, including comments

Why isn’t ChatGPT inside our refrigerators?

by philg

Some years ago people envisioned a refrigerator that would track contents via RFID and alert a consumer to being low on milk or whatever. Making this a reality would have required cooperation among all of the companies that make packaged food (to add the RFID tags) so of course it never happened.

A human can inventory a fridge. Anything a human can do ChatGPT can do better, or so we’re told. If a fridge costs $15,000 (see Sub-Zero refrigerator with R600a owner’s review) why can’t it use a handful of inexpensive video cameras to look at everything going in and out in detail? It can make some good guesses about quantities, e.g., every time the eggs are removed there will be three fewer eggs remaining in the carton (refine this guess after some experience in a household as to when the carton stops being returned to the fridge (assume this means the egg count is zero)). The in-the-fridge AI could email with a list of expired stuff to throw out and a list of stuff to buy. It could email at 3 pm every day with a suggestion for what to cook for dinner given the ingredients present in the fridge, adding critical items via an Instacart order if approved.

“New AI-powered fridge technology generates recipes based on diet, food on its shelves” (Deplorable Fox) describes a Samsung fridge introduced at CES 2024, but it turns out to be not that smart:

The fridge’s technology also reportedly enables users to add expiration dates for items purchased, and the refrigerator will alert them once that expiration date is near.

Why is it the human’s job to read expiration dates off the packages? Why can’t the brilliant AI do that? Let’s give some credit to Samsung, though, for including an epic 32-inch TV on the $4500 fridge:

So the Samsung fridge is missing the Instacart ordering support, I think, as well as the automation of ferreting out expired food.

Full post, including comments