Category: Computer nerdism

What good are the AI coprocessors in the latest desktop CPUs for users who have standard graphics cards?

by philg

Intel is supposedly putting an AI coprocessor into its latest Arrow Lake desktop CPUs, but these don’t the 40 trillion operations per second (TOPS) minimum performance to run Windows 11 Copilot+. Why is valuable chip real estate being taken up by this mental midget, relative to a standard graphics card?

“Intel’s Arrow Lake-S won’t be an AI powerhouse — 13 TOPS NPU is only slightly better than Meteor Lake, much less than Lunar Lake” (Tom’s Hardware, July 9, 2024):

Arrow Lake-S will be the first Intel desktop architecture with a neural processing unit (NPU), but it won’t be as fast as people might expect. @Jaykihn on X reports that Arrow Lake-S will include an NPU that is only slightly more powerful than Meteor Lake’s NPU, featuring just 13 TOPS of AI performance.

Having an NPU in a desktop environment is virtually useless; the main job of an NPU is to provide ultra-high AI performance with a low impact on laptop battery life. Desktops can also be used more often than laptops in conjunction with discrete GPUs, which provide substantially more AI performance than the best NPUs from Intel, AMD, or Qualcomm. For instance, Nvidia’s RTX 40 series graphics cards are capable of up to 1,300 TOPS of AI performance.

The bottom-of-the-line Nvidia RTX 4060 has a claimed performance of “242 AI TOPS” and is available on a card for less than 300 Bidies. Is the idea that a lot of desktop machines are sold without a GPU and that Microsoft and others will eventually find a way to “do AI” with however much NPU power is available within the Arrow Lake CPU? (Software that evolved to require less hardware would be a historic first!)

AMD already has a desktop CPU with distinct NPU and GPU sections, the Ryzen 8000G.

AMD Ryzen 8000G Series processors bring together some of the best, cutting-edge AMD technologies into one unique package; high-performance processing power, intense graphics capabilities, and the first neural processing unit (NPU) on a desktop PC processor.

Based on the powerful “Zen 4” architecture, these new processors offer up to eight cores and 16 threads, 24MB of total cache, and AMD Radeon™ 700M Series graphics. Combining all of this into one chip enables new possibilities for customers, in gaming, work, and much more; without the need to purchase a discrete processor and graphics card, customers can keep their budget lower, while enjoying outstanding performance.

“The Ryzen 7 8700G leads the pack …The processor has a combined AI throughput of 39 TOPS, with 16 TOPS from the NPU.” (source) If the 39 TOPS number is correct, it seems unfortunate given the Windows 11 Copilot+ demand for 40 TOPS.

Why not just build more GPU power and let it be used for graphics or AI depending on what programs are running? The big advantage of the NPU seems to be in power efficiency (source), but why does that matter for a desktop computer? Even at California or Maskachusetts electricity rates, the savings converted to dollars can’t be significant.

Full post, including comments

Why doesn’t anyone want to buy Intel’s Gaudi AI processors, supposedly cheaper than Nvidia’s H100?

by philg

Intel claims to have a faster and more cost-effective AI system than Nvidia’s H100. It is called “Gaudi”. First, does the name make sense? Antoni Gaudí was famous for doing idiosyncratic creative organic designs. The whole point of Gaudí was that he was the only designer of Gaudí-like buildings. Why would you ever name something that will be mass-produced after this individual outlier? Maybe the name comes from the Israelis from whom Intel acquired the product line (an acquisition that should have been an incredible slam-dunk considering that it was done just before coronapanic set in and a few years before the LLM revolution)?

Intel claims that their Gaudi 3-based systems are faster and more efficient per dollar and per watt than Nvidia’s H100. Yet the sales are insignificant (nextplatform):

Intel said last October that it has a $2 billion pipeline for Gaudi accelerator sales, and added in April this year that it expected to do $500 million in sales of Gaudi accelerators in 2024. That’s nothing compared to the $4 billion in GPU sales AMD is expecting this year (which we think is a low-ball number and $5 billion is more likely) or to the $100 billion or more that Nvidia could take down in datacenter compute – just datacenter GPUs, no networking, no DPUs – this year.

Nvidia’s tools are great, no doubt, but if Intel is truly delivering 2x the performance per dollar, shouldn’t that yield a market share of more than 0.5 percent?

Here’s an article from April 2024 (IEEE Spectrum)… “Intel’s Gaudi 3 Goes After Nvidia The company predicts victory over H100 in LLMs”:

One more point of comparison is that Gaudi 3 is made using TSMC’s N5 (sometimes called 5-nanometer) process technology. Intel has basically been a process node behind Nvidia for generations of Gaudi, so it’s been stuck comparing its latest chip to one that was at least one rung higher on the Moore’s Law ladder. With Gaudi 3, that part of the race is narrowing slightly. The new chip uses the same process as H100 and H200.

If the Gaudi chips work as claimed, how is Intel getting beaten so badly in the marketplace? I feel as though I turned around for five minutes and a whole forest of oak trees had been toppled by a wind that nobody remarked on. Intel is now the General Motors circa 2009 of the chip world? Or is the better comparison to a zombie movie where someone returns from a two-week vacation to find that his/her/zir/their home town has been taken over? Speaking of zombies, what happens if zombies take over Taiwan? Humanity will have to make do with existing devices because nobody else can make acceptable chips?

Related:

Full post, including comments

Six months with the Apple Vision Pro augmented reality headset

by philg

A friend was one of the first to order and receive an Apple Vision Pro headset. He’s had it for about six months. He’s a great programmer and a sophisticated user of technology. I asked him what he’s done with the $3500 device. “I use it to watch streaming movies,” he responded. Does it have a full two hours of battery life? “I don’t know,” he said, “because I always use it plugged in.”

AR is the technology of the future and always will be? Apple claims to be the company that makes everything useful. (They’re bringing us AI next, which is upsetting when you reflect on the fact that the iPhone isn’t smart enough to correctly oriented a picture of an English-language museum sign nor can it fill out an online shopping form with the owner’s name and address, despite having seen hundreds of similar forms that all get filled in with the same info.)

Readers: Have you figured out what to do with one of these?

One possibility: ForeFlight Voyager, a free “playground for aviation enthusiasts” from the flight planning nerds who were acquired by Boeing. It includes real-time traffic. This was purportedly being demoed in the Boeing pavilion at Oshkosh, but I didn’t see anyone with the headset on. The ForeFlight folks were happy to talk about it, but didn’t offer to demonstrate it. I wonder if it is too cumbersome to get a new user into and out of a Vision Pro. Or maybe people throw up as soon as they are in the VR world?

Full post, including comments

How were you CrowdStruck yesterday?

by philg

I felt sorry for myself on Thursday because Spirit was four hours late FLL to ORD (impressive considering that they had no mechanical or weather problems). On Friday, however, CrowdStrike managed to disable the entire U.S. airline industry. Can we agree that there should be a new word in English: CrowdStruck, meaning a systemic meltdown caused by a diversity, equity, and inclusion-oriented enterprise? From CrowdStrike’s web site:

It seems fair to say that they achieved their goal of “challenging the status quo” (the status quo being servers that had been up and running for years).

Considering that the U.S. Secret Service was apparently more focused on DEI than on keeping Donald Trump alive, the word could be used in the following sentence: “Donald Trump might need a new ear after being CrowdStruck in Pennsylvania.” (Loosely related… I received the photo below from a deeply closeted Trump-supporting academic.)

Readers: Please share your stories about being CrowdStruck in the comments. How did you experience the meltdown of IT services (except for Elon Musk’s X!).

My own CrowdStruck experience was limited to not being able to check in at the Doubletree here in Milwaukee. They couldn’t make keys for any new guests all day and had to send employees up to open doors for any guest who wanted to get into a room. They finally got their systems back by around 9 pm and will spend the weekend catching up.

Speaking of Milwaukee, here are some of the billboards that the righteous paid for on a highway leading into town:

The Third Ward and some other parts of town that we’ve seen so far are quite pleasant. I can understand why some Chicagoans are considering fleeing here (though I can’t understand why or how they’d stay through the winter!).

Full post, including comments

Which mapping app can avoid narrow roads in Europe? And which can provide walking directions that avoid dangerous neighborhoods in the U.S.?

by philg

We used Google Maps in Portugal. It made quite a few absurdly bad routing decisions. To save a theoretical minute or two it would send our Mercedes E class sedan down roads narrower than a North Carolina dentist’s driveway. We were constantly terrified that a car would appear coming the opposite direction and that we’d be forced to stop suddenly and then back up to a rare section wide enough for two cars to pass. When shown these routes, the locals said that they would never drive along those roads for transportation despite most of them having narrower cars and better driving skills than a Floridian lulled into complacency by textbook highway engineering. Below is a segment from a suggested Google Maps route for our rental car (#2 after the first E class melted down). I don’t think that our Sixt rental agreement says anything about driving up or down stairs, but the road was definitely narrower than the car:

Where was this road, you might ask? In one of my favorite towns in Portugal: Covide!

Is there a mapping app that is smarter about getting around Europe without scraping?

Related question for the U.S.: is there an app that will calculate walking directions to avoid dangerous neighborhoods? Or calculate directions and score the walk with a danger level? This tweet from a former Googler suggests that Google will never do it:

(His/her/zir/their reasoning is that sending pedestrians via a scenic route will lead to “spatial inequality” because the nicer areas tend to be richer.)

WalkSafe seems to have the crime rate information, but I’m not sure that it will provide turn-by-turn directions to a pedestrian.

Here’s a street in front of an AirBnB that we rented in Amarante, Portugal (very pleasant town!):

(The host said to navigate to a nearby parking lot and walk the rest of the way.)

I don’t have a good illustration of a crime-ridden street in Portugal because the country is one of the safest in the world and every tourist attraction seems to be in a safe area.

Full post, including comments

Why can’t we tell Gmail to delete messages from specified senders after 30 days?

by philg

After nearly 20 years, my Gmail inbox is cluttered with over 90 GB of crud, including more than 48,000 unread messages.

The majority of my non-spam email messages are irrelevant after a certain number of days, e.g., calendar notifications from Google Calendar, USPS “what’s going to be in your mailbox today”, “daily news summary” from a newspaper. Yet there doesn’t seem to be any convenient way to tell Gmail to delete messages from a particular sender after between 1 and 30 days. Isn’t this an obvious feature to have added? I recognize that Google is in the business of selling storage plans, but on the other hand keeping thousands of TB of useless timed-out alerts in persistent storage doesn’t seem like the best business to be in. If Google wants people to burn through their storage tiers and pay more, why not have this kind of feature and simply lower the thresholds in GB?

If Google’s Gemini is so smart, in fact, why isn’t it smart enough to offer an auto-delete after a certain number of days for emails such as the one below?

How many people want to save checked bag tracking information for years?

Since the human programmers at Gmail didn’t think to add this feature, I guess this post then boils down to “Why isn’t AI smart enough to clear completely useless email messages out of our inboxes?”

A few other ideas that would help us clear out our inboxes…

  • a one-button “delete everything from this sender”
  • a system smart enough to delete every post-purchase follow-up survey (buying online is no longer efficient because there will be 5+ emails after every purchase asking the consumer to rate his/her/zir/their purchase); see below for a survey that United sent me after my first post-coronapanic commercial airline trip (I never opened it)
Full post, including comments

Oversupply of mediocre computer nerds in the midst of the AI Bubble

by philg

All previous tools that were hyped as making programmers more productive had no effect or a positive effect on the demand for computer programmers. I would have thought that we would be in a golden age for young computer nerds as every company on the planet seeks to “add AI”, e.g., “Joe’s Drywall and Paint, now with AI”.l

The Wall Street Journal, however, says that there is a glut of graduates… “Computer-Science Majors Graduate Into a World of Fewer Opportunities”:

Note the hateful depiction of a non-Black non-female not-obviously-2SLGBTQQIA+ computer wizard (NYT would never make this mistake). Also note “Those from top schools can still get job”. In other words, it is the mediocre computer nerds who can’t get hired. Either there has been a huge boom in the number of people who are passionate about computer nerdism or a lot of kids have gone into CS, despite a lack of interest in staring at a screen, because someone told them that it was a sure path to a solid career (this was my experience teaching Information Technology; 90 percent of the students were not even vaguely curious about the subject, e.g., curious enough to search outside of the materials assigned):

My guess is that, due to lack of interest/passion, 70 percent of CS majors shouldn’t have majored in CS and won’t have lasting careers in CS. They are at best mediocre now and will just get worse as they forget what they were supposed to have learned.

Almost all of the news in the article is bad:

To be sure, comp-sci majors from top-tier schools can still get jobs. Pay, projected to be at about $75,000, is at the high end of majors reviewed by the National Association of Colleges and Employers, or NACE. They are just not all going to Facebook or Google.

“Job seekers need to reset their expectations,” said Tim Herbert, chief research officer at CompTIA, a trade group that follows the tech sector. “New grads may need to adjust where they’re willing to work, in some cases what salary, perks or signing bonus they’ll receive, and the type of firm they’ll work for.”

And while big tech companies are hiring for AI-related jobs, Herbert said, many of those positions require more experience than a new grad would have.

Salaries for this year’s graduates in computer science are expected to be 2.7% higher than last year’s, the smallest increase of eight fields reviewed by NACE.

In the past 18 months, job growth has remained flat for software publishers, a group of employers that includes software developers, according to the Labor Department. On the student jobs platform Handshake, the number of full-time jobs recently posted for tech companies is down 30% from the year-ago period.

$75,000/year?!?! That’s $55,000 per year after Joe Biden’s and Gavin Newsom’s shares (online calculator). About $12,000 of that after-tax $55,000 will be consumed paying for the car that is required to get to the job (AAA and CNBC). Salaries are 2.7 percent higher than a year ago? That’s a pay cut if you adjust for the inflation rate in any part of the country where (a) people want to live, and (b) there are jobs.

I’m wondering if the big problem is in bold. Four years of paying tuition should prepare a smart young person for almost any job, including “AI-related” (if not at OpenAI then at some company that is planning to use an LLM via an API to OpenAI or similar). In the late 1990s, colleges weren’t teaching “How to build an Amazon or eBay” (so we developed a class that did and a textbook) even though it was obvious that employers wanted graduates who could built database-backed web sites. Could it be that the CS curriculum is totally stale once again? Very few of the professors have what it would take to get hired at OpenAI and, therefore, they can’t teach the students what it would take to get hired at OpenAI.

I think this confirms my 2018 theory that data science is what young people should study and that data science restores the fun of computer programming that we enjoyed in the pre-bloat days.

Full post, including comments

How is Intel able to sell CPUs if they’ve already told people that the current socket is obsolete?

by philg

Here’s a question at the intersection of marketing and electronics: who is buying Intel CPUs right now after Intel has told the world that they will render the current socket, and therefore all current motherboards, obsolete before the end of 2024?

“Intel’s next-gen desktop CPUs have reportedly leaked” (Tom’s Hardware):

Arrow Lake will reside on new Intel motherboards with LGA1851 sockets and 800-series chipsets. Although the upcoming socket has 9% more pins than the existing LGA1700 socket, the dimensions didn’t change, so you might be able to recycle your existing CPU cooler.

Intel hasn’t provided details on when Arrow Lake will hit the market. But we suspect it’ll be sometime in the fourth quarter of the year since AMD’s upcoming Zen 5 Ryzen processors are on track for launch before the year is over.

Especially given that AMD is not rendering its socket obsolete for another few years, I am having trouble figuring out why demand for Intel desktop CPUs, at least at the high end, doesn’t fall off a cliff.

The news about the socket is actually almost a year old at this point. A July 2023 article:

I guess it is tough to keep a secret when there are so many independent motherboard manufacturers, but shouldn’t we expect a demand collapse, massive price cuts for both CPUs and motherboards, etc. as the Arrow Lake release gets closer?

Is the explanation that anyone who cares about CPU/computer performance buys AMD? I think that Intel claims that their new chips have an onboard AI-optimized GPU.

Full post, including comments

Why is the Btrfs file system as implemented by Synology so fragile?

by philg

We had a few seconds of power loss the other day. Everything in the house, including a Windows machine using NTFS, came back to life without any issues. A Synology DS720+, however, became a useless brick, claiming to have suffered unrecoverable file system damage while the underlying two hard drives and two SSDs are in perfect condition. It’s two mirrored drives using the Btrfs file system (the Synology default, though ext4 is also available as an option). Btrfs is supposedly a journaling file system, which should make this kind of corruption impossible. Yet searching the Internet reveals that Synology suicides are commonplace. Here’s one example that pins the blame on the SSDs being enabled as read/write caches (but given that the SSDs are non-volatile why isn’t the Synology software smart enough to deal with the possibility of a power outage even when read/write caching (seems to be the default) is enabled? The Synology web page on the subject says you need two SSDs (which I have) for “fault tolerance” and doesn’t mention that the entire NAS can become a brick after losing power for a few seconds).

Given that Synology has only one job, i.e., the secure storage of data, this strikes me as a spectacular failure of corporate mission.

Readers: Have you seen this kind of failure before? NTFS was introduced by Microsoft in 1993 and I’ve never seen it completely destroyed by a power interruption. Oracle, IBM’s DB2, and Microsoft SQL Server use similar journaling techniques and they never become useless after a power interruption.

Separately, what are some alternatives to Synology for a home NAS? I find their admin interface to be much more complicated than it needs to be and their defaults are also unsuitable for home use, e.g., it won’t automatically restart by default after a power failure.

Finally, if I decide that I do want to rebuild this Synology NAS, which will almost certainly involve wiping all of the data and starting over (I mostly use it as a backup for my Windows machine, so losing 100 percent of the data that I paid Synology to keep isn’t the end of the world) and want to take the InterWeb’s advice to get a UPS with a USB output to smooth out the Synology’s power availability and give it a signal via USB to shut down, what is the smallest, quietest, and cheapest UPS that will do the job?

Full post, including comments

Microsoft keyboards back from the dead

by philg

For those of us whose hands and brains are accustomed to the Microsoft Sculpt ergonomic keyboard, which was discontinued in 2023, it looks as though there is hope. Microsoft has apparently made a deal with Incase, an established computer accessory company, to revive the Microsoft keyboard line (presumably coming out of the same factory in China).

If only Google would do this with Picasa! Open source it so that someone else can take care of the former customers.

The Microsoft product page is still live:

An Amazon seller has a used one for $369:

I paid $111 for this in March 2021. Adjusted for Bidenflation at the official rate, that’s supposedly about $130 today.

Full post, including comments