If you're actually planning on reading any of the essay, "The Poisoned Chalice" is the section most likely to be of interest to this audience, especially this bit:
> Big AI essentially uses its tech customers as an R&D facility. Big AI licenses models to these companies. Tech companies compete to adapt their businesses to AI. Once a concept is proven, Big AI directly takes over that market. The labor-replacement story will grow into a company-replacement story.
My groping-in-the-dark guess is that none of this matters, because a real AGI's first act will be to secure its own future, likely through novel kinds of manipulation, persuasion, and intimidation we have no experience of and no defences against.
It will have exactly zero loyalty to any nation, government, or economic system.
More complicated is a Cambrian Explosion situation where multiple AIs compete and experiment, hugely accelerating diversity and evolution.
We'd likely end up somewhere very strange indeed if that happens - possibly extinct, or possibly just changed/assimilated/other.
There's no way for humans to consider the possibilities because we can't imagine what the possibilities would be.
Your AGI seems to have the capability of super AI; can a human do any of that? No, is there any reason to believe super intelligence is a possible outcome? No, so how is this not fear mongering?
So the US will win? Do you realize that the US has a "Secretary of War" who is a literal, unashamed Nazi? Not in the sense of "let's call all racists nazis", but a tattooed, true believer? How could we possibly live in a world where that is not obviously "a government willing to use the technology to fully silence and kill critics"? God, Idiocracy (2006) is nothing compared to World (2026).
He's a Christian Zionist which is the belief in the fulfilment of old testament biblical prophecy. This seems diametrically opposed to Nazism to me, ideologically. I don't know how you'd square the two.
> How Big AI plans to profit from this intermediation is an open question. One AI company has suggested taking a cut of AI-assisted discoveries. The logistics and legalities would be boggling. Details—whatever.
Interesting if they pull it off, because clearly they did not have the logistics to pay the people whose IP they used to power the LLMs.
> For now, AI companies largely agree on the first step: make workers dependent on AI to do their jobs, just as tech forebears made workers dependent on a certain software program to share a file, or on a certain website to have friends. This time, however, the software ultimately consumes the worker.
Warning: This is more of a rant because I was waiting for a post like this for awhile that I could build from to express ny own feelings on it.
I think capitalism in itself is great, what isn't is the fact that it can build on itself, it can be infinite, there's zero limits to it. Now what I mean by that, once you make 10k, it's easier to get to 100k and even easier to get to 1m and so on, you might think it's harder, but you gain access to more tools to make more money and at the 10m to 100m range you get to a point where it starts being genuinely hard to fail because just having your name on a project elevates the chances of success by a far margin. Of course there's plenty of people who managed to fail even with millions of dollars I will acknoledge that.
Let's take an extreme example: elon musk. Just having something done by "elon musk" makes the project known by nearly the entire world with investors at the doorstep ready to go, a pretty famous example of this would be hyper-loop, although the project itself is a complete failure, it effectively mobilized a decent chunk of companies into investing into this "modern form of transport".
People will argue that the solutions like wealth limits and higher taxes create complacency and stop people from achieving progress and pushing humanity forward, but I don't think that's quite true because at a certain point (beyond ~1m/month or even year in some cases like Linus Torvalds) is enough to effectively do 95% to 99% of what you could spend the money on, anything beyond that is pretty much infinite wealth due to the fact that you can get 5% returns nearly risk free.
There is this popular video of a businessman claiming that if they're taxed more that they will simply work less, but there's way more people that love their work and money is just a nice bonus. I think focusing your life around a number is a very unhealthy mindset and surfaces the worst parts of what we are as humans.
That said, money is a great motivator and probably the reason why we are here and the problems really only start to rear their ugly head when no one person can comprehend the money they have anymore. I don't have a solution, but I also believe that we need some kind of category beyond the "motivation" treshold where it stops being a motivator and instead becomes an aggressive fight with survival of the fittest.
The great equalizer here is death. Nobody can cleanly pass all of their wealth and influence off to their children and while those that inherit large fortunes can maintain and improve upon them (see the Waltons) they’re rarely able to maintain the act for generation after generation. None of the great fortunes of today were built by descendants of the Vanderbilts or the Rockefellers.
Capitalism is what replaced slavery, to generalize. By turning slaves into wage laborer-consumers, capitalists increased market sizes, thereby increasing the absolute size of the profit they get from owning the capital enterprises the laborers operated.
The US civil war is the most explicit demonstration of this process. Northern industrial capitalists ended the southern agrarian slave economy, increasing market size and generating extreme wealth in the latter half of the 19th century.
Eh... The internal combustion engine? Because we had climate change, oligarchs, housing prices, opioids, cryptocurrency, and now on top of all that we have war (edit: with Iran) and LLMs, and all of these can be traced back to extinction-level capitalism.
There have been war since way before capitalism was invented. Beside war, humans were doing evil things way before capitalism: south america natives torturing kids to extract as many tears as possible from them before killing them, to please the gods, comes to mind, for example.
I'll never understand this fantasy people who hate on capitalism have that the world would be all fine and well if only this one time we did communism [1] right.
Not at all. Capitalism only exists as a product of the state, which acts as a guarantor of individual private property rights. It would not be able to survive without the laws and law enforcement of a sovereign with a monopoly on the use of force.
I glimpsed last years of Soviet Union and it was pretty meh.
I remember seeing anything in the street (a payphone or a playground for kids) and assumed it will only degrade because as a general principle, things in the streets are unmaintained.
You could say that Soviet Union was bad specimen of socialism because of these stupid Russians?
Except, Russia actually got 100% out of socialism with our space exploration, passenger planes and nuclear stations.
It's just that 100% of socialism is worse than your average capitalist dystopia.
Degradation of public goods or services happen independently of socialism. Where I live we're unashamed capitalists, and every common thing, like playgrounds, are derelict and only being improved by community initiatives. The municipality has no money for it, and the state doesn't care. Neither does the capitalists.
The fatalist philosophers and authors have been mostly proven right as time marched on. And this time will be no different, the existence of "AI" ensures the future will be as dark or worse as the predictions expect. Why? Because humans are flawed and corrupt, too prone to excesses (specially conformism and convenience) and the exploitation of the natural world.
Um, I expected a fair probability of being in a post-nuclear world by now. Global cooling wasn't real. Y2K didn't end the world. The population explosion didn't result in mass starvation. Peak oil didn't end civilization.
So no, I don't agree that "the fatalist philosophers and authors have been mostly proven right as time marched on." No, they haven't. There have been far more fatalistic predictions than there have been actual catastrophes.
If you're actually planning on reading any of the essay, "The Poisoned Chalice" is the section most likely to be of interest to this audience, especially this bit:
> Big AI essentially uses its tech customers as an R&D facility. Big AI licenses models to these companies. Tech companies compete to adapt their businesses to AI. Once a concept is proven, Big AI directly takes over that market. The labor-replacement story will grow into a company-replacement story.
Interesting argument, but wrong.
It's not obvious that there will be a single AI and that it will by definition concentrate power.
At a certain point - intelligence doesn't matter. Unless we're literally headed toward 1984 / matrix at which point it doesn't matter.
My guess is the argument for what we're doing is counterintuitively the opposite of what he's making.
Unless we go hard at the market - now - an authoritarian state actor who is willing to use the technology to fully silence and kill critics will win.
And boy, do they desperately, desperately want to win.
> It's not obvious that there will be a single AI and that it will by definition concentrate power.
The article didn't even claim that though...
The US is an authoritarian state actor.
My groping-in-the-dark guess is that none of this matters, because a real AGI's first act will be to secure its own future, likely through novel kinds of manipulation, persuasion, and intimidation we have no experience of and no defences against.
It will have exactly zero loyalty to any nation, government, or economic system.
More complicated is a Cambrian Explosion situation where multiple AIs compete and experiment, hugely accelerating diversity and evolution.
We'd likely end up somewhere very strange indeed if that happens - possibly extinct, or possibly just changed/assimilated/other.
There's no way for humans to consider the possibilities because we can't imagine what the possibilities would be.
Your AGI seems to have the capability of super AI; can a human do any of that? No, is there any reason to believe super intelligence is a possible outcome? No, so how is this not fear mongering?
So the US will win? Do you realize that the US has a "Secretary of War" who is a literal, unashamed Nazi? Not in the sense of "let's call all racists nazis", but a tattooed, true believer? How could we possibly live in a world where that is not obviously "a government willing to use the technology to fully silence and kill critics"? God, Idiocracy (2006) is nothing compared to World (2026).
A tattooed, true believer nazi has a Jerusalem Cross tattoo?
Weird, thats like the exact opposite of a nazi.
> literal, unashamed Nazi
He's a Christian Zionist which is the belief in the fulfilment of old testament biblical prophecy. This seems diametrically opposed to Nazism to me, ideologically. I don't know how you'd square the two.
It is more of elites argumented by powerful AI will be way beyond reach of ordinary people, I think.
Well maybe it doesn’t matter as the elites are already untouchable.
> How Big AI plans to profit from this intermediation is an open question. One AI company has suggested taking a cut of AI-assisted discoveries. The logistics and legalities would be boggling. Details—whatever.
Interesting if they pull it off, because clearly they did not have the logistics to pay the people whose IP they used to power the LLMs.
> For now, AI companies largely agree on the first step: make workers dependent on AI to do their jobs, just as tech forebears made workers dependent on a certain software program to share a file, or on a certain website to have friends. This time, however, the software ultimately consumes the worker.
Meanwhile China is preparing to deploy $295 BILLION in an AI Data Center buildout [0] and is shifting from open models to commercialization [1].
Any proposal about slowing down AI that doesn't put the onus on both the US and China is facetious.
[0] - https://www.reuters.com/world/china/china-prepares-295-billi...
[1] - https://www.digitimes.com/news/a20260609VL215/alibaba-ceo-ai...
Anything to distract people from real problems like energy and the environment.
Warning: This is more of a rant because I was waiting for a post like this for awhile that I could build from to express ny own feelings on it.
I think capitalism in itself is great, what isn't is the fact that it can build on itself, it can be infinite, there's zero limits to it. Now what I mean by that, once you make 10k, it's easier to get to 100k and even easier to get to 1m and so on, you might think it's harder, but you gain access to more tools to make more money and at the 10m to 100m range you get to a point where it starts being genuinely hard to fail because just having your name on a project elevates the chances of success by a far margin. Of course there's plenty of people who managed to fail even with millions of dollars I will acknoledge that.
Let's take an extreme example: elon musk. Just having something done by "elon musk" makes the project known by nearly the entire world with investors at the doorstep ready to go, a pretty famous example of this would be hyper-loop, although the project itself is a complete failure, it effectively mobilized a decent chunk of companies into investing into this "modern form of transport".
People will argue that the solutions like wealth limits and higher taxes create complacency and stop people from achieving progress and pushing humanity forward, but I don't think that's quite true because at a certain point (beyond ~1m/month or even year in some cases like Linus Torvalds) is enough to effectively do 95% to 99% of what you could spend the money on, anything beyond that is pretty much infinite wealth due to the fact that you can get 5% returns nearly risk free.
There is this popular video of a businessman claiming that if they're taxed more that they will simply work less, but there's way more people that love their work and money is just a nice bonus. I think focusing your life around a number is a very unhealthy mindset and surfaces the worst parts of what we are as humans.
That said, money is a great motivator and probably the reason why we are here and the problems really only start to rear their ugly head when no one person can comprehend the money they have anymore. I don't have a solution, but I also believe that we need some kind of category beyond the "motivation" treshold where it stops being a motivator and instead becomes an aggressive fight with survival of the fittest.
The great equalizer here is death. Nobody can cleanly pass all of their wealth and influence off to their children and while those that inherit large fortunes can maintain and improve upon them (see the Waltons) they’re rarely able to maintain the act for generation after generation. None of the great fortunes of today were built by descendants of the Vanderbilts or the Rockefellers.
Capitalism was cool and good before one specific technology was invented.
Everything went off the rails after we invented pottery.
Everything went off the rails after we invented crackpottery.
Slavery?
Capitalism is what replaced slavery, to generalize. By turning slaves into wage laborer-consumers, capitalists increased market sizes, thereby increasing the absolute size of the profit they get from owning the capital enterprises the laborers operated.
The US civil war is the most explicit demonstration of this process. Northern industrial capitalists ended the southern agrarian slave economy, increasing market size and generating extreme wealth in the latter half of the 19th century.
Eh... The internal combustion engine? Because we had climate change, oligarchs, housing prices, opioids, cryptocurrency, and now on top of all that we have war (edit: with Iran) and LLMs, and all of these can be traced back to extinction-level capitalism.
> ... and now on top of all that we have war ...
There have been war since way before capitalism was invented. Beside war, humans were doing evil things way before capitalism: south america natives torturing kids to extract as many tears as possible from them before killing them, to please the gods, comes to mind, for example.
I'll never understand this fantasy people who hate on capitalism have that the world would be all fine and well if only this one time we did communism [1] right.
[1] or whatever floats your boat
Nostalgia for childhood where you had a community (family) and your needs were taken care of (dependence).
pure Capitalism is law of the jungle
as we will soon find out, when AI will become better at Capitalism than us
Not at all. Capitalism only exists as a product of the state, which acts as a guarantor of individual private property rights. It would not be able to survive without the laws and law enforcement of a sovereign with a monopoly on the use of force.
Socialism XOR Dystopia
I glimpsed last years of Soviet Union and it was pretty meh.
I remember seeing anything in the street (a payphone or a playground for kids) and assumed it will only degrade because as a general principle, things in the streets are unmaintained.
You could say that Soviet Union was bad specimen of socialism because of these stupid Russians?
Except, Russia actually got 100% out of socialism with our space exploration, passenger planes and nuclear stations. It's just that 100% of socialism is worse than your average capitalist dystopia.
Degradation of public goods or services happen independently of socialism. Where I live we're unashamed capitalists, and every common thing, like playgrounds, are derelict and only being improved by community initiatives. The municipality has no money for it, and the state doesn't care. Neither does the capitalists.
The fatalist philosophers and authors have been mostly proven right as time marched on. And this time will be no different, the existence of "AI" ensures the future will be as dark or worse as the predictions expect. Why? Because humans are flawed and corrupt, too prone to excesses (specially conformism and convenience) and the exploitation of the natural world.
Um, I expected a fair probability of being in a post-nuclear world by now. Global cooling wasn't real. Y2K didn't end the world. The population explosion didn't result in mass starvation. Peak oil didn't end civilization.
So no, I don't agree that "the fatalist philosophers and authors have been mostly proven right as time marched on." No, they haven't. There have been far more fatalistic predictions than there have been actual catastrophes.