I have a more depressing theory. I think class is part of it, but I'm starting to suspect that a shockingly large number of people lack the critical thinking skills needed to think out the implications of this stuff. I say that because I've met so many people of the managerial class that seem to think it's going to replace the annoying people they have to pay, but somehow they don't think it's going to come for them. It's like we have some sort of society-wide main-character syndrome where a bunch of people just think that somehow the machine can replace every other job, and yet after ingesting all of human knowledge it somehow won't be able to compete with someone with "domain expertise"? Shit, even the cynical people that think their wealth is going to protect them are probably ignoring that if unemployment hits 50+%, society is just going to stay stable and they're going to be safe? I don't think this doomsday scenario is going to happen, but I think the people that are cheering for it need to really consider what they're cheering for.
I don't think its class. I don't think anyone is really thinking about the second order effects. For example, nurses and plumbers--who are in fact among the most insulated professions--fail to realize half their clients will be broke. This is not an exaggeration. If AI really does displace white collar labor and suppress wages, many, many people will default on their mortgages.
This is just one devastating outcome. There will be far more, and for some reason, no one is talking about them.
Some get it, though. When I quipped that Claude may eventually replace me, my manager was visibly shaken as he mentioned it would probably come for him first.
I feel better about it than I did a few months ago. Still seems like there’s something missing that is going to be hard to make happen. I forsee humans being necessary for a long while yet.
> Still seems like there’s something missing that is going to be hard to make happen.
I think part of the negative societal response to AI is the uncertainty of it all. AI killing me, taking my job, augmenting me, curing me of old age, all seem like viable futures within my lifetime given the information I have. People want to know it's going to be okay and even the smartest experts can't credibly promise them that.
Realistically, I've only seen evidence about AI taking people's jobs, as is the case with LLMs, and in the case of robotics, killing people. Augmentation has always been something that gets talked about in research labs but I've never seen an actual product; and curing disease and old age is the domain of more traditional ML methods.
I know Altman has been going around selling visions of curing cancer and the like, but when he talks about standing up new DC or getting a new investment most of that is going towards LLMs, not cancer research.
> society is just going to stay stable and they're going to be safe?
I think this ties in with the rise of racism over the last two decades. Historically, antisemitism has been instrumentalized by the elites in order to deflect animosity from the lower classes: "look, it's not us that are robbing you blind, it's the Jews!"
I see the same thing happening today in the Western world. The elites are squeezing more and more of our countries' wealth, turning governments them into hollow sources of rent, while at the same time backing corrupt politicians who tell us our misery is all because of (illegal) immigrants.
> the people that are cheering for it need to really consider what they're cheering for.
Indeed. To me if there's one reason to oppose the use of AI technology, it is political. Sadly, the SWE class (if it can be called that), being better off than the majority of humans on earth, seems to care very little for class issues and inequality in general, maybe out of self-interest. But as you said, the machine will be coming for them too, it's just a question of when, not if.
Perhaps this is a form of Gell-Mann Amnesia (but kinda inverted) where everyone views AI as too inaccurate for their own niche, but perfectly fine for every other field that they know comparably little about.
I’ve been thinking about this, or a variant of it.
Hypothetically, I’m scared and sad that AI can replace me (it currently can’t, not literally, but a lot of my skill and expertise, built up over say 30 years, that used to be valuable and rare is now cheap to get from an AI).
Let’s try to see the upside. How ‘powerful’ would it make me if, at the cost of my own edge being dulled, can access everyone else’s edge?
I am still my own expert. Now with AI I have a minor expert in everything else as well. What is the best way to use that? don’t have an answer but it’s an aspect I haven’t seen discussed much and I think it is worth bringing up.
As of today, "minor expert" is the wrong way to phrase it. The flagship LLMs today are at best at an initiate or apprentice level in every field.[1] This is not meant as a derisive remark – having something at hand that is initiate level in every field is remarkable and useful. But it's nowhere near expertise anywhere.
The real trouble isn't that it can replace us. Instead, consider that when there have been two comparable technologies in the market, the market has invariably chosen the lowest-quality/worst one. Why? That's easy to understand... while the chosen option is objectively the worst, it's always the cheapest. And cheapest wins. It's not "can AI do his job?" so much as "is the AI cheaper than a human?". And I think we all know the answer to that... even if the silicon's expensive now, volume pricing, data center buildouts, and other economic forces will soon make it cheaper.
The thing that is truly mysterious to the managerial and ruling classes though... when everyone is unemployed, who will be able to afford to buy your junk? Whatever industry you're in, whatever it is you're selling, the people buying that have the money to buy it because they still have jobs. If you're cutting jobs at your company, that helps the bottom line, but every other company is doing the same thing. And they're laying off your customers.
You think every person only thinks of their own job and no one dreams of anything bigger from humanity’s perspective? I’m never going to be in some kind of space colony but do I want to see them happen? You betcha.
I don't see why that should be the case. The only reason software is getting focused on first is:
1. Software devs are obviously going to have a better idea how to apply AI to software development compared to other fields. So of course the coding tools are going to be the first things made.
2. Formal verification makes the problem easier by allowing for iterative feedback (compilers, proofs, etc.)
The second argument is, I think, somewhat valid, but ignores that a lot of other professions also have similar verification systems even if they're a bit less rigorous. The first argument just explains why things are the way they are now, it's not indicative of the future. I don't want to fall into the trap of thinking that other jobs than mine require less cognitive horsepower or whatever, but I don't see what's particularly special about other jobs if it can do hard STEM stuff.
Exactly. Regarding software, it is trained on a massive corpus of code and the feedback loop can be very fast (playing well into LLM's upsides) and results are ... mediocre.
Recently I had to go through some building regulations and Claude's advices were catastrophic.
Or there’s other people like me that know it’s coming for my job but see it as paradigm shifting technology and want to see where it goes. I’m not going to stifle progress just because it’s in my backyard. I think it’s small minded people like you, only preoccupied with your small little niche that can’t see why more open minded people see so much potential in the technology, whether it directly benefits them or not.
One would hope progress directly benefits more than the tech CEOs and investors. They're the ones going around heralding fully automated societies and god-level AIs. Open minded people also recognize the dangers of technological disruption. Let us not forget how social media went from being viewed as utopian to dystopian in a decade.
The title is a little misleading as it sounds like it might hint at the end state of next token prediction - i.e. will we reach AGI with it and so on. Instead, this is entirely about "means of production" and "class war" stuff. While that may be coming, it isn't clear that this is the most likely outcome, its just the most pessimistic one. The author might as well have included that it will "be the end of the human race" as well, since once you get to the future state described, there is no alternative outcome that I can see.
I would argue we're already in a pseudo class war. Most of us in tech are isolated so far, but the middle class is disappearing worldwide, and the wealth gulf is growing.
Most propaganda seems to be centered on increasing division along gender / race / sexuality boundaries, so that infighting keeps people from looking up for too long.
> Non-technical middle managers who have not written a line of code in their lives, now feel that the biggest obstacle between them and greatness has lifted.
I find it interesting how this is almost the “democratization” you mentioned that AI provides. While AI “democratizes” certain technical ability, in some ways the democratization of things can actually be bad, in that this “democratization” pushes us towards a system in which people are completely fungible, and so lose their individual bargaining capability. By democratizing this ability to the non-technical middle manager, the junior software engineer ends up losing their unique contribution and hence vote.
I read a while ago about boycotting AI if you can, and I would love to, but this issue makes me wonder if that could even be effective. If the goal is to remove every unique contribution you provide, what can you take away with a boycott?
I'm thinking of some similarity between the non-technical middle manager using agents to avoid the biggest obstacle of their developers, and a person installing a wall socket at home.
It's not too difficult to install a wall socket, but many things can go wrong, so people would normally call an electrician. In some jurisdictions you are not allowed to install it without a license, or probably you can install it only for yourself but not for anyone else if you don't have a license, and you need to call an inspector and check your work when you're done. Some other jurisdictions truly don't care, you can do whatever, and all the possible damage is on you.
Electricians are somewhat fungible, because you often don't care who will do the job for you, but the profession exists and they can pay their bills.
I wonder how far we are until, at least in some jurisdictions, a person won't be legally allowed to update the website that stores customers' data or processes payments without being a licensed software developer, and how rules, should they be adopted, will change our profession.
> in that this “democratization” pushes us towards a system in which people are completely fungible, and so lose their individual bargaining capability.
Hasn't this been the case since the power loom put hand weavers out of a job?
I hate the term "democratization". It's putting a respectable face on something that shouldn't be respectable at all. In the age of the internet, coding and other creative skills have always been largely democratized to people that care enough to learn. Nothing is being "democratized" by AI, there's simply (an attempt) at driving the value of actual skill to zero so the skill-less and stupid can purchase their way to mediocrity (without the benefit of transferring that money to someone who has worked to be skilled). There is NOTHING "democratic" about that.
It’s not coding that’s being democratized, it’s the ability to create games, tools, etc, that require coding.
Suppose it became possible to buy a Ferrari (not that new hideous one) for $5k. That would be democratizing Ferrari ownership: far more people could do it. I’m sure there would be investment bankers who would complain it devalues their hard work (I am not a fan of those people, but it is notoriously long hours). Does that make a $5k Ferrari less of a democratization?
That's a product, not a skill. We're not talking about people getting cheap cars, we're talking about nobody being able to make a living anymore. And no, this isn't like other revolutions, because there's no "upskilling" into it when it threatens all white collar work. (I'm sorry but I don't think prompting AI is going to be some amazing new profession, it'll be at best a transitory phase). Best case is to buy some durable jeans and learn a trade?
Of course, I don't think this is happening any time soon because I don't believe the hype machine. But if you do believe the hype machine, you should be honest about what you're facing.
Also, for what it's worth as a person that's written (and will continue to write) games and tools etc., there were already plenty of forces democratizing that (free engines, asset stores with reusable code and art, etc.) The difference was that there was still skill involved, and at the end of the day you were paying another human being for their effort if you bought or used pre-existing components. There's no skill in AI, not really.
Also, I'm really not that interested in playing a game created by AI, and judging from the reaction gamers and game developers have had to this technology, I don't think many people are. So, yay, you can make a game that has a tiny audience with zero satisfaction from having done something hard and learned something? Also it's likely going to be extremely derivative, because the AI can only really work with what it's been trained on.
Wait a second, you say its not coding thats getting democratized. It is the "ability to create" software... hm i wonder what would be the ability to write software? Maybe something like software engineering?
Also your example with Ferrari is completely flawed, and I dont give a fuck about what investment bankers think or do. What are you even talking about here?
Please explain the first part more than you hand waved away to make a completely unrelated metaphorical case. What is the ability to create software if not software engineering? How are both different?
>But lately I’ve been thinking if it is just a class issue? This cohort of people likely have a cushion that softens the concussive blows they are doling out right now. They perhaps have the luxury of a somewhat functioning government and a social safety net that they are witness to in all walks of life. Over half the world does not. Science and technology, I feel, has always had a certain apathy towards the plight the people at the bottom rungs.
In the data I've seen, the US and European countries have a more negative view of AI than China and developing countries. Doesn't that fly in the face of the premise here that only people that have economic security support AI?
One large part of this is that all the heads of AI companies and business leaders in the US and Europe keep talking about how AI is going to take jobs away and displace "lower-value human capital", while we see power shortages and higher energy rates in areas where AI datacenters are built. Meanwhile, China's 15th 5 year plan involves integrating AI across the whole economy while expanding vocational retraining programs, and building out new renewable infrastructure to power datacenters. The Chinese Human Resources Ministry expects AI to create 6-10 million new jobs in the short term, and the Chinese government plans to use it in the long term to fill in gaps in the labor force caused by its demographic shortages.
I think this is a big reason why the Chinese have a more positive view of AI than the West: their leaders have a clear plan to mitigate the negative externalities of introducing AI, and ours don't.
The classic progression of an economy is resource -> manufacturing -> knowledge.
AI turns this line into a circle by making knowledge a resource problem. Less developed economies with a lot of natural resources and manufacturing like China's are less at risk than heavily knowledge-based economies like Europe's.
Seems to me that it is downstream from the fact that China's economy is growing strongly and strong state power means that they see infrastructural improvements. US and European governments reflect the views of their people which are generally retrogressive and aimed at a fictional view of the 1970s as described in The Simpsons. Consequently, the Chinese are enthusiastic about Nuclear Power, and Solar Power, and Wind Energy, and AI, and ship building, and space programs, and trains, and electric vehicles, and so on ad infinitum to the degree that they don't mind smoggy cities to get these. Meanwhile Western nations mostly want to live in whatever they already have and would prefer nothing change or if it does that this change moves them closer to a past world, while nonetheless enjoying clean air and water.
I, personally, think that this is somewhat like hoping that mining coal will lead to a great leap forward in development because mining coal led in the past to a great leap forward in development.
Maybe it has to do with % of blue collar vs. white collar jobs in those countries? I also wonder if they have stronger safety nets for displaced workers? It is curious. My anecdata has shown that people who feel "safe" job wise are either neutral or pro. Otherwise negative.
> In the data I've seen, the US and European countries have a more negative view of AI than China and developing countries.
I think that's more a sign of the relative state of these economies and the rate of progress. In developing economies, people see progress as something that will improve their lives. In developed economies, they see it as something that will disrupt their current status quo and must be stopped.
From what I've seen, The Chinese are more likely to believe the government has their back and the benefits of AI will be spread across the population. While Americans believe it will go towards billionaire mega yachts while they starve on the street.
The attitudes aren’t 1-1 comparable. China is on a winning streak in terms of socioeconomic development, and AI is likely seen as merely a new technology in the context of the social contract. The US is going the opposite way, and people here view AI through the lens of oligarchy more often than not. I wouldn’t say that a lot of people feel as optimistic, even if they are actually more economically secure.
Do you have direct experience with this? From what I understand China has huge youth employment issues right now, and the 35 and out (at even non tech companies) meme has some basis in reality.
China historically has had a poor social safety net, but made up for it with a more dynamic labor market (well, we could say the same about the USA vs Europe).
The idea of the social contract impacting perceptions of AI is interesting to me. I hate to use the words “permanent underclass”, but perhaps the main difference is a fear of that permanent underclass actually materializing. In the US, it seems that that would be the logical endpoint of the capitalist system and many people predict AI simply replacing them permanently. Of course, China is not completely communist, but since their social contract is much less individualistic and more collectivist, maybe that makes people see AI as much more likely to uplift society as a whole or at least “trickle down”.
I think this might be a bigger reason as China’s economy for the youth isn’t looking the brightest right now.
People have bargaining power through their military services. If even that job is taken by AI, there is truly no recourse for the people left at all. These are the sentiments of Yuval Noah Harari.
Every time someone invokes the "playing God" / "the hubris of creation" card, it makes me think they're the proud ones, and this case is none the different.
I don't know why the author is so surprised people want freedom from others. We're the original bullshit machines, and with every useful invention, an additional chunk of utter snakeoil is snuffed out. We're not particularly reliable either. In an old post I can't find for example, I remarked how people can apparently do figure out how to document and coach properly, as long as the target is an LLM, not a human. Suddenly the limits and importance of attention, contextualization, clarity, working memory size, etc. are not so elusive and debatable after all.
I'm sure I need not to remind the author of the "certainty of steel" quote, as ironic as that'd be for an indeed inherently stochastic system. A parrot though, I'm not so sure. "Not sure" why the author feels compelled to conditionally deny it is absolutely meant to be pejorative either.
> So, where does next token prediction leave us? In a perpetual loop of rent-seeking for something made with humanity’s collective output of centuries. It is not a good place for an individual to be in, regardless of class.
The question is what classes of jobs will replace the ones made obsolete by a particular innovation, and how many vacancies there will be. With electricity we had managerial, creative, high dexterity, etc. jobs. If a sufficient amount of human capabilities are replaced then that may be different to electricity. Also it's about how many top-tier "power plants" (AI, robotics) there are, who owns them, and once "everything is solved" (which it isn't yet) how much of that will trickle down to everyone else's life quality.
Calling this a straw man would be attributing an unnecessarily sturdy construction.
Yes there are people making stupid claims on all sides. Attributing phrasing like solved or cooked is as if it is some sort of fanatic specific jargon simply ignores the terminology of different groups of people. I don't use cooked myself, but I am not so ignorant of the younger generation that is see it is just another in the long line of terms like sick, bogus, hip, radical, macaroni, etc.
The author plays the trick of flipping the situation from stochastic parrots or next token prediction. Those are "taken as pejorative" whereas cooked and solved "to signal"
The flip is done to place the fault on the other party. You could equally uncharitably say that invoking next token prediction or stochastic parrots is signalling, whereas AI skeptics take terms like cooked as pejorative.
Specifically on the topic of next token prediction, we are already past that phase. Even then I don't think that a model trained by prediction has the limitations that people think it does. As a thought terminating cliche it is simply obsolete when models are trained on reinforcement techniques where there is no template next word to predict. Diffusion models don't even have an autoregressive nature.
I am not terribly fond of the claims made by people at the extremes of any particular to issue. We can perhaps try debating the facts of the matter rather than assuming the internal thoughts of people who might disagree with you.
I generally do not attribute malice to people who describe models as next word predictors. Most are simply uninformed and if you query what their understanding is of a model then you see that what they are imagining is a Markov chain. Investigating if their imagined model could correctly use "an" before "alligator" but obviously not choosing an animal beginning with a vowel just because it had just said "an" often leads them to think that there's more going on than just the next word.
> Specifically on the topic of next token prediction, we are already past that phase.
We really aren't past that phase at all. Reasoning models are just next-token prediction trained in a way where it thinks out loud, essentially. (Source: books on how LLMs actually work, and asking ChatGPT directly!) Harnesses and tool use help a little bit, but it doesn't change fundamentally what an LLM is.
> The compartmentalisation that must be required by the scientists and engineers to reconcile with the fact that their work being used to bomb and kill people must be crazy.
I think about a related question pretty often: What proportion of people working at these companies are "true believers", that their work will be a net benefit for humanity? And for those people (if they are at all numerous), how do they plan to fight back against the obvious harms that are already occurring?
I just can't imagine working at one of these companies without hating myself. But I suppose with what they're being paid, they can afford a very good therapist...
If one works for a gun manufacturer, should they feel personally guilty when crimes are committed? What about when police arrest a criminal without injury? Perhaps the balance is determined whether the viewpoint is from one of killing or one of deterrence.
If a doctor provides medical care that extends a to-be murder’s lifespan, is that a good thing? Sure, hopefully most patients aren’t and the provided care is a “net positive”, but does that make it okay?
Sure, one can say, I’ll do paper sales at Dunder Miflin and not have to worry about these problems. Few have been murdered by paper cut. However if they aren’t the #1 paper supplier for almost every “evil” entity one can imagine, it’s ONLY because they failed to do so. It’s easy to pretend to be virtuous after failing otherwise.
That said, I’m not sure if I have ever met any true believers. The executives that claim to be clearly aren’t. The intellectually curious are motivated by the problem, not the product.
> If one works for a gun manufacturer, should they feel personally guilty when crimes are committed?
I would be very conflicted, because the inherent purpose of a gun is its ability to harm people, regardless of whether any given concrete case constitutes a crime or not.
> If one works for a gun manufacturer, should they feel personally guilty when crimes are committed?
I don't think this analogy holds. You could use a gun to commit a crime, but you could also use a gun to defend yourself. On the other hand, if your CEO is talking about getting rid of all labor, well, you're kind of complicit in the crime if you keep working there. There's no ambiguity as to "what will this be used for", like there is with a gun.
Honestly, I'm baffled that a software engineer would want to work at those companies. Like, you think they're going to keep you around once their AI is good enough to improve itself? Or are they just holding out hope that they can get their bag before billionaires create a permanent underclass, and they can be a step above the untouchables?
This author fundamentally doesn't understand the mental model of the people they are describing and makes huge sweeping claims in what they think is a in a savvy manner but they are completely wrong.
Here's the RIGHT mental model of the people the author is talking about.
1. If AI is good enough that it can boost productivity by 20% then it is good for society in general because the gains will be redistributed (as it has always been). So even if it is ME who is getting laid off, I will still say it is still good for society because that's how progress happens - by breaking eggs to make omelettes
2. If AI is so good that it can replace full professions altogether like Mathematics, it is a profound joyful moment for humanity. What better thing can happen to the curious ones amongst us to get an oracle that can answer every question? Why does the author seem to scoff at this?
3. If AI is so good that it is a complete superset of humans itself, it is much much more profound moment when civilisation will be changed in ways that English doesn't even have the vocabulary to describe. It can't be stopped nor is it clear that it should.
The author is in curious and has a bad mental model of the people they are describing. They say it is a "class" issue and bring old outdated Marxist terminology to prop up their weak argument.
> Science and technology, I feel, has always had a certain apathy towards the plight [of] the people at the bottom rungs.
Does that apply to medical advances too? e.g. antibiotics, vaccines, etc. too? We are living longer today thanks to advances in science and technology. Not just the people at the top; but also the peopl at the bottom rungs. Most scientific research does not take into account who the beneficiaries of that research would be.
Especially basic scientific research is so far removed from actually useful applications that there is no direct cause and effect relationship between single items of research and any eventual effect on society.
Were the vaccines accessible because of the sympathy towards the people at the bottom, or so that they can go back to work as soon as possible? Medicine is subsidised when there is a global pandemic, but not when individual lives are at stake.
I have a more depressing theory. I think class is part of it, but I'm starting to suspect that a shockingly large number of people lack the critical thinking skills needed to think out the implications of this stuff. I say that because I've met so many people of the managerial class that seem to think it's going to replace the annoying people they have to pay, but somehow they don't think it's going to come for them. It's like we have some sort of society-wide main-character syndrome where a bunch of people just think that somehow the machine can replace every other job, and yet after ingesting all of human knowledge it somehow won't be able to compete with someone with "domain expertise"? Shit, even the cynical people that think their wealth is going to protect them are probably ignoring that if unemployment hits 50+%, society is just going to stay stable and they're going to be safe? I don't think this doomsday scenario is going to happen, but I think the people that are cheering for it need to really consider what they're cheering for.
I don't think its class. I don't think anyone is really thinking about the second order effects. For example, nurses and plumbers--who are in fact among the most insulated professions--fail to realize half their clients will be broke. This is not an exaggeration. If AI really does displace white collar labor and suppress wages, many, many people will default on their mortgages.
This is just one devastating outcome. There will be far more, and for some reason, no one is talking about them.
Some get it, though. When I quipped that Claude may eventually replace me, my manager was visibly shaken as he mentioned it would probably come for him first.
I feel better about it than I did a few months ago. Still seems like there’s something missing that is going to be hard to make happen. I forsee humans being necessary for a long while yet.
> Still seems like there’s something missing that is going to be hard to make happen.
I think part of the negative societal response to AI is the uncertainty of it all. AI killing me, taking my job, augmenting me, curing me of old age, all seem like viable futures within my lifetime given the information I have. People want to know it's going to be okay and even the smartest experts can't credibly promise them that.
Realistically, I've only seen evidence about AI taking people's jobs, as is the case with LLMs, and in the case of robotics, killing people. Augmentation has always been something that gets talked about in research labs but I've never seen an actual product; and curing disease and old age is the domain of more traditional ML methods.
I know Altman has been going around selling visions of curing cancer and the like, but when he talks about standing up new DC or getting a new investment most of that is going towards LLMs, not cancer research.
The billionaires building doomsday bunkers get it.
> society is just going to stay stable and they're going to be safe?
I think this ties in with the rise of racism over the last two decades. Historically, antisemitism has been instrumentalized by the elites in order to deflect animosity from the lower classes: "look, it's not us that are robbing you blind, it's the Jews!"
I see the same thing happening today in the Western world. The elites are squeezing more and more of our countries' wealth, turning governments them into hollow sources of rent, while at the same time backing corrupt politicians who tell us our misery is all because of (illegal) immigrants.
> the people that are cheering for it need to really consider what they're cheering for.
Indeed. To me if there's one reason to oppose the use of AI technology, it is political. Sadly, the SWE class (if it can be called that), being better off than the majority of humans on earth, seems to care very little for class issues and inequality in general, maybe out of self-interest. But as you said, the machine will be coming for them too, it's just a question of when, not if.
Perhaps this is a form of Gell-Mann Amnesia (but kinda inverted) where everyone views AI as too inaccurate for their own niche, but perfectly fine for every other field that they know comparably little about.
I’ve been thinking about this, or a variant of it.
Hypothetically, I’m scared and sad that AI can replace me (it currently can’t, not literally, but a lot of my skill and expertise, built up over say 30 years, that used to be valuable and rare is now cheap to get from an AI).
Let’s try to see the upside. How ‘powerful’ would it make me if, at the cost of my own edge being dulled, can access everyone else’s edge?
I am still my own expert. Now with AI I have a minor expert in everything else as well. What is the best way to use that? don’t have an answer but it’s an aspect I haven’t seen discussed much and I think it is worth bringing up.
As of today, "minor expert" is the wrong way to phrase it. The flagship LLMs today are at best at an initiate or apprentice level in every field.[1] This is not meant as a derisive remark – having something at hand that is initiate level in every field is remarkable and useful. But it's nowhere near expertise anywhere.
[1]: https://entropicthoughts.com/stop-using-junior-and-senior
The real trouble isn't that it can replace us. Instead, consider that when there have been two comparable technologies in the market, the market has invariably chosen the lowest-quality/worst one. Why? That's easy to understand... while the chosen option is objectively the worst, it's always the cheapest. And cheapest wins. It's not "can AI do his job?" so much as "is the AI cheaper than a human?". And I think we all know the answer to that... even if the silicon's expensive now, volume pricing, data center buildouts, and other economic forces will soon make it cheaper.
The thing that is truly mysterious to the managerial and ruling classes though... when everyone is unemployed, who will be able to afford to buy your junk? Whatever industry you're in, whatever it is you're selling, the people buying that have the money to buy it because they still have jobs. If you're cutting jobs at your company, that helps the bottom line, but every other company is doing the same thing. And they're laying off your customers.
You think every person only thinks of their own job and no one dreams of anything bigger from humanity’s perspective? I’m never going to be in some kind of space colony but do I want to see them happen? You betcha.
Actually I think AI will largely automate software and math and really not much else in the short to medium term. (speaking as a computer/math person)
I don't see why that should be the case. The only reason software is getting focused on first is:
1. Software devs are obviously going to have a better idea how to apply AI to software development compared to other fields. So of course the coding tools are going to be the first things made.
2. Formal verification makes the problem easier by allowing for iterative feedback (compilers, proofs, etc.)
The second argument is, I think, somewhat valid, but ignores that a lot of other professions also have similar verification systems even if they're a bit less rigorous. The first argument just explains why things are the way they are now, it's not indicative of the future. I don't want to fall into the trap of thinking that other jobs than mine require less cognitive horsepower or whatever, but I don't see what's particularly special about other jobs if it can do hard STEM stuff.
Exactly. Regarding software, it is trained on a massive corpus of code and the feedback loop can be very fast (playing well into LLM's upsides) and results are ... mediocre.
Recently I had to go through some building regulations and Claude's advices were catastrophic.
It's normal Gell-Mann amnesia and, yeah, it's a really big part of why AI is so accepted.
Or there’s other people like me that know it’s coming for my job but see it as paradigm shifting technology and want to see where it goes. I’m not going to stifle progress just because it’s in my backyard. I think it’s small minded people like you, only preoccupied with your small little niche that can’t see why more open minded people see so much potential in the technology, whether it directly benefits them or not.
One would hope progress directly benefits more than the tech CEOs and investors. They're the ones going around heralding fully automated societies and god-level AIs. Open minded people also recognize the dangers of technological disruption. Let us not forget how social media went from being viewed as utopian to dystopian in a decade.
The title is a little misleading as it sounds like it might hint at the end state of next token prediction - i.e. will we reach AGI with it and so on. Instead, this is entirely about "means of production" and "class war" stuff. While that may be coming, it isn't clear that this is the most likely outcome, its just the most pessimistic one. The author might as well have included that it will "be the end of the human race" as well, since once you get to the future state described, there is no alternative outcome that I can see.
I would argue we're already in a pseudo class war. Most of us in tech are isolated so far, but the middle class is disappearing worldwide, and the wealth gulf is growing.
Most propaganda seems to be centered on increasing division along gender / race / sexuality boundaries, so that infighting keeps people from looking up for too long.
But I might just be overly radicalised.
> Non-technical middle managers who have not written a line of code in their lives, now feel that the biggest obstacle between them and greatness has lifted.
I find it interesting how this is almost the “democratization” you mentioned that AI provides. While AI “democratizes” certain technical ability, in some ways the democratization of things can actually be bad, in that this “democratization” pushes us towards a system in which people are completely fungible, and so lose their individual bargaining capability. By democratizing this ability to the non-technical middle manager, the junior software engineer ends up losing their unique contribution and hence vote.
I read a while ago about boycotting AI if you can, and I would love to, but this issue makes me wonder if that could even be effective. If the goal is to remove every unique contribution you provide, what can you take away with a boycott?
I'm thinking of some similarity between the non-technical middle manager using agents to avoid the biggest obstacle of their developers, and a person installing a wall socket at home.
It's not too difficult to install a wall socket, but many things can go wrong, so people would normally call an electrician. In some jurisdictions you are not allowed to install it without a license, or probably you can install it only for yourself but not for anyone else if you don't have a license, and you need to call an inspector and check your work when you're done. Some other jurisdictions truly don't care, you can do whatever, and all the possible damage is on you.
Electricians are somewhat fungible, because you often don't care who will do the job for you, but the profession exists and they can pay their bills.
I wonder how far we are until, at least in some jurisdictions, a person won't be legally allowed to update the website that stores customers' data or processes payments without being a licensed software developer, and how rules, should they be adopted, will change our profession.
> in that this “democratization” pushes us towards a system in which people are completely fungible, and so lose their individual bargaining capability.
Hasn't this been the case since the power loom put hand weavers out of a job?
I hate the term "democratization". It's putting a respectable face on something that shouldn't be respectable at all. In the age of the internet, coding and other creative skills have always been largely democratized to people that care enough to learn. Nothing is being "democratized" by AI, there's simply (an attempt) at driving the value of actual skill to zero so the skill-less and stupid can purchase their way to mediocrity (without the benefit of transferring that money to someone who has worked to be skilled). There is NOTHING "democratic" about that.
It’s not coding that’s being democratized, it’s the ability to create games, tools, etc, that require coding.
Suppose it became possible to buy a Ferrari (not that new hideous one) for $5k. That would be democratizing Ferrari ownership: far more people could do it. I’m sure there would be investment bankers who would complain it devalues their hard work (I am not a fan of those people, but it is notoriously long hours). Does that make a $5k Ferrari less of a democratization?
That's a product, not a skill. We're not talking about people getting cheap cars, we're talking about nobody being able to make a living anymore. And no, this isn't like other revolutions, because there's no "upskilling" into it when it threatens all white collar work. (I'm sorry but I don't think prompting AI is going to be some amazing new profession, it'll be at best a transitory phase). Best case is to buy some durable jeans and learn a trade?
Of course, I don't think this is happening any time soon because I don't believe the hype machine. But if you do believe the hype machine, you should be honest about what you're facing.
Also, for what it's worth as a person that's written (and will continue to write) games and tools etc., there were already plenty of forces democratizing that (free engines, asset stores with reusable code and art, etc.) The difference was that there was still skill involved, and at the end of the day you were paying another human being for their effort if you bought or used pre-existing components. There's no skill in AI, not really.
Also, I'm really not that interested in playing a game created by AI, and judging from the reaction gamers and game developers have had to this technology, I don't think many people are. So, yay, you can make a game that has a tiny audience with zero satisfaction from having done something hard and learned something? Also it's likely going to be extremely derivative, because the AI can only really work with what it's been trained on.
They’re letting us test drive the Ferrari while they extract our data.
Then they’ll raise the price.
Wait a second, you say its not coding thats getting democratized. It is the "ability to create" software... hm i wonder what would be the ability to write software? Maybe something like software engineering?
Also your example with Ferrari is completely flawed, and I dont give a fuck about what investment bankers think or do. What are you even talking about here?
Please explain the first part more than you hand waved away to make a completely unrelated metaphorical case. What is the ability to create software if not software engineering? How are both different?
> democratizing this ability to the non-technical middle manager
Works the other way too. Now a junior engineer can use AI to do much of what a middle manager had been doing in the past.
Frankly I think the middle manager ought to be WAY more worried than most.
Coding was always "democratized". You just had to put in the effort of learning and understanding. Finding free resources is very easy.
AI doesn't make that learning and understanding easy but just allows people to skip it.
That doesn't mean democratization at all.
[dead]
>But lately I’ve been thinking if it is just a class issue? This cohort of people likely have a cushion that softens the concussive blows they are doling out right now. They perhaps have the luxury of a somewhat functioning government and a social safety net that they are witness to in all walks of life. Over half the world does not. Science and technology, I feel, has always had a certain apathy towards the plight the people at the bottom rungs.
In the data I've seen, the US and European countries have a more negative view of AI than China and developing countries. Doesn't that fly in the face of the premise here that only people that have economic security support AI?
https://www.visualcapitalist.com/survey-how-21-countries-vie...
https://www.ipsos.com/en/conflicting-global-perceptions-arou...
https://www.mexc.com/news/161986
One large part of this is that all the heads of AI companies and business leaders in the US and Europe keep talking about how AI is going to take jobs away and displace "lower-value human capital", while we see power shortages and higher energy rates in areas where AI datacenters are built. Meanwhile, China's 15th 5 year plan involves integrating AI across the whole economy while expanding vocational retraining programs, and building out new renewable infrastructure to power datacenters. The Chinese Human Resources Ministry expects AI to create 6-10 million new jobs in the short term, and the Chinese government plans to use it in the long term to fill in gaps in the labor force caused by its demographic shortages.
I think this is a big reason why the Chinese have a more positive view of AI than the West: their leaders have a clear plan to mitigate the negative externalities of introducing AI, and ours don't.
The classic progression of an economy is resource -> manufacturing -> knowledge.
AI turns this line into a circle by making knowledge a resource problem. Less developed economies with a lot of natural resources and manufacturing like China's are less at risk than heavily knowledge-based economies like Europe's.
Seems to me that it is downstream from the fact that China's economy is growing strongly and strong state power means that they see infrastructural improvements. US and European governments reflect the views of their people which are generally retrogressive and aimed at a fictional view of the 1970s as described in The Simpsons. Consequently, the Chinese are enthusiastic about Nuclear Power, and Solar Power, and Wind Energy, and AI, and ship building, and space programs, and trains, and electric vehicles, and so on ad infinitum to the degree that they don't mind smoggy cities to get these. Meanwhile Western nations mostly want to live in whatever they already have and would prefer nothing change or if it does that this change moves them closer to a past world, while nonetheless enjoying clean air and water.
I, personally, think that this is somewhat like hoping that mining coal will lead to a great leap forward in development because mining coal led in the past to a great leap forward in development.
Maybe it has to do with % of blue collar vs. white collar jobs in those countries? I also wonder if they have stronger safety nets for displaced workers? It is curious. My anecdata has shown that people who feel "safe" job wise are either neutral or pro. Otherwise negative.
> In the data I've seen, the US and European countries have a more negative view of AI than China and developing countries.
I think that's more a sign of the relative state of these economies and the rate of progress. In developing economies, people see progress as something that will improve their lives. In developed economies, they see it as something that will disrupt their current status quo and must be stopped.
Maybe chinese are generally more positive about everything. Many European countries and the US are in a decline for some years now.
From what I've seen, The Chinese are more likely to believe the government has their back and the benefits of AI will be spread across the population. While Americans believe it will go towards billionaire mega yachts while they starve on the street.
The attitudes aren’t 1-1 comparable. China is on a winning streak in terms of socioeconomic development, and AI is likely seen as merely a new technology in the context of the social contract. The US is going the opposite way, and people here view AI through the lens of oligarchy more often than not. I wouldn’t say that a lot of people feel as optimistic, even if they are actually more economically secure.
Do you have direct experience with this? From what I understand China has huge youth employment issues right now, and the 35 and out (at even non tech companies) meme has some basis in reality.
China historically has had a poor social safety net, but made up for it with a more dynamic labor market (well, we could say the same about the USA vs Europe).
The idea of the social contract impacting perceptions of AI is interesting to me. I hate to use the words “permanent underclass”, but perhaps the main difference is a fear of that permanent underclass actually materializing. In the US, it seems that that would be the logical endpoint of the capitalist system and many people predict AI simply replacing them permanently. Of course, China is not completely communist, but since their social contract is much less individualistic and more collectivist, maybe that makes people see AI as much more likely to uplift society as a whole or at least “trickle down”.
I think this might be a bigger reason as China’s economy for the youth isn’t looking the brightest right now.
It’s a cliff we’re all being push to. Don’t know about you, I’m just struggling to put on my parachute !!!
Where did you find a parachute?
People have bargaining power through their military services. If even that job is taken by AI, there is truly no recourse for the people left at all. These are the sentiments of Yuval Noah Harari.
Author seems ill informed. The D.O.W is not exactly a fan of Anthropic…
It’s a case of sour grapes though, in reality they’d still want Anthropic’s grapes.
They're not now, but they went so far as to strong arm them because they wanted their product.
For a more thoughtful and very different analysis:
"Implications Of Predicting The Next Token"
https://minihf.com/posts/2026-05-07-implications-of-predicti...
Every time someone invokes the "playing God" / "the hubris of creation" card, it makes me think they're the proud ones, and this case is none the different.
I don't know why the author is so surprised people want freedom from others. We're the original bullshit machines, and with every useful invention, an additional chunk of utter snakeoil is snuffed out. We're not particularly reliable either. In an old post I can't find for example, I remarked how people can apparently do figure out how to document and coach properly, as long as the target is an LLM, not a human. Suddenly the limits and importance of attention, contextualization, clarity, working memory size, etc. are not so elusive and debatable after all.
I'm sure I need not to remind the author of the "certainty of steel" quote, as ironic as that'd be for an indeed inherently stochastic system. A parrot though, I'm not so sure. "Not sure" why the author feels compelled to conditionally deny it is absolutely meant to be pejorative either.
One does not need to be blind to the mentioned prospective pitfalls either: https://news.ycombinator.com/item?id=48196923
Don't worry guys, if the situation gets too bad, the social contract breaks, heads will roll for a while and then it will get better.
"Inevitable"
> So, where does next token prediction leave us? In a perpetual loop of rent-seeking for something made with humanity’s collective output of centuries. It is not a good place for an individual to be in, regardless of class.
In what way is this different to electricity?
The question is what classes of jobs will replace the ones made obsolete by a particular innovation, and how many vacancies there will be. With electricity we had managerial, creative, high dexterity, etc. jobs. If a sufficient amount of human capabilities are replaced then that may be different to electricity. Also it's about how many top-tier "power plants" (AI, robotics) there are, who owns them, and once "everything is solved" (which it isn't yet) how much of that will trickle down to everyone else's life quality.
Calling this a straw man would be attributing an unnecessarily sturdy construction.
Yes there are people making stupid claims on all sides. Attributing phrasing like solved or cooked is as if it is some sort of fanatic specific jargon simply ignores the terminology of different groups of people. I don't use cooked myself, but I am not so ignorant of the younger generation that is see it is just another in the long line of terms like sick, bogus, hip, radical, macaroni, etc.
The author plays the trick of flipping the situation from stochastic parrots or next token prediction. Those are "taken as pejorative" whereas cooked and solved "to signal"
The flip is done to place the fault on the other party. You could equally uncharitably say that invoking next token prediction or stochastic parrots is signalling, whereas AI skeptics take terms like cooked as pejorative.
Specifically on the topic of next token prediction, we are already past that phase. Even then I don't think that a model trained by prediction has the limitations that people think it does. As a thought terminating cliche it is simply obsolete when models are trained on reinforcement techniques where there is no template next word to predict. Diffusion models don't even have an autoregressive nature.
I am not terribly fond of the claims made by people at the extremes of any particular to issue. We can perhaps try debating the facts of the matter rather than assuming the internal thoughts of people who might disagree with you.
I generally do not attribute malice to people who describe models as next word predictors. Most are simply uninformed and if you query what their understanding is of a model then you see that what they are imagining is a Markov chain. Investigating if their imagined model could correctly use "an" before "alligator" but obviously not choosing an animal beginning with a vowel just because it had just said "an" often leads them to think that there's more going on than just the next word.
> Specifically on the topic of next token prediction, we are already past that phase.
We really aren't past that phase at all. Reasoning models are just next-token prediction trained in a way where it thinks out loud, essentially. (Source: books on how LLMs actually work, and asking ChatGPT directly!) Harnesses and tool use help a little bit, but it doesn't change fundamentally what an LLM is.
> The compartmentalisation that must be required by the scientists and engineers to reconcile with the fact that their work being used to bomb and kill people must be crazy.
I think about a related question pretty often: What proportion of people working at these companies are "true believers", that their work will be a net benefit for humanity? And for those people (if they are at all numerous), how do they plan to fight back against the obvious harms that are already occurring?
I just can't imagine working at one of these companies without hating myself. But I suppose with what they're being paid, they can afford a very good therapist...
I’ve wondered about this.
If one works for a gun manufacturer, should they feel personally guilty when crimes are committed? What about when police arrest a criminal without injury? Perhaps the balance is determined whether the viewpoint is from one of killing or one of deterrence.
If a doctor provides medical care that extends a to-be murder’s lifespan, is that a good thing? Sure, hopefully most patients aren’t and the provided care is a “net positive”, but does that make it okay?
Sure, one can say, I’ll do paper sales at Dunder Miflin and not have to worry about these problems. Few have been murdered by paper cut. However if they aren’t the #1 paper supplier for almost every “evil” entity one can imagine, it’s ONLY because they failed to do so. It’s easy to pretend to be virtuous after failing otherwise.
That said, I’m not sure if I have ever met any true believers. The executives that claim to be clearly aren’t. The intellectually curious are motivated by the problem, not the product.
> If one works for a gun manufacturer, should they feel personally guilty when crimes are committed?
I would be very conflicted, because the inherent purpose of a gun is its ability to harm people, regardless of whether any given concrete case constitutes a crime or not.
> If one works for a gun manufacturer, should they feel personally guilty when crimes are committed?
I don't think this analogy holds. You could use a gun to commit a crime, but you could also use a gun to defend yourself. On the other hand, if your CEO is talking about getting rid of all labor, well, you're kind of complicit in the crime if you keep working there. There's no ambiguity as to "what will this be used for", like there is with a gun.
Honestly, I'm baffled that a software engineer would want to work at those companies. Like, you think they're going to keep you around once their AI is good enough to improve itself? Or are they just holding out hope that they can get their bag before billionaires create a permanent underclass, and they can be a step above the untouchables?
This author fundamentally doesn't understand the mental model of the people they are describing and makes huge sweeping claims in what they think is a in a savvy manner but they are completely wrong.
Here's the RIGHT mental model of the people the author is talking about.
1. If AI is good enough that it can boost productivity by 20% then it is good for society in general because the gains will be redistributed (as it has always been). So even if it is ME who is getting laid off, I will still say it is still good for society because that's how progress happens - by breaking eggs to make omelettes
2. If AI is so good that it can replace full professions altogether like Mathematics, it is a profound joyful moment for humanity. What better thing can happen to the curious ones amongst us to get an oracle that can answer every question? Why does the author seem to scoff at this?
3. If AI is so good that it is a complete superset of humans itself, it is much much more profound moment when civilisation will be changed in ways that English doesn't even have the vocabulary to describe. It can't be stopped nor is it clear that it should.
The author is in curious and has a bad mental model of the people they are describing. They say it is a "class" issue and bring old outdated Marxist terminology to prop up their weak argument.
I tuned out when I came across this:
> Science and technology, I feel, has always had a certain apathy towards the plight [of] the people at the bottom rungs.
Does that apply to medical advances too? e.g. antibiotics, vaccines, etc. too? We are living longer today thanks to advances in science and technology. Not just the people at the top; but also the peopl at the bottom rungs. Most scientific research does not take into account who the beneficiaries of that research would be.
Especially basic scientific research is so far removed from actually useful applications that there is no direct cause and effect relationship between single items of research and any eventual effect on society.
>Most scientific research does not take into account who the beneficiaries of that research would be.
I would consider that apathy.
do people at the bottom have access to medical advances or are they in lifelong debt in the unfortunate case when they need those medical advances?
How accessible were covid vaccines?
It costs between 4k and 10k if you break your leg and call an ambulance in the US. Forget medical advances, it costs you a car to get a cast
Were the vaccines accessible because of the sympathy towards the people at the bottom, or so that they can go back to work as soon as possible? Medicine is subsidised when there is a global pandemic, but not when individual lives are at stake.
[flagged]
[flagged]