I would summarize the central claim of the paper as: the widespread use of AI to mediate human interaction will rob people of agency, understanding and skill development, as well as destroying the social the social links necessary to maintain and improve institutions, while at the same time allowing powerful unaccountable actors (AI cabal) to interject into those relations and impose their institutional goals; by "institution" we mean a shared set of beneficial social rules, not merely an organization tasked with promoting them, say "justice" vs. "US justice system".
The authors then break down the mechanisms by which AI achieves these outcome (that seem quite reductive and dated compared to the frontier, for example they take it as granted that AI cannot be creative, that it can only work prospectively and can't react to new situations and events etc.), as well as exemplifying those mechanism in a few areas like journalism, academia etc.
> We should be more worried what AI will due to the ability of an average human to think.
I had a wake up call on this yesterday. After a recent HN thread about Zed editor, I decided to give it another try, so I loaded it up, disabled AI, and tried writing some code from scratch. No AI completion, no intellisence. Two things came to mind. First, my editor seems so much more peaceful without being told what to do. Second, it was a bit scary how lost I felt. It was obvious that my own ability to communicate through code had declined a bit since I began using AI coding assistants. It turns out that as expected, coding assistants really are competitive cognitive artifacts. After that experience, I've decided that I am going to do at least part of my coding with all completions turned off. Unfortunately at work you are paid to produce quickly, so I think my AI free editor will have to be reserved for personal projects.
Further related to your statement about thought, the hallucinations persist, and even last night I got a response about 80's pop culture that was over 50% bullshit. Just imagine what intentional persuasion through LLM models will do to society. Independent thought has never been more important.
I mean we were seeing this even before AI. It's the same type of person. To slop is human.
It's like for some reason we thought that like some good percentage of us aren't just tribal worker drones who fundamentally just want fats, sugars, salts, dopamine and seratonin. People actively vote against things like UBI, higher corporate taxes, making utilities public. People actively choose to believe misinformation because it suits their own personal tribal narratives.
Just from reading the abstract, it feels like the authors didn't even attempt at trying to be objective. It hard to take what they're saying seriously when the language is so loaded and full of judgments. The kind of language you'd expect in an Op-Ed and not a research paper
I think you may be confused.
This is not a research paper, it's an op-ed in a law journal.
SSRN is where most draft law review/journal articles are published, which may be the source of confusion.
For most other fields, it is a source of draft/published science papers, but for law, it's pretty much any kind of article that is going to show up in a law review/journal.
This is nothing but speculation written by lawyers in the format of a scientific paper to feign legitimacy. Of course those $500 an hour nitpickers are terrified of AI because it threatens the exorbitant income of their cartel protected profession.
Care to actually engage with the text instead of deciding to paint the entire profession with a crappy brush?
I guess i'll start with calling two well known law professors "$500 an hour nitpickers" when they don't earn 500 an hour and have been professors for 15+ years (20+ in Jessica's case), so aren't earning anything close to 500 an hour, is not a great start?
I don't know if they are nitpickers, i've never taken their classes :)
Also, this is an op-ed, not a science paper.
Which you'd know if you had bothered to read it at all.
You say elsewhere you didn't bother to read anything other than the abstract, because "you didn't need to", so besides being a totally uninformed opinion, complaining about something else being speculation when you are literally speculating on the contents of the paper is pretty ironic.
I also find it amazingly humorous given that Jessica's previous papers on IP has been celebrated by HN, in part because she roughly believes copyright/patents as they currently exist are all glorified BS that doesn't help anything, and has written many papers as to why :)
1. It is entirely based on speculation of what is going to happen in the future.
2. The authors have a clear financial (and status based) interest in the outcome.
3. I have a negative opinion of lawyers and universities due to personal experience. (This is, of course, the weakest point by far.)
Speculation on future outcomes is not by itself a bad thing, but when that speculation is formatted like a scientific paper describing an experimental result I immediately feel I am being manipulated by appeal to authority. And the conflict of interest of the authors is about as irrelevant as pointing out that a paper on why Oxycodone is not addictive is paid for by Perdue Pharma. Perhaps Jessica's papers on IP are respected because they do not suffer from these obvious flaws? I owe the author no deference for the quality of her previous writing nor for her status as a professor.
Please go to court using only ChatGPT as legal defense, I'd love to see it, it's going to make for great entertainment. The judge a little bit less so.
You can criticise the hourly cost of lawyers all you like, and it should be a beautiful demonstration to people like you that no, "high costs means more people go into the profession and lower the costs" is not and has never been a reality. But to think that any AI could ever be efficient in a system such common law, the most batshit insane, inefficient, "rethoric matters more than logic" system is delusional.
Enough people have gotten owned for using these things in court that I think the more likely response is laughing at the ignorance then feeling threatened.
1. Get owned in court because you used an LLM that made a poor legal argument.
2. Get owned out of court because you couldn't afford the $100K (minimum) that you have to pay to the lawyer's cartel to even be able to make your argument in front of a judge.
I'll take number 1. At least you have a fighting chance. And it's only going to get better. LLMs today are the worst they will ever be, whereas the lawyer's cartel rarely gets better and never cuts its prices.
It's written in the future tense, so I can safely call it speculation. I've read the abstract which is all I need to decide the full text is not worth my time.
Cool, then we can safely give your comments exactly the same treatment - since they are completely uninformed speculation about a paper you haven't read.
Is he incorrect that the paper is speculating about future events? I don't think it's completely uninformed either. He said that he's read the abstract, which is supposed to give you an impression of the structure of the argument. Why don't you engage with the criticism?
And you must have read all 40 pages of it, right? Because if not you are a hypocrite. I claim that the Bible is the literal truth. Oh, you haven't read every word of the Bible? Your arguments against me are worthless!
> Purpose-driven institutions built around transparency, cooperation, and accountability empower individuals to take intellectual risks and challenge the status quo.
I am not sure if I am off-topic, but I am having a lot of trouble with this statement. Institutions are often opaque, and I have never belonged to an institution that empowered me to "take intellectual risks and challenge the status quo." Quite the contrary.
None of these paper's arguments are AI specific. The IRS doesn't need AI to make mistakes and be unable to tell you why it did so. You can find stories of that happening to people already.
i think when most people bring up mistakes that these models make, much of their concern is that little can be done.
when one of the juniors makes a mistake, i can talk to them about it and help them understand where they went wrong, if they continue to make mistakes we can change their position to something more suited for them. we can always let them go if they have too much hubris to learn.
who do we hold to account when a model makes a mistake? we’re already beginning to see, after major fuckups, companies blackhole nullrouting accountability into “not our fault, don’t look at us, ai was wrong”
the other thing is, if you have done a good job selecting your team, you’ll have people who understand their limits, who understand when to ask for help, who understand when they don’t know something. a major problem with current models is that it will always just guess or stretch toward random rather than halt.
so yes, people will make mistakes, but at least you can count on being able to mitigate for those after.
>Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life. They are the mechanisms through which complex societies encourage cooperation and stability, while also adapting to changing circumstances.
Who do institutions serve? To me AI democratises information. Allows access to information that would normally be gatekept. AI reduces barriers, and they don't like that because those barriers gave them authority.
I fear the title of this article is going drive most of the conversation.
I haven’t read through the whole thing yet, but so far the parts of the argument I can pull out are about how Institutions actually work, as in a collection of humans. AI, as it currently stands, interacts with humans themselves in ways that hollow out the kind of behavior we want from institutions.
“ Perhaps if human nature were a little less vulnerable to the siren’s call of
shortcuts, then AI could achieve the potential its creators envisioned for it. But that is not the world we live in. Short-term political and financial incentives amplify the worst aspects of AI systems, including domination of human will, abrogation of accountability, delegation of responsibility, and obfuscation of knowledge and control”
An analogy that I find increasingly useful is that of someone using a forklift to lift weights at gym. There is an observable tendency when using LLMs, to cede agency entirely to the machine.
You can intentionally market the use cases without knowing exactly how they work, though. So it's intentional investment and use case targeting, rather than directly designing for purpose. Though, the market also drives the measures...so they iteratively get better at things you pour money into.
If you just met a dog for the first time, you can't :) - my guess is LLMs are somewhere in between. It would be cool to see what happens if somebody tried to make an LLM that somehow has ethical principles (instead of guardrails) and is much less eager to please.
The institutions have been doing a fine job of destroying all their credibility and utility all on their own for far longer than this new AI hype cycle.
ZIRP, Covid, Anti-nuclear power, immigration crisis across the west, debt enslavement of future generations to buy votes, socializing losses and privatizing gains... Nancy is a better investor than Warren.
I am not defending billionaires, the vast majority of them are grifting scum. But to put this at their feet is not the right level of analysis when the institutions themselves are actively working to undermine the populace for the benefit of those that are supposed to be stewards of said institutions.
This dire warning against AI echoes the anxieties of a much earlier elite: the late-medieval clergy facing the invention of the printing press. For centuries, they held a privileged monopoly on knowledge, controlling its interpretation and dissemination. The printing press threatened to shatter that authority by democratizing access to information and empowering individuals.
Similarly, today's critics, often from within the very institutions they defend, frame AI as a threat to "expertise" and "civic life" when in reality, they fear it as a threat to their own status as the sole arbiters of truth. Their resistance is less a principled defense of democracy and more a desperate attempt to protect a crumbling monopoly on knowledge.
> a desperate attempt to protect a crumbling monopoly on knowledge
More like a war on the traditional, human-based knowledge, leveraged by people who believe that via coveting the world's supply of RAM, SSDs, GPUs, and what not, can achieve their own monopoly on knowledge under the pretense of liberating it. Note that running your own LLM becomes impossible if you can no longer afford the hardware to run it on.
Come now. You mean the highly regulated, more competitive world of law? That too, as it is practiced in America? The once capital of economic competition?
That “cartel”?
Vs the leaders of an industry that built their tools through insane amounts of copyright infringement, and have forced the coining of “enshittification” to describe all pervasive business strategies?
The same industry which employs acqui-hire to find ways to cull competition?
Surely we'll all beat monopolies by running our own local LLMs, storing whole blockchains on our local storage, building our own atomic power plants, flying our own airlines and launching our own satellites via our own rocket fleets. And producing our own trillion-transistor silicon in our own fabs.
We just have to start printing our own money and buying us some pocket armies and puppet politicians first.
The current bubble's effect on hardware is alarming but if they think they are going to create a permanent economic manipulation they are deluded. The US' hold on controls is eroding at a faster rate and China will be making good enough all the faster if its price/spec ratio is absurdly high.
Crypto currency makers can have artificial limits but no amount of limiting gpt-next access is cutting access to good enough.
If what you say was true, why are people from not within those institutions also try to warn others about the potential downfall of "expertise" and "civic life"? Are they just misinformed? Paid by these "institutional defenders" or what is your hypothesis?
In most cases those people are members of the upper class who hold credentials issued by those institutions, and often are in professions protected by state enforced cartels where the ticket for entry is one of said credentials.
> In most cases those people are members of the upper class who hold credentials issued by those institutions
Right, but in my comment I'm explicitly asking about the ones that don't have any relation yet seem to defend it anyways? "Don't people don't actually exists" isn't really an argument...
So you're saying codemonkeys are mad they don't get seen as the 'cool guys', we have to kill the jobs 'cool guys' have. The codemonkeys will never be cool, just accept it, there's no way to fix it. These cool guys will for the most part be 'cool' even if you take away their jobs right now.
The alarm isn't coming from outside the institutions; it's coming from a wider, more modern clergy. The new priestly class isn't defined by a specific building, but by a shared claim to the mastery of complex symbolic knowledge.
The linguists who call AI a "stochastic parrot" are the perfect example. Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory. They are watching their entire intellectual paradigm—their very claim to authority—be rendered obsolete.
This isn't a grassroots movement. It's an immune response from the cognitive elite, desperately trying to delegitimize a technology that threatens to trivialize their expertise. They aren't defending society; they're defending their status.
> Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory.
It's some wild claim. Every linguist worth their salt had known that you don't need grammatical theory to reach native level. Grammar being descriptive rather than prescriptive is the mainstream idea and had been long before LLM.
If you actually ask them, I bet most linguists will say they are not even excellent English (or whichever language they studied the most) teachers.
Plus, "stochastic parrot" was coined before ChatGPT. If linguists really felt that threatened by the time when people's concerns over AI was like "sure it can beat go master but how about league of legends?" you have to admit they did have some special insights, right?
You've mistaken the battlefield. This isn't about descriptive grammar. It's about the decades-long dominance of Chomsky's entire philosophy of language.
His central argument has always been that language is too complex and nuanced to be learned simply from exposure. Therefore, he concluded, humans must possess an innate, pre-wired "language organ"—a Universal Grammar.
LLMs are a spectacular demolition of that premise. They prove that with a vast enough dataset, complex linguistic structure can be mastered through statistical pattern recognition alone.
The panic from Chomsky and his acolytes isn't that of a humble linguist. It is the fury of a high priest watching a machine commit the ultimate heresy: achieving linguistic mastery without needing his innate, god-given grammar.
> LLMs are a spectacular demolition of that premise.
It really isn't. While I personally think the Universal Grammar theory is flawed (or at least Chomsky's presentation is flawed), LLM doesn't debunk it.
Right now we have machines that recognized faces better than humans. But it doesn't mean humans do not have some innate biological "hardware" for facial recognition that machines don't possess. The machines simply outperform the biological hardware with their own different approach.
Also, I highly recommend you express your ideas with your own words instead of letting an LLM present them. It's painfully obvious.
I do not see how it can be claimed that "LLMs are a spectacular demolition of that premise", because LLMs must be trained on an amount of text far greater than that to what a human is exposed.
I have learned one foreign language just by being exposed to it almost daily, by watching movies spoken in that language, without using any additional means, like a dictionary or a grammar (because none were available where I lived; this was before the Internet). However, I have been helped in guessing the meaning of the words and the grammar of the language, not only by seeing what the characters of the movie were doing, correlated to the spoken phrases, but also by the fact that I knew a couple of languages that had many similarities with the language of the movies that I was watching.
In any case, the amount of the spoken language to which I had been exposed for a year or so, until becoming fluent in it, had been many orders of magnitudes less than what is used by a LLM for training.
I do not know whether any innate knowledge of some grammar was involved, but certainly the knowledge of the grammar of other languages had helped tremendously in reducing the need for being exposed to greater amounts of text, because after seeing only a few examples I could guess the generally-applicable grammar rules.
There is no doubt that the way by which a LLM learns is much dumber than how a human learns, which is why this must be compensated by a much bigger amount of training data.
Seeing how the current inefficiency of LLM training has already caused serious problems for a great number of people, who either had to give up on buying various kinds of electronic devices or they had to accept to buy devices of a much worse quality than previously desired and planned, because the prices for DRAM modules and for big SSDs have skyrocketed, due to the hoarding of memory devices by the rich who hope to become richer by using LLMs, I believe that it has been proven beyond doubt that the way how LLMs learn, for now, is not good enough and it is certainly not a positive achievement, as more people have been hurt by it than the people who have benefited from it.
The first weakness of your claim is that it is inherently one of the elite.
You read the works of the cognitive elite, when they support AI. When most people sing its praises, it’s from the highest echelons of white collar work priesthood.
AI is fundamentally a tool of the cognitively trained, and shows its greatest capability in the hands of those capable of assessing its output as accurate at a glance. The more complex the realm, the deeper the expertise to find value in it.
Secondly, linguists are not the sole group espousing the concerns with these tools. I’ve seen rando streamers and normal folk in WhatsApp groups, completely disconnected from the AI elite hating what is being wrought. Students and young adults outright wonder if they will have any worthwhile economic future.
Perhaps it is not a “movement”, but there is an all pervasive fear and concern in the population when it comes to AI.
Finally, position is eerily similar to the dismissal of concerns from mid level and factory floor job workers in the 80s and 90s. It was forgivable given the then prevalent belief that people would be retrained and reabsorbed into equivalently sustaining roles in other new industries.
Not all of them, but given the same questionable or outright false assumptions (e.g. AI companies are doing interference at a loss, the exaggerated water consumption number, etc) keeping getting repeated on YouTube, Reddit and even HN where the user base is far more tech-savvy than the population, I think misinformation is the primary reason.
It was the same clergy (or rather parts of it) that used the printing press to great success.
Martin Luther used it to spread his influence extremely quickly for example. Similarly, the clergy used new innovations in book layout and writing to spread Christianity across Europe a thousand years before that.
What is weird about LLMs though, is that it isn't a simple catalyst of human labor. The printing press or the internet can be used to spread information quickly that you have previously compiled or created. These technologies both have a democratizing effect and have objectively created new opportunities.
But LLMs are to some degree parasitical to human labor. I feel like their centralizing effect is stronger than their democratizing one.
Everyone who tells the story of the reformation leaves out that Martin Luther also used this new technology to widely disseminate his deranged anti-Semitic lies and conspiracies, leading to pogroms against Jews, a hundred years of war across Europe, and providing the ideological basis for the rise of Nazism.
I think this could be applied to most fields where LLMs move in. Let's take the field we are probably most familiar with.
Currently companies start to shift from enhancing productivity of their employees with giving them access to LLMs, they start to offshore to lower cost countries and give the cheap labor LLMs to bypass language and quality barriers. The position isn't lost, it's just moving somewhere else.
In the field of software development this won't be a an anxiety of an elite or threat to expertise or status, but rather a direct consequence to livelihood when people won't be hired and lose access to the economy until they retrain for a different field. So a layer on top of that you can argue with authority and control, but it rather has economic factors to it that produce the anxiety.
In that sense, doesn't any knowledge work have a monopoly on knowledge? It is the entire point to have experts in fields that know the details and have the experience, so that things can be done as expected, since not many have the time nor the capabilities to get into the critical details.
If you believe there is any good will when you can centralize that knowledge to the hands of even less people, you produce the same pattern you are complaining about, especially when it comes to how businesses are tweaking their margins. It really is a force multiplier and equalizer, but a tool, that can be used in good ways or bad ways depending on how you look at it.
It is funny watching people debate at length with your LLM word-vomit. I'm not sure whether you yourself are convinced that the soup you've copypasted across multiple replies means anything, but apparently some people are convinced enough to argue with it, so this is pretty great satire in one way or another.
it feels good to watch the aforementioned clergy kvetch about AI while multiple multi-trillion dollar corporations backed by a friendly administration continue to run their bulldozers :)
The printing press was also used to print witch hunting books and caused 200 years of mass hysteria around witches and witch trials.
Before the printing press, only the clergy could "identity" witches but the printing press "democratized knowledge" of witch identification at larger scale.
The algorithmic version of "It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so" is going to cause huge trouble in the short and medium term.
this is much much closer to going in reverse back to when the church were the deciders rather than liberating knowledge the way the printing press did.
the church did the thinking for the peasants. the church decided what the peasants heard, etc… this is moving absolutely in that direction.
the models now do the thinking for us, the ai companies decide what we get to see, these companies decide how much we pay to access it. this is the future.
An institution is worth nothing without the spirit, humanity and exchange of knowledge among the humanity behind it. The fostering of real expertise is difficult, but without this expertise you are doomed to believe whatever your Corporate AI is telling you.
It's so ridiculous to make this argument when the people who stand to benefit the most from this technology are the massive corporations that can subsidize the compute and capital costs of this technology. Is it democratization when Google pulls something your wrote on your website then runs it through an LLM so they can serve it directly to a user? You say people see this as a threat to their status but the reality is this is a massive consolidation of the information economy of the internet in the hands of a few corporate interests.
This is a good analogy, but you made it backwards. The "Clergy" fears the "Printing Press", as it acts as a tool of decentralized information spreading. But LLMs are not decentralized and thus are not the "Printing Press". LLMs are what the "Clergy" (say, for example, all the AI companies led by billionaires in cahoots with the west's most powerful government) uses to suppress the real "Printing Press" (the decentralized, open internet, where everybody can host and be reached).
Is that what happened? In Nexus, Harari looks at this exact same situation: the invention of the printing press, and shows how clergy used it to stoke witch hunts (ahem, misinformation) for decades--if not centuries. It was not for hundreds of years until after the invention of the printing press that we had The Enlightenment. What gave rise to The Enlightenment? Harari argues it is modern institutions.
It's not so simple that we can say "printing press good, nobody speak ill of the printing press."
I would summarize the central claim of the paper as: the widespread use of AI to mediate human interaction will rob people of agency, understanding and skill development, as well as destroying the social the social links necessary to maintain and improve institutions, while at the same time allowing powerful unaccountable actors (AI cabal) to interject into those relations and impose their institutional goals; by "institution" we mean a shared set of beneficial social rules, not merely an organization tasked with promoting them, say "justice" vs. "US justice system".
The authors then break down the mechanisms by which AI achieves these outcome (that seem quite reductive and dated compared to the frontier, for example they take it as granted that AI cannot be creative, that it can only work prospectively and can't react to new situations and events etc.), as well as exemplifying those mechanism in a few areas like journalism, academia etc.
We should be more worried what AI will due to the ability of an average human to think.
Not that I think there is a lot of thinking going on now anyway, thanks to our beloved smartphones.
But just think about a time when human ability to reason has atrophied globally. AI might even give us true Idiocracy!
[delayed]
> We should be more worried what AI will due to the ability of an average human to think.
I had a wake up call on this yesterday. After a recent HN thread about Zed editor, I decided to give it another try, so I loaded it up, disabled AI, and tried writing some code from scratch. No AI completion, no intellisence. Two things came to mind. First, my editor seems so much more peaceful without being told what to do. Second, it was a bit scary how lost I felt. It was obvious that my own ability to communicate through code had declined a bit since I began using AI coding assistants. It turns out that as expected, coding assistants really are competitive cognitive artifacts. After that experience, I've decided that I am going to do at least part of my coding with all completions turned off. Unfortunately at work you are paid to produce quickly, so I think my AI free editor will have to be reserved for personal projects.
Further related to your statement about thought, the hallucinations persist, and even last night I got a response about 80's pop culture that was over 50% bullshit. Just imagine what intentional persuasion through LLM models will do to society. Independent thought has never been more important.
Similar rhetoric was allways there with new technologies. Calculators, radio, cameras, phones, computers, smartphones, social networks...
Regulations do more harm than learning process from mistakes.
Yes we have this before and everytime it was correct.
I mean we were seeing this even before AI. It's the same type of person. To slop is human.
It's like for some reason we thought that like some good percentage of us aren't just tribal worker drones who fundamentally just want fats, sugars, salts, dopamine and seratonin. People actively vote against things like UBI, higher corporate taxes, making utilities public. People actively choose to believe misinformation because it suits their own personal tribal narratives.
Just from reading the abstract, it feels like the authors didn't even attempt at trying to be objective. It hard to take what they're saying seriously when the language is so loaded and full of judgments. The kind of language you'd expect in an Op-Ed and not a research paper
I think you may be confused. This is not a research paper, it's an op-ed in a law journal.
SSRN is where most draft law review/journal articles are published, which may be the source of confusion.
For most other fields, it is a source of draft/published science papers, but for law, it's pretty much any kind of article that is going to show up in a law review/journal.
It is literally called “ Boston Univ. School of Law Research Paper No. 5870623”
It's an essay. Being opinionated is a feature.
This is nothing but speculation written by lawyers in the format of a scientific paper to feign legitimacy. Of course those $500 an hour nitpickers are terrified of AI because it threatens the exorbitant income of their cartel protected profession.
Tech workers know it all, no way a non-tech job could be worth anything more than 20 dollars an hour.
Care to actually engage with the text instead of deciding to paint the entire profession with a crappy brush?
I guess i'll start with calling two well known law professors "$500 an hour nitpickers" when they don't earn 500 an hour and have been professors for 15+ years (20+ in Jessica's case), so aren't earning anything close to 500 an hour, is not a great start?
I don't know if they are nitpickers, i've never taken their classes :)
Also, this is an op-ed, not a science paper. Which you'd know if you had bothered to read it at all.
You say elsewhere you didn't bother to read anything other than the abstract, because "you didn't need to", so besides being a totally uninformed opinion, complaining about something else being speculation when you are literally speculating on the contents of the paper is pretty ironic.
I also find it amazingly humorous given that Jessica's previous papers on IP has been celebrated by HN, in part because she roughly believes copyright/patents as they currently exist are all glorified BS that doesn't help anything, and has written many papers as to why :)
I dismiss the paper for 3 reasons:
1. It is entirely based on speculation of what is going to happen in the future.
2. The authors have a clear financial (and status based) interest in the outcome.
3. I have a negative opinion of lawyers and universities due to personal experience. (This is, of course, the weakest point by far.)
Speculation on future outcomes is not by itself a bad thing, but when that speculation is formatted like a scientific paper describing an experimental result I immediately feel I am being manipulated by appeal to authority. And the conflict of interest of the authors is about as irrelevant as pointing out that a paper on why Oxycodone is not addictive is paid for by Perdue Pharma. Perhaps Jessica's papers on IP are respected because they do not suffer from these obvious flaws? I owe the author no deference for the quality of her previous writing nor for her status as a professor.
3. Same, including press that is no longer unbiased and serve as propaganda of political opinions.
One might say that deinstitutionalization is actually good for plurality of opinions (some call it a democracy). If AI cause it, I'm fine with that.
Please go to court using only ChatGPT as legal defense, I'd love to see it, it's going to make for great entertainment. The judge a little bit less so.
You can criticise the hourly cost of lawyers all you like, and it should be a beautiful demonstration to people like you that no, "high costs means more people go into the profession and lower the costs" is not and has never been a reality. But to think that any AI could ever be efficient in a system such common law, the most batshit insane, inefficient, "rethoric matters more than logic" system is delusional.
Yeah, unfortunately, it's the lawyers that are using ChatGPT.
Enough people have gotten owned for using these things in court that I think the more likely response is laughing at the ignorance then feeling threatened.
1. Get owned in court because you used an LLM that made a poor legal argument.
2. Get owned out of court because you couldn't afford the $100K (minimum) that you have to pay to the lawyer's cartel to even be able to make your argument in front of a judge.
I'll take number 1. At least you have a fighting chance. And it's only going to get better. LLMs today are the worst they will ever be, whereas the lawyer's cartel rarely gets better and never cuts its prices.
Does it cost 100k minimum in the US to get a lawyer? Or am I misunderstanding something?
> This is nothing but speculation
Did you read the paper?
It's written in the future tense, so I can safely call it speculation. I've read the abstract which is all I need to decide the full text is not worth my time.
Cool, then we can safely give your comments exactly the same treatment - since they are completely uninformed speculation about a paper you haven't read.
Is he incorrect that the paper is speculating about future events? I don't think it's completely uninformed either. He said that he's read the abstract, which is supposed to give you an impression of the structure of the argument. Why don't you engage with the criticism?
And you must have read all 40 pages of it, right? Because if not you are a hypocrite. I claim that the Bible is the literal truth. Oh, you haven't read every word of the Bible? Your arguments against me are worthless!
> Purpose-driven institutions built around transparency, cooperation, and accountability empower individuals to take intellectual risks and challenge the status quo.
I am not sure if I am off-topic, but I am having a lot of trouble with this statement. Institutions are often opaque, and I have never belonged to an institution that empowered me to "take intellectual risks and challenge the status quo." Quite the contrary.
"purpose-driven" is the relevant qualifier here.
None of these paper's arguments are AI specific. The IRS doesn't need AI to make mistakes and be unable to tell you why it did so. You can find stories of that happening to people already.
i think when most people bring up mistakes that these models make, much of their concern is that little can be done.
when one of the juniors makes a mistake, i can talk to them about it and help them understand where they went wrong, if they continue to make mistakes we can change their position to something more suited for them. we can always let them go if they have too much hubris to learn.
who do we hold to account when a model makes a mistake? we’re already beginning to see, after major fuckups, companies blackhole nullrouting accountability into “not our fault, don’t look at us, ai was wrong”
the other thing is, if you have done a good job selecting your team, you’ll have people who understand their limits, who understand when to ask for help, who understand when they don’t know something. a major problem with current models is that it will always just guess or stretch toward random rather than halt.
so yes, people will make mistakes, but at least you can count on being able to mitigate for those after.
>Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life. They are the mechanisms through which complex societies encourage cooperation and stability, while also adapting to changing circumstances.
https://www.pewresearch.org/politics/2025/12/04/public-trust...
In the 1960s, trust in institutions was around 70%.
In 2025 it's about 17%.
This is not something like we had 70% until 2023 and then AI dropped our trust suddenly. If anything, AI doesnt even register on the graph.
So correlation here is practically non-existent. Gallup and Pew have the similar trends for journalists and universities.
You dont get to blame AI for this.
Interesting bump and question. How did clinton improve reputation and then bush destroy it? Or is that a false hump?
Who do institutions serve? To me AI democratises information. Allows access to information that would normally be gatekept. AI reduces barriers, and they don't like that because those barriers gave them authority.
Dupe of https://news.ycombinator.com/item?id=46622870
Not really a dupe if no one really discussed it. And I'm glad, didn't see it that previous time :)
I was amused at how they quote War Games.
I fear the title of this article is going drive most of the conversation.
I haven’t read through the whole thing yet, but so far the parts of the argument I can pull out are about how Institutions actually work, as in a collection of humans. AI, as it currently stands, interacts with humans themselves in ways that hollow out the kind of behavior we want from institutions.
“ Perhaps if human nature were a little less vulnerable to the siren’s call of shortcuts, then AI could achieve the potential its creators envisioned for it. But that is not the world we live in. Short-term political and financial incentives amplify the worst aspects of AI systems, including domination of human will, abrogation of accountability, delegation of responsibility, and obfuscation of knowledge and control”
An analogy that I find increasingly useful is that of someone using a forklift to lift weights at gym. There is an observable tendency when using LLMs, to cede agency entirely to the machine.
> The affordances of AI systems have the effect of eroding expertise, short-circuiting decision-making, and isolating people from each other.
This affordability is HEAVILY subsidized by billionaires who want to destroy institutions for selfish and ideological reasons.
I think you have misread the word “affordances”. It’s not about affordability [0]. The main text also explains what it means.
[0] https://en.wikipedia.org/wiki/Affordance
This is literally corporate textbook 101. Subsidize your product, become market leader, cause lock-in and make your customers dependant.
Every large enough corporate wants to become the new Oracle.
Given how nobody properly understands LLMs, I doubt that they are intentionally designed like that. But the effect... yeah. I can see that happening.
(By the way, are you confusing affordance, the UX concept, with affordability?)
You can intentionally market the use cases without knowing exactly how they work, though. So it's intentional investment and use case targeting, rather than directly designing for purpose. Though, the market also drives the measures...so they iteratively get better at things you pour money into.
Nobody properly understands dog brains either and yet you can still train a dog to sit.
If you just met a dog for the first time, you can't :) - my guess is LLMs are somewhere in between. It would be cool to see what happens if somebody tried to make an LLM that somehow has ethical principles (instead of guardrails) and is much less eager to please.
The institutions have been doing a fine job of destroying all their credibility and utility all on their own for far longer than this new AI hype cycle.
ZIRP, Covid, Anti-nuclear power, immigration crisis across the west, debt enslavement of future generations to buy votes, socializing losses and privatizing gains... Nancy is a better investor than Warren.
I am not defending billionaires, the vast majority of them are grifting scum. But to put this at their feet is not the right level of analysis when the institutions themselves are actively working to undermine the populace for the benefit of those that are supposed to be stewards of said institutions.
> free press
Stopped reading here, as these people still believe in that fairytale of theirs.
The American press isn't perfectly free, but you should see what a state-controlled press is like.
A thought provoking essay on impact of AI systems civic institutions.
This dire warning against AI echoes the anxieties of a much earlier elite: the late-medieval clergy facing the invention of the printing press. For centuries, they held a privileged monopoly on knowledge, controlling its interpretation and dissemination. The printing press threatened to shatter that authority by democratizing access to information and empowering individuals.
Similarly, today's critics, often from within the very institutions they defend, frame AI as a threat to "expertise" and "civic life" when in reality, they fear it as a threat to their own status as the sole arbiters of truth. Their resistance is less a principled defense of democracy and more a desperate attempt to protect a crumbling monopoly on knowledge.
Isn't that just an ad-hominem against the writers? A threat to the status quo is still a threat to people and could have negative consequences.
> a desperate attempt to protect a crumbling monopoly on knowledge
More like a war on the traditional, human-based knowledge, leveraged by people who believe that via coveting the world's supply of RAM, SSDs, GPUs, and what not, can achieve their own monopoly on knowledge under the pretense of liberating it. Note that running your own LLM becomes impossible if you can no longer afford the hardware to run it on.
Better that I'm forced to rent an LLM from a tech monopolist for a few dollars than be forced to hire a member of the lawyers cartel for $500 an hour.
Come now. You mean the highly regulated, more competitive world of law? That too, as it is practiced in America? The once capital of economic competition?
That “cartel”?
Vs the leaders of an industry that built their tools through insane amounts of copyright infringement, and have forced the coining of “enshittification” to describe all pervasive business strategies?
The same industry which employs acqui-hire to find ways to cull competition?
Surely we'll all beat monopolies by running our own local LLMs, storing whole blockchains on our local storage, building our own atomic power plants, flying our own airlines and launching our own satellites via our own rocket fleets. And producing our own trillion-transistor silicon in our own fabs.
We just have to start printing our own money and buying us some pocket armies and puppet politicians first.
The current bubble's effect on hardware is alarming but if they think they are going to create a permanent economic manipulation they are deluded. The US' hold on controls is eroding at a faster rate and China will be making good enough all the faster if its price/spec ratio is absurdly high.
Crypto currency makers can have artificial limits but no amount of limiting gpt-next access is cutting access to good enough.
If what you say was true, why are people from not within those institutions also try to warn others about the potential downfall of "expertise" and "civic life"? Are they just misinformed? Paid by these "institutional defenders" or what is your hypothesis?
In most cases those people are members of the upper class who hold credentials issued by those institutions, and often are in professions protected by state enforced cartels where the ticket for entry is one of said credentials.
> In most cases those people are members of the upper class who hold credentials issued by those institutions
Right, but in my comment I'm explicitly asking about the ones that don't have any relation yet seem to defend it anyways? "Don't people don't actually exists" isn't really an argument...
So you're saying codemonkeys are mad they don't get seen as the 'cool guys', we have to kill the jobs 'cool guys' have. The codemonkeys will never be cool, just accept it, there's no way to fix it. These cool guys will for the most part be 'cool' even if you take away their jobs right now.
The alarm isn't coming from outside the institutions; it's coming from a wider, more modern clergy. The new priestly class isn't defined by a specific building, but by a shared claim to the mastery of complex symbolic knowledge.
The linguists who call AI a "stochastic parrot" are the perfect example. Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory. They are watching their entire intellectual paradigm—their very claim to authority—be rendered obsolete.
This isn't a grassroots movement. It's an immune response from the cognitive elite, desperately trying to delegitimize a technology that threatens to trivialize their expertise. They aren't defending society; they're defending their status.
> Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory.
It's some wild claim. Every linguist worth their salt had known that you don't need grammatical theory to reach native level. Grammar being descriptive rather than prescriptive is the mainstream idea and had been long before LLM.
If you actually ask them, I bet most linguists will say they are not even excellent English (or whichever language they studied the most) teachers.
Plus, "stochastic parrot" was coined before ChatGPT. If linguists really felt that threatened by the time when people's concerns over AI was like "sure it can beat go master but how about league of legends?" you have to admit they did have some special insights, right?
You've mistaken the battlefield. This isn't about descriptive grammar. It's about the decades-long dominance of Chomsky's entire philosophy of language.
His central argument has always been that language is too complex and nuanced to be learned simply from exposure. Therefore, he concluded, humans must possess an innate, pre-wired "language organ"—a Universal Grammar.
LLMs are a spectacular demolition of that premise. They prove that with a vast enough dataset, complex linguistic structure can be mastered through statistical pattern recognition alone.
The panic from Chomsky and his acolytes isn't that of a humble linguist. It is the fury of a high priest watching a machine commit the ultimate heresy: achieving linguistic mastery without needing his innate, god-given grammar.
> LLMs are a spectacular demolition of that premise.
It really isn't. While I personally think the Universal Grammar theory is flawed (or at least Chomsky's presentation is flawed), LLM doesn't debunk it.
Right now we have machines that recognized faces better than humans. But it doesn't mean humans do not have some innate biological "hardware" for facial recognition that machines don't possess. The machines simply outperform the biological hardware with their own different approach.
Also, I highly recommend you express your ideas with your own words instead of letting an LLM present them. It's painfully obvious.
I do not see how it can be claimed that "LLMs are a spectacular demolition of that premise", because LLMs must be trained on an amount of text far greater than that to what a human is exposed.
I have learned one foreign language just by being exposed to it almost daily, by watching movies spoken in that language, without using any additional means, like a dictionary or a grammar (because none were available where I lived; this was before the Internet). However, I have been helped in guessing the meaning of the words and the grammar of the language, not only by seeing what the characters of the movie were doing, correlated to the spoken phrases, but also by the fact that I knew a couple of languages that had many similarities with the language of the movies that I was watching.
In any case, the amount of the spoken language to which I had been exposed for a year or so, until becoming fluent in it, had been many orders of magnitudes less than what is used by a LLM for training.
I do not know whether any innate knowledge of some grammar was involved, but certainly the knowledge of the grammar of other languages had helped tremendously in reducing the need for being exposed to greater amounts of text, because after seeing only a few examples I could guess the generally-applicable grammar rules.
There is no doubt that the way by which a LLM learns is much dumber than how a human learns, which is why this must be compensated by a much bigger amount of training data.
Seeing how the current inefficiency of LLM training has already caused serious problems for a great number of people, who either had to give up on buying various kinds of electronic devices or they had to accept to buy devices of a much worse quality than previously desired and planned, because the prices for DRAM modules and for big SSDs have skyrocketed, due to the hoarding of memory devices by the rich who hope to become richer by using LLMs, I believe that it has been proven beyond doubt that the way how LLMs learn, for now, is not good enough and it is certainly not a positive achievement, as more people have been hurt by it than the people who have benefited from it.
The first weakness of your claim is that it is inherently one of the elite.
You read the works of the cognitive elite, when they support AI. When most people sing its praises, it’s from the highest echelons of white collar work priesthood.
AI is fundamentally a tool of the cognitively trained, and shows its greatest capability in the hands of those capable of assessing its output as accurate at a glance. The more complex the realm, the deeper the expertise to find value in it.
Secondly, linguists are not the sole group espousing the concerns with these tools. I’ve seen rando streamers and normal folk in WhatsApp groups, completely disconnected from the AI elite hating what is being wrought. Students and young adults outright wonder if they will have any worthwhile economic future.
Perhaps it is not a “movement”, but there is an all pervasive fear and concern in the population when it comes to AI.
Finally, position is eerily similar to the dismissal of concerns from mid level and factory floor job workers in the 80s and 90s. It was forgivable given the then prevalent belief that people would be retrained and reabsorbed into equivalently sustaining roles in other new industries.
> Are they just misinformed?
Not all of them, but given the same questionable or outright false assumptions (e.g. AI companies are doing interference at a loss, the exaggerated water consumption number, etc) keeping getting repeated on YouTube, Reddit and even HN where the user base is far more tech-savvy than the population, I think misinformation is the primary reason.
It was the same clergy (or rather parts of it) that used the printing press to great success.
Martin Luther used it to spread his influence extremely quickly for example. Similarly, the clergy used new innovations in book layout and writing to spread Christianity across Europe a thousand years before that.
What is weird about LLMs though, is that it isn't a simple catalyst of human labor. The printing press or the internet can be used to spread information quickly that you have previously compiled or created. These technologies both have a democratizing effect and have objectively created new opportunities.
But LLMs are to some degree parasitical to human labor. I feel like their centralizing effect is stronger than their democratizing one.
Martin Luther was clergy, but he was absolutely not "the same clergy."
Everyone who tells the story of the reformation leaves out that Martin Luther also used this new technology to widely disseminate his deranged anti-Semitic lies and conspiracies, leading to pogroms against Jews, a hundred years of war across Europe, and providing the ideological basis for the rise of Nazism.
I think this could be applied to most fields where LLMs move in. Let's take the field we are probably most familiar with.
Currently companies start to shift from enhancing productivity of their employees with giving them access to LLMs, they start to offshore to lower cost countries and give the cheap labor LLMs to bypass language and quality barriers. The position isn't lost, it's just moving somewhere else.
In the field of software development this won't be a an anxiety of an elite or threat to expertise or status, but rather a direct consequence to livelihood when people won't be hired and lose access to the economy until they retrain for a different field. So a layer on top of that you can argue with authority and control, but it rather has economic factors to it that produce the anxiety.
In that sense, doesn't any knowledge work have a monopoly on knowledge? It is the entire point to have experts in fields that know the details and have the experience, so that things can be done as expected, since not many have the time nor the capabilities to get into the critical details.
If you believe there is any good will when you can centralize that knowledge to the hands of even less people, you produce the same pattern you are complaining about, especially when it comes to how businesses are tweaking their margins. It really is a force multiplier and equalizer, but a tool, that can be used in good ways or bad ways depending on how you look at it.
It is funny watching people debate at length with your LLM word-vomit. I'm not sure whether you yourself are convinced that the soup you've copypasted across multiple replies means anything, but apparently some people are convinced enough to argue with it, so this is pretty great satire in one way or another.
it feels good to watch the aforementioned clergy kvetch about AI while multiple multi-trillion dollar corporations backed by a friendly administration continue to run their bulldozers :)
This is a criticism of the author's backgrounds rather than the content of the article.
True. I myself try to read articles without looking up the authors.
It is hard though. When someone makes an extraordinary claim I feel the urge to look them up. It is a shortcut to some legitimacy to that claim.
Most of the comments here are. HN hates lawyers.
The printing press was also used to print witch hunting books and caused 200 years of mass hysteria around witches and witch trials.
Before the printing press, only the clergy could "identity" witches but the printing press "democratized knowledge" of witch identification at larger scale.
The algorithmic version of "It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so" is going to cause huge trouble in the short and medium term.
this is much much closer to going in reverse back to when the church were the deciders rather than liberating knowledge the way the printing press did.
the church did the thinking for the peasants. the church decided what the peasants heard, etc… this is moving absolutely in that direction.
the models now do the thinking for us, the ai companies decide what we get to see, these companies decide how much we pay to access it. this is the future.
An institution is worth nothing without the spirit, humanity and exchange of knowledge among the humanity behind it. The fostering of real expertise is difficult, but without this expertise you are doomed to believe whatever your Corporate AI is telling you.
So is the AI better?
No. It's quicker, easier, more seductive.
It's so ridiculous to make this argument when the people who stand to benefit the most from this technology are the massive corporations that can subsidize the compute and capital costs of this technology. Is it democratization when Google pulls something your wrote on your website then runs it through an LLM so they can serve it directly to a user? You say people see this as a threat to their status but the reality is this is a massive consolidation of the information economy of the internet in the hands of a few corporate interests.
This is a good analogy, but you made it backwards. The "Clergy" fears the "Printing Press", as it acts as a tool of decentralized information spreading. But LLMs are not decentralized and thus are not the "Printing Press". LLMs are what the "Clergy" (say, for example, all the AI companies led by billionaires in cahoots with the west's most powerful government) uses to suppress the real "Printing Press" (the decentralized, open internet, where everybody can host and be reached).
Is that what happened? In Nexus, Harari looks at this exact same situation: the invention of the printing press, and shows how clergy used it to stoke witch hunts (ahem, misinformation) for decades--if not centuries. It was not for hundreds of years until after the invention of the printing press that we had The Enlightenment. What gave rise to The Enlightenment? Harari argues it is modern institutions.
It's not so simple that we can say "printing press good, nobody speak ill of the printing press."