>>SF wants to come for your inner life and pimp you out and mediate every interaction and there’s not even a so.
I have been saying this for a while. I visited SF a couple months ago and god do people feel empty from the inside. Everything is revolving around AI this and AI that. Half of these people were not paying attention when we were training gradient boost models and now all of these people are 'AI Agents experts'.
I think they are trying to express that everyone and their brother is now an expert in AI. But everyone was already a decorated expert with decades of hands on "AI" experience by the summer of 2023. So "experts in agentic AI" is just the continuation of that trend.
Even if Dario and Altman originally believed what they were saying, their scaretactics worked wonders for investment. Their companies are now incredibly incentivized to keep the AI Apocalypse narrative going further and further. It is hard to imagine them stopping, as that may lose them investment.
The last 10 years have been a decade of Big Tech Vaporware. NFTs and the Metaverse were assured to be the future. Once this narrative fails too (which I am almost certain is inevitable) I think society's love affair with Technology being the solver-of-all-things will finally fracture.
I think technology can actually solve a lot more things than it has, the problem is that foundational problem solving does not make a unicorn startup with earthshaking economies of scale in most cases, and especially so if there's a physical world piece involved, and even more so if it's going to go asymptotic.
Just to throw out a random example - could technology, in terms of advances and improvements to allocation, distribution, and consumption, play a part in solving the western US's water issues? Probably. But would it be something that make a trillion dollars and a household name? Maybe not. And could saner policy, like making farmers have to bid for acre feet instead of getting it for basically free + distribution cost also help? Sure.
Likewise, even with AI most software is still crap. Like when have you heard about a doctor who loves their EHR? Like never.
So I think technology could be a solver of at least a lot more things. But we've created a market where people want to exhaust every flavor of flim flam and trend of the moment first. Because we've glorified the business of tech more than the actual improvements and aspirations that should be possible.
It's significantly worse! All NFTs did was separate fools from their money, and crypto in general increased GPU costs. All the Metaverse did was cause Facebook to throw away a bunch of money (lol).
Meanwhile, AI has ruined the whole Internet and inflated the price of everything electronic.
The whole internet AND local computing. This may be cool for people on SF salaries but for the vast majority of the world memory has become a luxury good
While it is driving new hardware costs up remember you can run Linux on a potato and just wait it out. My pixel 8a is a phenomenal computer if ignore the hype monster.
If you look at Anthropic's blogs about their model timelines, there is roughly a ~3m period between a model being in internal preview until release. That means that inside of Anthropic, the next version beyond Mythos/Fable is already in preview, already being tested internally. Despite what Geohot is describing here about GLM, my understanding is that Anthropic employees have spent a significant amount of time grappling with a technology that is considerably ahead of what is available to the public today.
In addition, if you look at the graph of LOC written by Claude vs Ants (I.e. AI vs human), there is an incredibly sharp uptick post-Mythos internal preview. Something like from 30% to 75% of code inside the company being written by AI.
While I sympathize with the viewpoint here, I still have to admit that that there's a very different feeling to working inside of a company where they've had months of time with a model that's at the frontier, quickly changing the way everyone around them works, and that _they themselves_ control the keys to.
If Geohot had those keys, I can be 100% confident he'd be raising the alarm at the top of his lungs about it.
HN is not immune to this. I do not take HN seriously, because empirically so many takes across so many subjects have been wrong in a melodramatic fashion and the “adults”, often people with training and first hand experience have to show up to set the subject matter right before someone gets hurt. But as a barometer of unhinged hype? Only X is comparable.
I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export control issue, it's obviously not good for profits. If you assume AGI is possible, you must agree that it would be at the very least profoundly destabilizing. The companies are built around this assumption.
1. Those, like OP, that believe AI danger and disruption is all hype to boost valuations.
2. Those, like myself, that believe AI danger and disruption is very real and presents dilemmas (but also opportunities) for society, so we must tread carefully.
The first position is not logically consistent with reality. The danger is real: we have already seen hints about how this will impact everything from jobs to warfare to mental health. that danger does not increase valuation, in fact it does the opposite because of the need for government regulation.
I remember a meme that spread to that effect, but it was similarly divorced from reality. Hundreds of banks failed during the financial crisis, including multiple which were very large. There were a few cases like Wachovia, where an unhealthy bank was required to sell itself in order to avoid a technical failure that would have impacted over 10 million Americans' access to their money. But this was still unpleasant for existing Wachovia shareholders.
> Why would a company warn that their technology is dangerous if it wasn't?
Because it makes their technology seem more important and powerful?
Do you usually believe 100% what a company says about itself?
> Mythos export control issue, it's obviously not good for profits
Who cares about profits? None of these companies make money like a business currently. It's all speculative valuation and influenced by the same hype/doom cycle
What else is there? The product as delivered today doesn't come anywhere near a justification for these valuations. All of it is built on an expectation of future capabilities, and that's where the Dario-penned doom papers come in.
>Why would a company warn that their technology is dangerous if it wasn't?
Because Americans are obsessed with eschatological narratives and action hero stories. I talked to a Palantir guy once who told me that he loves it when journalists describe his company as a James Bond villain because every single time the market cap goes up.
If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring
Regulatory capture and false AGI hyping, what else? You'd have to be willingly ignoring the writings on the wall if you still haven't identified it yet.
Weird to think profits matter half as much as ever-increasing valuations, driven by memetic bullshit. E.g. the entire point of the article you're commenting on.
Anthropic's valuation is driven by its revenue growth. Export controls harm this. Hype certainly plays a role, but it's simply not the case that revenue is not a major part of the picture.
> Why would a company warn that their technology is dangerous if it wasn't?
It helps to look at the history of the AI Alarmist, EA-adjacent mindset. They believe AI could be Dangerous (<--- capital D) if not responsibly managed by the right people (spoiler: themselves or politicians who listen to them). The capital D means more than just some economic and employment disruption, they're thinking much bigger, like "existential threat to humanity" big.
The most extreme alarmists are basically LARPing Terminator 2 and want to shut it down before SkyNet takes over but the more moderate alarmists believe AI is too potentially beneficial walk away from. And they acknowledge other nations and various bad actors won't stop anyway, so they believe it's almost a sacred duty to move forward quickly albeit very responsibly. Yes, this leads to some fairly conflicted reasoning and priorities, like they need to keep gaining access to gobs of capital to ensure they are the ones to reach AGI first, as that's the only way to ensure "god" is benevolent.
While I'm sure there are execs who are primarily (or purely) mercenary, I also think some, if not many, other people involved sincerely believe (or at least hope) they are "the good guys" playing a crucial role in an era of historical importance. Obviously, this has some heavy vibes as well as a being quite a buff to one's self-importance. They face each new day feeling the weight of bearing such a consequential role in shaping the future of humanity. Us mere mortals can only pray their wisdom and altruistic ethics match their humility. </s>
So... yeah, that's why we see some weirdly whiplash messaging.
What's the evidence that the doom narrative is connected to valuations? It seems more like a marketing/recruiting strategy. As the post points out, institutional investors generally think the idea of mass unemployment is BS, and they are investing accordingly.
AI obviously has military applications, no doom required to think the military will want a lot of it. I'm asking about the extreme scenarios like "AI is gonna take everyone’s jobs" and the other "totalizing narratives" that geohot is referencing.
If this is true: "AI is gonna take everyone’s jobs" then AI is the terminal technology and everyone who doesn't want to be poor forever needs to throw all their money in it.
Companies say all sorts of things which investors don't take at face value, often for the sake of marketing and recruiting. Meta circa 2020 wanted us to believe we'd be swapping NFTs in a virtual mall and having all our meetings strapped into VR headsets by now. That doesn't mean their market cap at the time was based on an assumption they were right, in fact it is much higher now that the metaverse idea flopped. Unlike that situation where money was being spent on projects that turned out to be mostly pointless, AI labs don't have much of a financial stake in a scenario such as "lots of demand for AI, leading to mass unemployment and/or doomsday" as opposed to one like "lots of demand for AI, leading to the world being mostly the same as before but with more AI and expensive RAM".
Simple minds want to believe one simple thing and then rationalize everything else to force consistency with that one simple idea.
If you want to believe the simple idea that AI is mostly hype, then you'll get stuck in a multi-year loop talking about stochastic parrots, ridiculous valuations, and doomer scaredycats.
But the real world isn't so simple. Multiple seemingly contradictory things can be true at the same time.
Some AI is useless. Some is incredibly powerful and useful even though it makes mistakes. Some companies are wildly overvalued. Some extremely large and expensive companies will quadruple from here. Some frightening scenarios will look silly in hindsight. Other frightening things will happen that none of the doomers foresee.
It would be great to explore those new ideas and possibilities.
It's so boring rehashing the same old tired and worn out ideas like "they're just hyping the danger to pump their shares up."
The question is not whether AI is useful. It is useful, full stop.
The question is whether AI is useful enough to justify valuations that dwarf the GDP of all but the top 20 countries in the world. As it stands right now, it's not even close! The leaked OpenAI financials put these AI companies in the same profitability territory as utilities, with zero justification for these crazy valuations.
> The question is not whether AI is useful. It is useful, full stop.
And nukes are useful by some metric too.
AI (llms) is not useful in any way that we as humanity should be pushing for. It’s more harmful than good in every way possible. It’s honestly astounding to me that everyone’s ethics are so weak as to believe that somehow these incredible destructive companies are somehow good for humanity.
Technically, it's the time discounted rate of future profits that determine valuations. If you take it as a given that they will provide exactly as much value as they do today, then, yes, your question has the obvious answer of no. If you think AI will provide greater value in the future, the answer depends on the value you think it will have.
Technically, it's the time discounted rate of future profits that determine valuations
Close, it's the time-discounted expectation of future returns. This seems related to future profits but it need not be. Historically, stocks tend to perform poorly after IPOs. There's no guarantee that (say) Anthropic's stock price would ever recover after a post-IPO drop.
The recent attempts by Anthropic et al. to circumvent the usual rules for inclusion in indices have raised red flags all over the place, with many calling it a naked attempt to raid everyone's pension funds for hundreds of billions in ill-gotten capital.
What you have written does not seem to be in close contact with the OP. He talks positively about the GLM news and whatnot. He is highly skeptical only of the "doom" scenarios, including upending most or a massive amount of jobs, and how that is deployed to keep the investment machine working at such breakneck pace.
> It's so boring rehashing the same old tired and worn out ideas like "they're just hyping the danger to pump their shares up."
Yeah poor you this must be much more tiring than being told constantly that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it. Sorry dude, hearing that people don't like that messaging and are replying with copypasta talking points must really be rough for you. Praying for you in these trying times.
Regardless of whether you believe in doom or not, it's this kind of wild, willful misunderstanding that makes people roll their eyes and dismiss you. Yes, people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as marketing hype by crypto bros to increase IPO valuations long ago passed into the realm of conspiracy thinking.
If I were building a technology that I thought posed a risk of exterminating the human race I would simply stop building it. But what do I know, I'm not a genius AI researcher.
people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe
The same could have been said (and has been said) about other tech company employees for all sorts of other reasons in alignment with those companies' goals. Don't you remember how much people used to laugh at Tesla employees for worshipping Elon Musk as some kind of god of engineering and entrepreneurial genius? Or Apple employees in the Steve Jobs reality distortion field?
I would have thought at this point that it'd be well known that the employees of all cult-like tech companies can't be trusted to make a sober evaluation of their companies' justified valuation. We can talk about conflicts of interest and we'd barely be getting started! How about biased selection by hiring managers for the most fervent believers in the company's mission from the get-go?
Sure; they might be caught in their own hype cycle. I am pointing out that the repeated claims in this thread (and by Geohot, ironically enough) that they are NFT crypto bros running a scam is incorrect.
> genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as marketing hype by crypto bros to increase IPO valuations long ago passed into the realm of conspiracy thinking.
"They are not in a thought bubble YOU are in the thought bubble"
No; I'm only stating that your model of reality is off, and I'm correcting it. No, the folks at the frontier labs do not believe they are trying to scam people with an NFT-tier fraud, and their warnings are not an attempt at marketing hype.
So to be clear, you're saying that ALL the people that you currently know of who work at these companies believe they are actively working on, in your words, something that "...poses a potential existential threat to humanity"
Oh nevermind I get it. They're only intentionally working on building something that they believe will end humanity. Well as long as it's not intentional crypto-tier scamming, it's all good then.
I'm convinced. Consider my model of reality corrected. Thanks homie.
Their usual stance is something along the lines of their company is creating AI correctly--not that AI will inherently destroy humanity--and that them working there helps that end. It's extremely reasonable to question their approach, but here you've jumped from "NFT crypto bros trying to run a scam" to "monsters excited to annihilate the human race," which is a wild leap, and betrays a point of view that seems more driven by anti-AI psychosis than consideration of any evidence.
>here you've jumped from "NFT crypto bros trying to run a scam" to "monsters excited to annihilate the human race," which is a wild leap, and betrays a point of view that seems more driven by anti-AI psychosis than consideration of any evidence.
This would be a great point if I had introduced either the NFT/crypto comparison OR the "existential threat" parameters, but if you read through the thread, you introduced both.
Introducing parameters for the other person, then use those to call them crazy when they operate inside them. Actual DA-tier tactic, my guy. Yikes.
My initial reply was "being told constantly that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it."
As in, we are constantly told that everything is going to be shit and there's nothing we can do, because there is a ton of moneyed interest in it.
You're the one that made the leap to "crypto" scamming in the reply and introduced the existential threat aspect. Both in the same comment, actually.
Then apparently tried to crazy-make me for (admittedly dickishly) pointing out that trying to build something that is an existential threat to humanity (your words, their belief, none of my words or belief) is actually worse than crypto scamming (again, both things which you introduced to the thread, btw).
But yeah shame on me for not taking you seriously and providing evidence I guess.
Odd how the bar for me is "providing evidence" but you're happy to outright lie that I was going from crypto scamming and existential threats in order to score a rhetorical point by claiming "AI psychosis" in a useless internet thread. Yeah I must be the unhinged one here, surely.
> Some AI is useless. Some is incredibly powerful... It would be great to explore those new ideas and possibilities.
I agree, but it's not so mysterious what will win out. Even if the criticism is repeated so often, that's because much of it is still valid.
LLMs are not AGI. A statistical model of language helps fill in gaps. This is super useful for new and much improved UI/UX ideas that converge with better accessibility. Similar is true for generating images, video, audio, etc. There are situations where it's the right tool for expressing an idea.
What we need is a sense of maturity. The limitations are very clear to everyone now, and we're already past the disillusionment. If we can rein in the abuse, there should be a good path forward. The technology is already boring and that's a very good sign.
I love this website because people will write stuff like “I believe the hype because my brain is big, ‘believing the hype’ is actually very complex and cannot be embraced by lesser intellects”
There are reasonable anti- and pro-AI views. There are also unreasonable ones on both sides, and it's best if they are ignored.
However, on a site like HN or Reddit, you're far more likely to hear squawking about "stochastic parrots" or a rant about AI water usage than the mirror on the pro-AI side, making them harder to ignore.
It’s funny that a lot of what gets framed as pro- vs anti- AI arguments would be better described as past/present vs future arguments. If one guy is saying “that’s a stochastic parrot” and the second guy says “it’s going to achieve apotheosis and bring about global communism or feudalism or whatever while maximizing shareholder value” they’re not arguing about technology, they’re arguing about the second guy’s status as a clairvoyant.
You could post “fortune tellers used to be considered fraudsters and charlatans” and reasonably expect a “Get a load of the Luddite over here. Go raise a barn, Josiah!” response on the internet these days
Specifically Anthropic's whole PR has all been about danger, safety, doomerism all to make themselves indirectly more important and central to the debate.
Calling meetings in Washington DC in order to let everyone know they made a cyberweapon is part of those PR moves. Then they seem surprised lawmakers actually called them out and asked to stop serving that model.
I know this is the cynical take but I cannot unsee the elephant in the room: This doomerism allowed Anthropic to be the center of every AI conversation right now. Their market cap and upcoming IPO is indirectly benefiting from this.
I also cannot take that Anthropic while letting everyone know that Doom is coming (or is already here), are also the ones that want to decide who can profit from this Doomerism. This is how every benevolent dictators start.
Journalism and public media in general needs to be a licensed profession with tangible standards, and malpractice suits should be pursued aggressively.
Would solve a lot of problems in the US, actually. Being financially incentivized to gleefully lie and spread misinformation at the expense of others should not be protected speech.
The SFBA culture has given me the ick for a while now. Anyone who has done web development in the last few years is inevitably exposed to it. Idk how to describe it without breaking the rules of the site so I’ll just say nothing.
> Can someone write an AI 2027 but instead of some totalizing doom propaganda it talks about the bubble unwinding and what we can do to prevent this kind of crap in the future? How do we build an economy and society that’s sustainable?
I imagine tons of people have written that article. But no one reads it. They're all busy with the doomiest bullshit Facebook or Tiktok will serve them. It's what gets engagement.
No one is clicking on the "None of the things your scared of are scary and here's why" video when it's up against the "$x is the end of the world and will eat your children" agitprop.
> Remember when everyone died during COVID? And when we finally ended racism during DEI? Can someone write an AI 2027 (...)
Oh no, the obvious strawmans that - despite the author's assertions - people did not actually widely believe in, were strawmans? Crazy, I tell you. It'd seem that people aren't actually as dumb as the author likes to characterize them being.
Sure love this genre of writing. It's a beautifully revealing tour de force in projection and narcissism. "Will somebody please think for all the fools who believe in the obvious nonsense I secretly fear?"
Even in this article, he claims COVID wasn’t a big deal because, essentially, only a million people died in the US instead of everyone. No one ever claimed 100% mortality.
Yet the measures taken far exceeded the damage than if we had let the virus take its course.
The problem wasn't the mortality rate, it's the fact that the media can cleverly turn something in a huge deal by talking endlessly about it. During COVID, there was no other news except the virus on news for 24/7.
>>SF wants to come for your inner life and pimp you out and mediate every interaction and there’s not even a so.
I have been saying this for a while. I visited SF a couple months ago and god do people feel empty from the inside. Everything is revolving around AI this and AI that. Half of these people were not paying attention when we were training gradient boost models and now all of these people are 'AI Agents experts'.
> AI Agents experts
you mean loop engineers?
I think they are trying to express that everyone and their brother is now an expert in AI. But everyone was already a decorated expert with decades of hands on "AI" experience by the summer of 2023. So "experts in agentic AI" is just the continuation of that trend.
I was trying to be funny
Somehow the Chinese phrases 内卷 https://en.wikipedia.org/wiki/Neijuan and 摆烂 ("let it rot" https://zh-yue.wikipedia.org/wiki/%E6%93%BA%E7%88%9B) are quite appropriately used, even in stylistic tone. Don't know how much Chinese geohot actually speaks but it's a nice touch.
Thanks for the references.
Even if Dario and Altman originally believed what they were saying, their scaretactics worked wonders for investment. Their companies are now incredibly incentivized to keep the AI Apocalypse narrative going further and further. It is hard to imagine them stopping, as that may lose them investment.
The last 10 years have been a decade of Big Tech Vaporware. NFTs and the Metaverse were assured to be the future. Once this narrative fails too (which I am almost certain is inevitable) I think society's love affair with Technology being the solver-of-all-things will finally fracture.
I think technology can actually solve a lot more things than it has, the problem is that foundational problem solving does not make a unicorn startup with earthshaking economies of scale in most cases, and especially so if there's a physical world piece involved, and even more so if it's going to go asymptotic.
Just to throw out a random example - could technology, in terms of advances and improvements to allocation, distribution, and consumption, play a part in solving the western US's water issues? Probably. But would it be something that make a trillion dollars and a household name? Maybe not. And could saner policy, like making farmers have to bid for acre feet instead of getting it for basically free + distribution cost also help? Sure.
Likewise, even with AI most software is still crap. Like when have you heard about a doctor who loves their EHR? Like never.
So I think technology could be a solver of at least a lot more things. But we've created a market where people want to exhaust every flavor of flim flam and trend of the moment first. Because we've glorified the business of tech more than the actual improvements and aspirations that should be possible.
> NFTs and the Metaverse
AI is not that bad
It's significantly worse! All NFTs did was separate fools from their money, and crypto in general increased GPU costs. All the Metaverse did was cause Facebook to throw away a bunch of money (lol).
Meanwhile, AI has ruined the whole Internet and inflated the price of everything electronic.
The whole internet AND local computing. This may be cool for people on SF salaries but for the vast majority of the world memory has become a luxury good
While it is driving new hardware costs up remember you can run Linux on a potato and just wait it out. My pixel 8a is a phenomenal computer if ignore the hype monster.
so, twitter went from 98% shit to 99% shit, and landfills get less smartphones; I'd say it's worth it
> will finally fracture.
If true, it will only be replaced with something else. With what is anyone’s guess.
If you look at Anthropic's blogs about their model timelines, there is roughly a ~3m period between a model being in internal preview until release. That means that inside of Anthropic, the next version beyond Mythos/Fable is already in preview, already being tested internally. Despite what Geohot is describing here about GLM, my understanding is that Anthropic employees have spent a significant amount of time grappling with a technology that is considerably ahead of what is available to the public today.
In addition, if you look at the graph of LOC written by Claude vs Ants (I.e. AI vs human), there is an incredibly sharp uptick post-Mythos internal preview. Something like from 30% to 75% of code inside the company being written by AI.
While I sympathize with the viewpoint here, I still have to admit that that there's a very different feeling to working inside of a company where they've had months of time with a model that's at the frontier, quickly changing the way everyone around them works, and that _they themselves_ control the keys to.
If Geohot had those keys, I can be 100% confident he'd be raising the alarm at the top of his lungs about it.
If the March leak of Claude Code was Mythos / Fable in preview ... I'm not sure I'm that worried about AI capabilities.
Seriously -- if you dig through that source code, and then listen to the messaging, it seems hard to keep a straight face.
Also, hasn't this company been claiming that almost all their code is written with AI for significantly longer than "post-Mythos internal preview"?
HN is not immune to this. I do not take HN seriously, because empirically so many takes across so many subjects have been wrong in a melodramatic fashion and the “adults”, often people with training and first hand experience have to show up to set the subject matter right before someone gets hurt. But as a barometer of unhinged hype? Only X is comparable.
I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export control issue, it's obviously not good for profits. If you assume AGI is possible, you must agree that it would be at the very least profoundly destabilizing. The companies are built around this assumption.
I agree. There are 2 groups of people:
1. Those, like OP, that believe AI danger and disruption is all hype to boost valuations.
2. Those, like myself, that believe AI danger and disruption is very real and presents dilemmas (but also opportunities) for society, so we must tread carefully.
The first position is not logically consistent with reality. The danger is real: we have already seen hints about how this will impact everything from jobs to warfare to mental health. that danger does not increase valuation, in fact it does the opposite because of the need for government regulation.
*why would they warn if it they didn’t believe it.
2 words: regulatory capture.
Ban Chinese models because they’re “bad”. (can’t have head to head competition).
Bad open weight model (can’t let people escape the subscription model).
People have been warning about the dangers of advanced AI since long before open weight competition was a consideration.
Setting up for the eventual bailout. Remember - "we can't let banks fail or it'll be a depression"?
I remember a meme that spread to that effect, but it was similarly divorced from reality. Hundreds of banks failed during the financial crisis, including multiple which were very large. There were a few cases like Wachovia, where an unhealthy bank was required to sell itself in order to avoid a technical failure that would have impacted over 10 million Americans' access to their money. But this was still unpleasant for existing Wachovia shareholders.
> Why would a company warn that their technology is dangerous if it wasn't?
Because it makes their technology seem more important and powerful?
Do you usually believe 100% what a company says about itself?
> Mythos export control issue, it's obviously not good for profits
Who cares about profits? None of these companies make money like a business currently. It's all speculative valuation and influenced by the same hype/doom cycle
Why would a company warn that their technology is dangerous if it wasn't?
The whole point of the article is to answer this question, and here's the answer:
Because all the AI doom fear-mongering is driving sky-high valuations. The more the public panics, the more investors open their wallets.
Is there any evidence of this? It seems like a thing people say all the time, but completely unsubstantiated.
What else is there? The product as delivered today doesn't come anywhere near a justification for these valuations. All of it is built on an expectation of future capabilities, and that's where the Dario-penned doom papers come in.
>Why would a company warn that their technology is dangerous if it wasn't?
Because Americans are obsessed with eschatological narratives and action hero stories. I talked to a Palantir guy once who told me that he loves it when journalists describe his company as a James Bond villain because every single time the market cap goes up.
If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring
It’s the same narrative as “the communists are developing nukes so we need to develop first”.
Regulatory capture and false AGI hyping, what else? You'd have to be willingly ignoring the writings on the wall if you still haven't identified it yet.
> it's obviously not good for profits.
Weird to think profits matter half as much as ever-increasing valuations, driven by memetic bullshit. E.g. the entire point of the article you're commenting on.
Anthropic's valuation is driven by its revenue growth. Export controls harm this. Hype certainly plays a role, but it's simply not the case that revenue is not a major part of the picture.
> Why would a company warn that their technology is dangerous if it wasn't?
It helps to look at the history of the AI Alarmist, EA-adjacent mindset. They believe AI could be Dangerous (<--- capital D) if not responsibly managed by the right people (spoiler: themselves or politicians who listen to them). The capital D means more than just some economic and employment disruption, they're thinking much bigger, like "existential threat to humanity" big.
The most extreme alarmists are basically LARPing Terminator 2 and want to shut it down before SkyNet takes over but the more moderate alarmists believe AI is too potentially beneficial walk away from. And they acknowledge other nations and various bad actors won't stop anyway, so they believe it's almost a sacred duty to move forward quickly albeit very responsibly. Yes, this leads to some fairly conflicted reasoning and priorities, like they need to keep gaining access to gobs of capital to ensure they are the ones to reach AGI first, as that's the only way to ensure "god" is benevolent.
While I'm sure there are execs who are primarily (or purely) mercenary, I also think some, if not many, other people involved sincerely believe (or at least hope) they are "the good guys" playing a crucial role in an era of historical importance. Obviously, this has some heavy vibes as well as a being quite a buff to one's self-importance. They face each new day feeling the weight of bearing such a consequential role in shaping the future of humanity. Us mere mortals can only pray their wisdom and altruistic ethics match their humility. </s>
So... yeah, that's why we see some weirdly whiplash messaging.
What's the evidence that the doom narrative is connected to valuations? It seems more like a marketing/recruiting strategy. As the post points out, institutional investors generally think the idea of mass unemployment is BS, and they are investing accordingly.
Dumb thing to ask tbh.
Dangerous new tech with military applications sounds harder to ignore than AI that spits out text.
AI obviously has military applications, no doom required to think the military will want a lot of it. I'm asking about the extreme scenarios like "AI is gonna take everyone’s jobs" and the other "totalizing narratives" that geohot is referencing.
If this is true: "AI is gonna take everyone’s jobs" then AI is the terminal technology and everyone who doesn't want to be poor forever needs to throw all their money in it.
Of course it impacts company valuation.
Companies say all sorts of things which investors don't take at face value, often for the sake of marketing and recruiting. Meta circa 2020 wanted us to believe we'd be swapping NFTs in a virtual mall and having all our meetings strapped into VR headsets by now. That doesn't mean their market cap at the time was based on an assumption they were right, in fact it is much higher now that the metaverse idea flopped. Unlike that situation where money was being spent on projects that turned out to be mostly pointless, AI labs don't have much of a financial stake in a scenario such as "lots of demand for AI, leading to mass unemployment and/or doomsday" as opposed to one like "lots of demand for AI, leading to the world being mostly the same as before but with more AI and expensive RAM".
Simple minds want to believe one simple thing and then rationalize everything else to force consistency with that one simple idea.
If you want to believe the simple idea that AI is mostly hype, then you'll get stuck in a multi-year loop talking about stochastic parrots, ridiculous valuations, and doomer scaredycats.
But the real world isn't so simple. Multiple seemingly contradictory things can be true at the same time.
Some AI is useless. Some is incredibly powerful and useful even though it makes mistakes. Some companies are wildly overvalued. Some extremely large and expensive companies will quadruple from here. Some frightening scenarios will look silly in hindsight. Other frightening things will happen that none of the doomers foresee.
It would be great to explore those new ideas and possibilities.
It's so boring rehashing the same old tired and worn out ideas like "they're just hyping the danger to pump their shares up."
Some AI is useless
The question is not whether AI is useful. It is useful, full stop.
The question is whether AI is useful enough to justify valuations that dwarf the GDP of all but the top 20 countries in the world. As it stands right now, it's not even close! The leaked OpenAI financials put these AI companies in the same profitability territory as utilities, with zero justification for these crazy valuations.
> The question is not whether AI is useful. It is useful, full stop.
And nukes are useful by some metric too.
AI (llms) is not useful in any way that we as humanity should be pushing for. It’s more harmful than good in every way possible. It’s honestly astounding to me that everyone’s ethics are so weak as to believe that somehow these incredible destructive companies are somehow good for humanity.
Technically, it's the time discounted rate of future profits that determine valuations. If you take it as a given that they will provide exactly as much value as they do today, then, yes, your question has the obvious answer of no. If you think AI will provide greater value in the future, the answer depends on the value you think it will have.
Technically, it's the time discounted rate of future profits that determine valuations
Close, it's the time-discounted expectation of future returns. This seems related to future profits but it need not be. Historically, stocks tend to perform poorly after IPOs. There's no guarantee that (say) Anthropic's stock price would ever recover after a post-IPO drop.
The recent attempts by Anthropic et al. to circumvent the usual rules for inclusion in indices have raised red flags all over the place, with many calling it a naked attempt to raid everyone's pension funds for hundreds of billions in ill-gotten capital.
What you have written does not seem to be in close contact with the OP. He talks positively about the GLM news and whatnot. He is highly skeptical only of the "doom" scenarios, including upending most or a massive amount of jobs, and how that is deployed to keep the investment machine working at such breakneck pace.
> It's so boring rehashing the same old tired and worn out ideas like "they're just hyping the danger to pump their shares up."
Yeah poor you this must be much more tiring than being told constantly that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it. Sorry dude, hearing that people don't like that messaging and are replying with copypasta talking points must really be rough for you. Praying for you in these trying times.
Regardless of whether you believe in doom or not, it's this kind of wild, willful misunderstanding that makes people roll their eyes and dismiss you. Yes, people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as marketing hype by crypto bros to increase IPO valuations long ago passed into the realm of conspiracy thinking.
If I were building a technology that I thought posed a risk of exterminating the human race I would simply stop building it. But what do I know, I'm not a genius AI researcher.
people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe
The same could have been said (and has been said) about other tech company employees for all sorts of other reasons in alignment with those companies' goals. Don't you remember how much people used to laugh at Tesla employees for worshipping Elon Musk as some kind of god of engineering and entrepreneurial genius? Or Apple employees in the Steve Jobs reality distortion field?
I would have thought at this point that it'd be well known that the employees of all cult-like tech companies can't be trusted to make a sober evaluation of their companies' justified valuation. We can talk about conflicts of interest and we'd barely be getting started! How about biased selection by hiring managers for the most fervent believers in the company's mission from the get-go?
Sure; they might be caught in their own hype cycle. I am pointing out that the repeated claims in this thread (and by Geohot, ironically enough) that they are NFT crypto bros running a scam is incorrect.
> genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as marketing hype by crypto bros to increase IPO valuations long ago passed into the realm of conspiracy thinking.
"They are not in a thought bubble YOU are in the thought bubble"
I am absolutely BTFO'd, you got me.
No; I'm only stating that your model of reality is off, and I'm correcting it. No, the folks at the frontier labs do not believe they are trying to scam people with an NFT-tier fraud, and their warnings are not an attempt at marketing hype.
So to be clear, you're saying that ALL the people that you currently know of who work at these companies believe they are actively working on, in your words, something that "...poses a potential existential threat to humanity"
Oh nevermind I get it. They're only intentionally working on building something that they believe will end humanity. Well as long as it's not intentional crypto-tier scamming, it's all good then.
I'm convinced. Consider my model of reality corrected. Thanks homie.
Their usual stance is something along the lines of their company is creating AI correctly--not that AI will inherently destroy humanity--and that them working there helps that end. It's extremely reasonable to question their approach, but here you've jumped from "NFT crypto bros trying to run a scam" to "monsters excited to annihilate the human race," which is a wild leap, and betrays a point of view that seems more driven by anti-AI psychosis than consideration of any evidence.
>here you've jumped from "NFT crypto bros trying to run a scam" to "monsters excited to annihilate the human race," which is a wild leap, and betrays a point of view that seems more driven by anti-AI psychosis than consideration of any evidence.
This would be a great point if I had introduced either the NFT/crypto comparison OR the "existential threat" parameters, but if you read through the thread, you introduced both.
Introducing parameters for the other person, then use those to call them crazy when they operate inside them. Actual DA-tier tactic, my guy. Yikes.
My initial reply was "being told constantly that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it."
As in, we are constantly told that everything is going to be shit and there's nothing we can do, because there is a ton of moneyed interest in it.
You're the one that made the leap to "crypto" scamming in the reply and introduced the existential threat aspect. Both in the same comment, actually.
Then apparently tried to crazy-make me for (admittedly dickishly) pointing out that trying to build something that is an existential threat to humanity (your words, their belief, none of my words or belief) is actually worse than crypto scamming (again, both things which you introduced to the thread, btw).
But yeah shame on me for not taking you seriously and providing evidence I guess.
Odd how the bar for me is "providing evidence" but you're happy to outright lie that I was going from crypto scamming and existential threats in order to score a rhetorical point by claiming "AI psychosis" in a useless internet thread. Yeah I must be the unhinged one here, surely.
> Some AI is useless. Some is incredibly powerful... It would be great to explore those new ideas and possibilities.
I agree, but it's not so mysterious what will win out. Even if the criticism is repeated so often, that's because much of it is still valid.
LLMs are not AGI. A statistical model of language helps fill in gaps. This is super useful for new and much improved UI/UX ideas that converge with better accessibility. Similar is true for generating images, video, audio, etc. There are situations where it's the right tool for expressing an idea.
What we need is a sense of maturity. The limitations are very clear to everyone now, and we're already past the disillusionment. If we can rein in the abuse, there should be a good path forward. The technology is already boring and that's a very good sign.
I love this website because people will write stuff like “I believe the hype because my brain is big, ‘believing the hype’ is actually very complex and cannot be embraced by lesser intellects”
There are reasonable anti- and pro-AI views. There are also unreasonable ones on both sides, and it's best if they are ignored.
However, on a site like HN or Reddit, you're far more likely to hear squawking about "stochastic parrots" or a rant about AI water usage than the mirror on the pro-AI side, making them harder to ignore.
It’s funny that a lot of what gets framed as pro- vs anti- AI arguments would be better described as past/present vs future arguments. If one guy is saying “that’s a stochastic parrot” and the second guy says “it’s going to achieve apotheosis and bring about global communism or feudalism or whatever while maximizing shareholder value” they’re not arguing about technology, they’re arguing about the second guy’s status as a clairvoyant.
You could post “fortune tellers used to be considered fraudsters and charlatans” and reasonably expect a “Get a load of the Luddite over here. Go raise a barn, Josiah!” response on the internet these days
"I cannot be contradicted because I can hold conflicting viewpoints" was Chomsky's bit, HN can only take credit for imitating it.
“Dario Amodei doesn’t seem like he can tell the future” - Simple. Pedestrian. The roughshod cogitation of a country oaf.
“Maybe he can” - Complex. Divine, possibly? A breathtaking filigree of nuance, like an Alex Grey painting of conceptual allemande.
I came to the same conclusion.
Specifically Anthropic's whole PR has all been about danger, safety, doomerism all to make themselves indirectly more important and central to the debate.
Calling meetings in Washington DC in order to let everyone know they made a cyberweapon is part of those PR moves. Then they seem surprised lawmakers actually called them out and asked to stop serving that model.
I know this is the cynical take but I cannot unsee the elephant in the room: This doomerism allowed Anthropic to be the center of every AI conversation right now. Their market cap and upcoming IPO is indirectly benefiting from this.
I also cannot take that Anthropic while letting everyone know that Doom is coming (or is already here), are also the ones that want to decide who can profit from this Doomerism. This is how every benevolent dictators start.
> This is how every benevolent dictator starts
I don’t think “benevolent” is necessary in that sentence. It’s how many non-benevolent dictatorships started.
> This is how every benevolent dictators start.
Every dictator is benevolent for the people that agree with them.
Journalism and public media in general needs to be a licensed profession with tangible standards, and malpractice suits should be pursued aggressively.
Would solve a lot of problems in the US, actually. Being financially incentivized to gleefully lie and spread misinformation at the expense of others should not be protected speech.
Great article. Holtz is growing on me ngl…
The SFBA culture has given me the ick for a while now. Anyone who has done web development in the last few years is inevitably exposed to it. Idk how to describe it without breaking the rules of the site so I’ll just say nothing.
> Can someone write an AI 2027 but instead of some totalizing doom propaganda it talks about the bubble unwinding and what we can do to prevent this kind of crap in the future? How do we build an economy and society that’s sustainable?
I imagine tons of people have written that article. But no one reads it. They're all busy with the doomiest bullshit Facebook or Tiktok will serve them. It's what gets engagement.
No one is clicking on the "None of the things your scared of are scary and here's why" video when it's up against the "$x is the end of the world and will eat your children" agitprop.
> Remember when everyone died during COVID? And when we finally ended racism during DEI? Can someone write an AI 2027 (...)
Oh no, the obvious strawmans that - despite the author's assertions - people did not actually widely believe in, were strawmans? Crazy, I tell you. It'd seem that people aren't actually as dumb as the author likes to characterize them being.
Sure love this genre of writing. It's a beautifully revealing tour de force in projection and narcissism. "Will somebody please think for all the fools who believe in the obvious nonsense I secretly fear?"
I stopped taking this guy seriously after he proposed "just" digging canals to bring desalinated ocean water into the U.S. desert.
Even in this article, he claims COVID wasn’t a big deal because, essentially, only a million people died in the US instead of everyone. No one ever claimed 100% mortality.
COVID was a risk management disaster that should not have escalated to the level that it did if there was not so much political overreach.
Yet the measures taken far exceeded the damage than if we had let the virus take its course.
The problem wasn't the mortality rate, it's the fact that the media can cleverly turn something in a huge deal by talking endlessly about it. During COVID, there was no other news except the virus on news for 24/7.
> If San Francisco was nuked tomorrow, the world would feel a weight off their shoulders.
Who Would Jesus Nuke?