> We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.”
What do people think is the probability that OpenAI would ever actually do this?
Glaring lack of, "We will not participate in the creation or operation of 'kill bots'."
I can't believe it has to be said. Yet, here we are. Nice to haves include: "We will not participate in the use of AI for mass surveillance," and "We will not participate in the use of AI for (cyber-)warfare."
Outside of coding, that is probably the most lucrative place for AI to be used in. The theatre of war is forgiving of small collateral damage and it’s like this technology was built for kill bots.
What a great opportunity to demonstrate the alignment problem between harmful AI, economic incentives, and standing up for ones principles in spite of having something to gain.
the technology openai-sells is actually not that good for kill bots, we have boston dynamics for that. I mean to be real here, they're already better than human soldiers, deploying 100 of the doggies and letting them run loose could wipe out any fortified group.
Especially if you include things that are not normally acceptable such as suicide bombers, poison gas, etc.
Also it has been proven that in real modern warfare cheap drones seem to dominate. So unless we have a kill-bot that can withstand explosives while staying lightweight and operable with good KD (drones are 1.0 or less). kill-bots would have to have a KD of 100 to break even.
> Power in the future can either be held by a small handful of companies using and controlling superintelligence, or it can be held in a decentralized way by people. We believe the latter is much better...
Superintelligence aside, power in the present is already held by a small handful of companies, at least in the west. The principles are pretty good, vacuous though they may be.
read: https://openai.com/index/our-principles/
then analyze it critically through the lens of https://www.orwellfoundation.com/the-orwell-foundation/orwell/essays-and-other-works/politics-and-the-english-language/
translate the original article using Orwells principles - what is it really saying?
Gemini gave a pretty harsh interpretation, ChatGPT was curiously fence-sitty. Make of it what you will.
tldr: We are building a machine that we do not fully understand, which will upend the global economy and introduce severe new risks, but we promise we are doing it for your own good.
This is the key, every AI company makes a big deal about how AGI will be transformative but we're just supposed to take it on faith that this transformation will be good.
SamAs hypothetical friends - not sure if he has any _real_ friends as that requires trust, which is famously antithetical to interacting with SamA - becoming more powerful and wealthy.
I read "widespread flourishing" as referring to a scope of influence and "at a level that is difficult to imagine" as referring to an amount of accumulated wealth.
But surely the people who aren't committing to not use technology for autonomous death deserve a more charitable reading.
Not succumbing to being cynical I can imagine total elimination of food shortages by completely making farming robot driven, AI driven gene therapy to end most disease, robot driven scalable power generation through building solar panels in an automated fashion, installation and maintenance, one robot at home that can replace all tradesmen etc. There is a lot of stuff that could be possible if the promises pan out and they don’t seem completely out of reach at this point.
When a company writes down principles, you should be highly skeptical.
- Democratization.
Why is it your prerogative sam bro? In other words, what he means is consolidate access so "We" can democratize. We choose who gets what.
- Empowerment.
People are empowered by default. Its the totalitarians who curtail the empowerment. The fact that sam things he has the power to "empower" people is arrogant at best. People are empowered already, you just build the tools and make the tools accessible at a reasonable price.
- Universal prosperity.
This one pisses me off the most. Who TF made you the benevolent mayor of universe? Are you running for president of the universe and people ask: Hey sam, what would you do as a president of the universe? "I will bring universal prosperity".. yaaaay Sama for president. FFS!
- Adaptability
Yep. we'll kiss the ring of whoever is in power, until we get in power. then we will adapt if needed to your needs.
- Give people a voice.
Read: ensure you control their voice.
My take: who tf are you to give anyone a voice? Everyone HAS a voice.
- Build connection and community.
Read: ensure that you control all the connections and communities so that you can steer elections and other important things.
My take: people have been connecting already for thousands of years.
- Serve everyone
Read: control who you serve.
My take: Serve everyone, except for totalitarian regimes and people with ideas that are not aligned with ours.
I wonder how much money OpenAI has spent lobbying for alternative economic models research (or better—spent on such research themselves). Then compare that to how much they’ve spent lobbying to enable deploying this all as fast as possible. Those numbers will speak louder than any words on their principles.
1. Democratization is centralization. We will resist the potential of this technology to consolidate power in the hands of the few, by consolidating it in the hands of us, who are not few but correct.
2. Empowrment is compliance. We believe AGI can empower everyone to achieve the goals we have determined are worth achieving.
3. Prosperity is scarcity. We want a future where everyone can have an excellent life, which will require new economic models because the old ones will no longer function, for reasons unrelated to us.
4.Resilience is dependence. AGI will introduce new risks, which only AGI can solve, which only we can build.
5. Adaptability is revisionism. We continue to believe the only way to meet the challenges of an unpredictable future is to be prepared to update our positions, our charter, our nonprofit status, our safety commitments, our board, our cofounders, and our prior statements, all of which were operative at the time and are now inoperative and were never said.
6. Please don't look at our financials. They are horrible and we are hoping to sucker people into an IPO before all of this implodes. The least your Grandma can do for us is give us 2% of her S&P 500 portfolio so we can exit before it goes to zero. This is AGI after all.
Why even put this out? This is weeks after the whole Department of War thing, days after the article about Sam Altman being a pathological liar (as if we needed one).
If the intention is to bury all that then I think it's going to have the exact opposite effect and make everyone remember.
Like who is the intended audience and what purpose does this serve?
I can't imagine that this will have the same powerful effect that Google's 'don't be evil' stuff did all those years ago.
People are just too cynical and have enough experience being burned by big tech companies. You might think that I'm speaking from a place of age and experience but I think this applies to everyone, young and old -- we're all using these devices and services from the cradle now it seems and we've all been burned by them or know someone who has been burned by them -- kids know the big tech rug pull just like they know they rug that they crawl on while sucking on a pacifier.
So what's the point of this? Is the intended audience internal? Like is it just for the people who work at openai to distract them from the news the stories that they hear in the news about their companies and the stuff they hear people say about them in social gatherings before they admit that they work for openai?
All the major AI shops are out trying to be the king of the jungle -- I don't think there can be a market in the end for all of them to be worth 2T+ giants.
> Here are the principles that guide our work.
> 1. Democratization. We will resist the potential of this technology to consolidate power in the hands of the few.
For example they could publish their models and research... instead of doing the opposite of what they claim being their very first principle.
Or they could resist harvesting everyone's work for free to turn into their own revenue
Remember this one of OpenAI's principles?
> We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.”
What do people think is the probability that OpenAI would ever actually do this?
Funny how true adherence to this is only with open-sourcing the models.
And this won’t happen.
So another “do no evil” bla-bla which will ultimately be dropped
Glaring lack of, "We will not participate in the creation or operation of 'kill bots'."
I can't believe it has to be said. Yet, here we are. Nice to haves include: "We will not participate in the use of AI for mass surveillance," and "We will not participate in the use of AI for (cyber-)warfare."
Outside of coding, that is probably the most lucrative place for AI to be used in. The theatre of war is forgiving of small collateral damage and it’s like this technology was built for kill bots.
What a great opportunity to demonstrate the alignment problem between harmful AI, economic incentives, and standing up for ones principles in spite of having something to gain.
the technology openai-sells is actually not that good for kill bots, we have boston dynamics for that. I mean to be real here, they're already better than human soldiers, deploying 100 of the doggies and letting them run loose could wipe out any fortified group.
Especially if you include things that are not normally acceptable such as suicide bombers, poison gas, etc.
Also it has been proven that in real modern warfare cheap drones seem to dominate. So unless we have a kill-bot that can withstand explosives while staying lightweight and operable with good KD (drones are 1.0 or less). kill-bots would have to have a KD of 100 to break even.
Counterpoint: Killbots are vulnerable to smaller, cheaper bots deployed in defensive positions.
> Power in the future can either be held by a small handful of companies using and controlling superintelligence, or it can be held in a decentralized way by people. We believe the latter is much better...
Superintelligence aside, power in the present is already held by a small handful of companies, at least in the west. The principles are pretty good, vacuous though they may be.
This principle would be good if not for the irony of ClosedAI saying that.
Having it in the hands of public companies or foundations seems preferable to me to having it in the hands of private companies or individuals.
"Principles" of "Open" AI
* This will change anytime we want, whether you agree or not
* For employees who are following current "principles", when we change, if you are strict about principles, then please leave, we will hire new people
A prompt I fired:
Gemini gave a pretty harsh interpretation, ChatGPT was curiously fence-sitty. Make of it what you will.tldr: We are building a machine that we do not fully understand, which will upend the global economy and introduce severe new risks, but we promise we are doing it for your own good.
> We envision a world with widespread flourishing at a level that is currently difficult to imagine
Help me imagine it, what are some examples of widespread flourishing we can look forward to?
This is the key, every AI company makes a big deal about how AGI will be transformative but we're just supposed to take it on faith that this transformation will be good.
absolute underpants gnomes reasoning
SamA becoming more powerful and wealthy.
SamAs hypothetical friends - not sure if he has any _real_ friends as that requires trust, which is famously antithetical to interacting with SamA - becoming more powerful and wealthy.
Need any more examples of flourishing?
In a simplified world with 1 trillionaire and 99 people who have $1 each, the average person has ~$10 billion. The average person is flourishing!
I read "widespread flourishing" as referring to a scope of influence and "at a level that is difficult to imagine" as referring to an amount of accumulated wealth.
But surely the people who aren't committing to not use technology for autonomous death deserve a more charitable reading.
Maybe something like swarms of autonomous killer drones invading Greenland and Canada?
There are people that will call you a luddite for this.
Not succumbing to being cynical I can imagine total elimination of food shortages by completely making farming robot driven, AI driven gene therapy to end most disease, robot driven scalable power generation through building solar panels in an automated fashion, installation and maintenance, one robot at home that can replace all tradesmen etc. There is a lot of stuff that could be possible if the promises pan out and they don’t seem completely out of reach at this point.
The reason for food shortages is not scarcity of food, nor the reason for shelter, clothing, or transportation.
We have enough technology and resources to make it a heaven for everyone except for unpreventable diseases or similar.
Yet we do not because we did not crack the human-alignment problem.
That’s the problem we need to solve, not the resource problem.
If we solve the resource problem before the human alignment problem, we will cause unimaginable suffering.
The more specific the better
"The world is going to be so much better! We're not really sure how, but, trust us, definitely better!"
Those are my principles, and if you don't like them... well, I have others!
First thing I thought about. I have no reason to believe that it’s not accurate for most contemporary tech corps.
When a company writes down principles, you should be highly skeptical.
- Democratization. Why is it your prerogative sam bro? In other words, what he means is consolidate access so "We" can democratize. We choose who gets what.
- Empowerment. People are empowered by default. Its the totalitarians who curtail the empowerment. The fact that sam things he has the power to "empower" people is arrogant at best. People are empowered already, you just build the tools and make the tools accessible at a reasonable price.
- Universal prosperity. This one pisses me off the most. Who TF made you the benevolent mayor of universe? Are you running for president of the universe and people ask: Hey sam, what would you do as a president of the universe? "I will bring universal prosperity".. yaaaay Sama for president. FFS!
- Adaptability Yep. we'll kiss the ring of whoever is in power, until we get in power. then we will adapt if needed to your needs.
You know who else has principles: Meta. (https://www.meta.com/about/company-info/?srsltid=AfmBOooT6i0...)
- Give people a voice. Read: ensure you control their voice. My take: who tf are you to give anyone a voice? Everyone HAS a voice.
- Build connection and community. Read: ensure that you control all the connections and communities so that you can steer elections and other important things. My take: people have been connecting already for thousands of years.
- Serve everyone Read: control who you serve. My take: Serve everyone, except for totalitarian regimes and people with ideas that are not aligned with ours.
etc. etc.
was expecting this page to be blank...
I believe Groucho Marx once said: "I'm a man of principles. If you don't like them, I have others!"
I wonder how much money OpenAI has spent lobbying for alternative economic models research (or better—spent on such research themselves). Then compare that to how much they’ve spent lobbying to enable deploying this all as fast as possible. Those numbers will speak louder than any words on their principles.
Here is the translated version:
1. Democratization is centralization. We will resist the potential of this technology to consolidate power in the hands of the few, by consolidating it in the hands of us, who are not few but correct.
2. Empowrment is compliance. We believe AGI can empower everyone to achieve the goals we have determined are worth achieving.
3. Prosperity is scarcity. We want a future where everyone can have an excellent life, which will require new economic models because the old ones will no longer function, for reasons unrelated to us.
4.Resilience is dependence. AGI will introduce new risks, which only AGI can solve, which only we can build.
5. Adaptability is revisionism. We continue to believe the only way to meet the challenges of an unpredictable future is to be prepared to update our positions, our charter, our nonprofit status, our safety commitments, our board, our cofounders, and our prior statements, all of which were operative at the time and are now inoperative and were never said.
6. Please don't look at our financials. They are horrible and we are hoping to sucker people into an IPO before all of this implodes. The least your Grandma can do for us is give us 2% of her S&P 500 portfolio so we can exit before it goes to zero. This is AGI after all.
What is true definition of "AGI" in this context?
PRINCIPLES.md
Interesting how this was released on the eve of the Musk v. Altman trial ...
Related:
Altman's 'beliefs' in his response to the moltov cocktail
https://blog.samaltman.com/2279512 (https://news.ycombinator.com/item?id=47724921)
Newsflash: they're changing... again
Why even put this out? This is weeks after the whole Department of War thing, days after the article about Sam Altman being a pathological liar (as if we needed one).
If the intention is to bury all that then I think it's going to have the exact opposite effect and make everyone remember.
Dystopian humblebrag gymnastics.
Why do companies put stuff like this out in 2026?
Like who is the intended audience and what purpose does this serve?
I can't imagine that this will have the same powerful effect that Google's 'don't be evil' stuff did all those years ago.
People are just too cynical and have enough experience being burned by big tech companies. You might think that I'm speaking from a place of age and experience but I think this applies to everyone, young and old -- we're all using these devices and services from the cradle now it seems and we've all been burned by them or know someone who has been burned by them -- kids know the big tech rug pull just like they know they rug that they crawl on while sucking on a pacifier.
So what's the point of this? Is the intended audience internal? Like is it just for the people who work at openai to distract them from the news the stories that they hear in the news about their companies and the stuff they hear people say about them in social gatherings before they admit that they work for openai?
lol "principles" like Sam Altman has any ...
All the major AI shops are out trying to be the king of the jungle -- I don't think there can be a market in the end for all of them to be worth 2T+ giants.