The decision to block all downloads is pretty disruptive, especially for people on pinned known good versions. Its breaking a bunch of my systems that are all launched with `uv run`
> Its breaking a bunch of my systems that are all launched with `uv run`
From a security standpoint, you would rather pull in a library that is compromised and run a credential stealer? It seems like this is the exact intended and best behavior.
You should be using build artifacts, not relying on `uv run` to install packages on the fly. Besides the massive security risk, it also means that you're dependent on a bunch of external infrastructure every time you launch. PyPI going down should not bring down your systems.
We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requirements we have for agent runtimes, lets use this momentum to make our dev environments safer! In such an environment the container would crash, we see the violations, delete it and dont' have to worry about it. We should treat this as an everyday possibility not as an isolated security incident.
This is the security shortcuts of the past 50 years coming back to bite us. Software has historically been a world where we all just trust each other. I think that’s coming to an end very soon.
We need sandboxing for sure, but it’s much bigger than that. Entire security models need to be rethought.
I've been thinking the same thing. And it's somewhat parallel to what happened to meditation vs. drugs. In the old world the dangerous insights required so many years of discipline that you could sort of trust that the person getting the insight would be ok. But then any idiot can get the insight by just eating some shrooms and oops, that's a problem. Mostly self-harm problem in that case. But the dynamic is somewhat similar to what's happening now with LLMs and coding.
Software people could (mostly) trust each other's OSS contributions because we could trust the discipline it took in the first place. Not any more.
This assumes that we can get a locked down, secure, stable bedrock system and sandbox that basically never changes except for tiny security updates that can be carefully inspected by many independent parties.
Which sounds great, but the way things work now tend to be the exact opposite of that, so there will be no trustable platform to run the untrusted code in. If the sandbox, or the operating system the sandbox runs in, will get breaking changes and force everyone to always be on a recent release (or worse, track main branch) then that will still be a huge supply chain risk in itself.
> This assumes that we can get a locked down, secure, stable bedrock system and sandbox that basically never changes except for tiny security updates that can be carefully inspected by many independent parties.
For the most part you can. Just version pin slightly-stale versions of dependencies, after ensuring there are no known exploits for that version. Avoid the latest updates whenever possible. And keep aware of security updates, and affected versions.
Don't just update every time the dependency project updates. Update specifically for security issues, new features, and specific performance benefits. And even then avoid the latest version when possible.
Sure, and that is basically what sane people do now, but that only works until something needs a security patch that was not provided for the old version, and changing one dependency is likely to cascade so now I am open to supply chain attacks in many dependencies again (even if briefly).
To really run code without trust would need something more like a microkernel that is the only thing in my system I have to trust, and everything running on top of that is forced to behave and isolated from everything else. Ideally a kernel so small and popular and rarely modified that it can be well tested and trusted.
> We just can't trust dependencies and dev setups.
In one of my vibe coded personal projects (Python and Rust project) I'm actually getting rid of most dependencies and vibe coding replacements that do just what I need. I think that we'll see far fewer dependencies in future projects.
Also, I typically only update dependencies when either an exploit is known in the current version or I need a feature present in a later version - and even then not to the absolute latest version if possible. I do this for all my projects under the many eyes principal. Finding exploits takes time, new updates are riskier than slightly-stale versions.
Though, if I'm filing a bug with a project, I do test and file against the latest version.
> In one of my vibe coded personal projects (Python and Rust project) I'm actually getting rid of most dependencies and vibe coding replacements that do just what I need. I think that we'll see far fewer dependencies in future projects.
No free lunch. LLMs are capable of writing exploitable code and you don’t get notifications (in the eg Dependabot sense, though it has its own problems) without audits.
So... I'm working on an open source technology to make a literal virtual machine shippable i.e. freezing everything inside it, isolated due to vm/hypervisor for sandboxing, with support for containers too since it's a real linux vm.
The problems you mentioned resonated a lot with me and why I'm building it, any interest in working to solve that together?: https://github.com/smol-machines/smolvm
Thanks for the pointer! Love the premise project. Just a few notes:
- a security focused project should NOT default to train people installing by piping to bash. If i try previewing the install script in the browser it forces download instead of showing as plain text. The first thing i see is an argument
# --prefix DIR Install to DIR (default: ~/.smolvm)
that later in the script is rm -rf deleting a lib folder. So if i accidentally pick a folder with ANY lib folder this will be deleted.
- Im not sure what the comparison to colima with krunkit machines is except you don't use vm images but how this works or how it is better is not 100% clear
- Just a minor thing but people don't have much attention and i just saw aws and fly.io in the description and nearly closed the project. it needs to be simpler to see this is a local sandbox with libkrun NOT a wrapper for a remote sandbox like so many of the projects out there.
Will try reaching you on some channel, would love to collaborate especially on devX, i would be very interested in something more reliable and bit more lightweight in placce of colima when libkrun can fully replace vz
Probably on the side of your project, but did you try SmolBSD? <https://smolbsd.org>
It's a meta-OS for microVMs that boots in 10–15 ms.
It can be dedicated to a single service (or a full OS), runs a real BSD kernel, and provides strong isolation.
Overall, it fits into the "VM is the new container" vision.
Disclaimer: I'm following iMil through his twitch streams (the developer of smolBSD and a contributor to NetBSD) and I truly love what he his doing. I haven't actually used smolBSD in production myself since I don't have a need for it (but I participated in his live streams by installing and running his previews), and my answer might be somewhat off-topic.
Run locally on macs, much easier to install/use, and designed to be "portable" meaning you can package a VM to preserve statefulness and run it somewhere else.
worked in AWS and specifically with firecracker in the container space for 4 years - we had a very long onboarding doc to dev on firecracker for containers... So I made sure to focus on ease of use here.
That's no solution. If you can't trust and/or verify dependencies, and they are malicious, then you have bigger problems than what a sandbox will protect against. Even if it's sandboxed and your host machine is safe, you're presumably still going to use that malicious code in production.
I'm supportive of going further - like restricting what a library is able to do. e.g. if you are using some library to compute a hash, it should not make network calls. Without sub-processes, it would require OS support.
Except that LiteLLM probably got pwned because they used Trivy in CI. If Trivy ran in a proper sandbox, the compromised job could not publish a compromised package.
(Yes, they should better configure which CI job has which permissions, but this should be the default or it won't always happen)
Containers can mean many things, if you mean plain docker default configured containers then no, they are a packaging mechanism not safe environment by themselves.
This is tied to the TeamPCP activity over the last few weeks. I've been responding, and keeping an up to date timeline. I hope it might help folks catch up and contextualize this incident:
Over the last ~15 years I have been shocked by the amount of spam on social networks that could have been caught with a Bayesian filter. Or in this case, a fairly simple regex.
Well, large companies/corporations don't care about Spam because they actually benefit from spam in a way as it boosts their engagement ratio
It just doesn't have to be spammed enough that advertisers leave the platform and I think that they sort of succeed in doing so.
Think about it, if Facebook shows you AI slop ragebait or any rage-inducing comment from multiple bots designed to farm attention/for malicious purposes in general, and you fall for it and show engagement to it on which it can show you ads, do you think it has incentive to take a stance against such form of spam
I just installed Harbor, and it instantly pegged my cpu.. i was lucky to see my processes before the system hard locked.
Basically it forkbombed `grep -r rpcuser\rpcpassword` processes trying to find cryptowallets or something. I saw that they spawned from harness, and killed it.
Got lucky, no backdoor installed here from what i could make out of the binary
Same experience with browser-use, it installs litellm as a dependency. Rebooted mac as nothing was responding; luckily only github and huggingface tokens were saved in .git-credentials and have invalidated them. This was inside a conda env, should I reinstall my os for any potential backdoors?
I wonder at what point ecosystems just force a credential rotation. Trivy and now LiteLLM have probably cleaned out a sizable number of credentials, and now it's up to each person and/or team to rotate. TeamPCP is sitting on a treasure trove of credentials and based on this, they're probably carefully mapping out what they can exploit and building payloads for each one.
It would be interesting if Python, NPM, Rubygems, etc all just decided to initiate an ecosystem-wide credential reset. On one hand, it would be highly disruptive. On the other hand, it would probably stop the damage from spreading.
Interesting tool, will definitely try - just curious, is there a tool (hexora checker) that ensures that hexora itself and its dependencies are not compromised ?
And of course if there is one, I'll need another one for the hexora checker....
A general question - how do frontier AI companies handle scenarios like this in their training data? If they train their models naively, then training data injection seems very possible and could make models silently pwn people.
Do the labs label code versions with an associated CVE to label them as compromised (telling the model what NOT to do)? Do they do adversarial RL environments to teach what's good/bad? I'm very curious since it's inevitable some pwned code ends up as training data no matter what.
Everyone’s (well, except Anthropic, they seem to have preserved a bit of taste) approach is the more data the better, so the databases of stolen content (erm, models) are memorizing crap.
This was a compromise of the library owners github acccounts apparently, so this is not a related scenario to dangerous code in the training data.
I assume most labs don't do anything to deal with this, and just hope that it gets trained out because better code should be better rewarded in theory?
It will only take one agent-led compromise to get some Claude-authored underhanded C into llvm or linux or something and then we will all finally need to reflect on trusting trust at last and forevermore.
Reflect in what way? The primary focus of that talk is that it’s possible to infect the binary of a compiler in a way that source analysis won’t reveal and the binary self replicates the vulnerability into other binaries it generates. Thankfully that particular problem was “solved” a while back [1] even if not yet implemented widely.
However, the broader idea of supply chain attacks remains challenging and AI doesn’t really matter in terms of how you should treat it. For example, the xz-utils back door in the build system to attack OpenSSH on many popular distros that patched it to depend on systemd predates AI and that’s just the attack we know about because it was caught. Maybe AI helps with scale of such attacks but I haven’t heard anyone propose any kind of solution that would actually improve reliability and robustness of everything.
To slightly rephrase a citation from Demobbed (2000) [1]:
The kernel is not just open source, it's a very fast-moving codebase. That's how we win all wars against AI-authored exploits. While the LLM trains on our internal APIs, we change the APIs — by hand. When the agent finally submits its pull request, it gets lost in unfamiliar header files and falls into a state of complete non-compilability. That is the point. That is our strategy.
If that would happen, The worry I would have is of all the sensitive Government servers from all over the world which might be then exploited and the amount of damage which can be caused silently by such a threat actor or something like AWS/GCP/these massive hyperscalers which are also used by the governments around the globe at times.
The possibilities within a good threat could be catastrophic if we assume so, and if we assume nation-states to be interested in sponsoring hacking attacks (which many nations already do) to attack enemy nations/gain leverage. We are looking at damage within Trillions at that point.
But I would assume that Linux might be safe for now, it might be the most looked at code and its definitely something safe.
LLVM might be a bit more interesting as it might go a little unnoticed but hopefully people who are working at LLVM are well funded/have enough funding to take a look at everything carefully to not have such a slip up.
Yeah, and they can write code with vulnerabilities by accident. But this is a new class of problem, where a known trusted contributor can accidentally allow a vulnerability that was added on purpose by the tooling.
But now you have compromise _at scale_. Before poor plebs like us had to artisinally craft every back door. Now we have a technology to automate that mundane exploitation process! Win!
This looks like the same TeamPCP that compromised Trivy. Notice how the issue is full of bot replies. It was the same in Trivy’s case.
This threat actor seems to be very quickly capitalising on stolen credentials, wouldn’t be surprised if they’re leveraging LLMs to do the bulk of the work.
This is bad, especially from a downstream dependency perspective. DSPy and CrewAI also import LiteLLM, so you could not be using LiteLLM as a gateway, but still importing it via those libraries for agents, etc.
Yep, I think the worst impact is going to be from libraries that were using LiteLLM as just an upstream LLM provider library vs for a model gateway. Hopefully, CrewAI and DSPy can get on top of it soon.
I completely removed nanobot after I found that. Luckily, I only used it a few times and inside a docker container. litellm 1.82.6 was the latest version I could find installed, not sure if it was affected.
Does anyone know a good alternate project that works similarly (share multipple LLMs across a set of users)? LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had issues with creating tokens for other users etc.
Virtual Keys is an Enterprise feature. I am not going to pay for something like this in order to provide my family access to all my models. I can do without cost control (although it would be nice) but I need for users to be able to generate a key and us this key to access all the models I provide.
A question from a non-python-security-expert: is committing uv.lock files for specific versions, and only infrequently updating versions a reasonable practice?
But, one of the arguments that I saw online from this was that when a security researcher finds a bug and reports it to the OSS project/Company they then fix the code silently and include it within the new version and after some time, they make the information public
So if you run infrequently updated versions, then you run a risk of allowing hackers access as well.
(An good example I can think of is OpenCode which had an issue which could allow RCE and the security researcher team asked Opencode secretly but no response came so after sometime of no response, they released the knowledge in public and Opencode quickly made a patch to fix that issue but if you were running the older code, you would've been vulnerable to RCE)
> On top of that, the room for vulnerabilities and supply chain attacks has increased dramatically
AI Is not about fancy models, is about plain old Software Engineering. I strongly advised our team of "not-so-senior" devs to not use LiteLLM or LangChain or anything like that and just stick to `requests.post('...')".
Valid, but for all the crap that LangChain gets it at least has its own layer for upstream LLM provider calls, which means it isn't affected by this supply chain compromise (unless you're using the optional langchain-litellm package). DSPy uses LiteLLM as its primary way to call OpenAI, etc. and CrewAI imports it, too, but I believe it prefers the vendor libraries directly before it falls back to LiteLLM.
Not just as a gateway in a lot cases, but CrewAI and DSPy use it directly. DSPy uses it as its only way to call upstream LLM providers and CrewAI falls back to it if the OpenAI, Anthropic, etc. SDKs aren't available.
Do you feel as if people will update litellm without looking at this discussion/maybe having it be automatic which would then lead to loss of crypto wallets/ especially AI Api keys?
Now I am not worried about the Ai Api keys having much damage but I am thinking of one step further and I am not sure how many of these corporations follow privacy policy and so perhaps someone more experienced can tell me but wouldn't these applications keep logs for legal purposes and those logs can contain sensitive information, both of businesses but also, private individuals perhaps too?
If it was not spinning so many Python processes and not overwhelming the system with those (friends found out this is consuming too much CPU from the fan noise!) it would have been much more successful. So similar to xz attack
it does a lot of CPU intensive work
spawn background python
decode embedded stage
run inner collector
if data collected:
write attacker public key
generate random AES key
encrypt stolen data with AES
encrypt AES key with attacker RSA pubkey
tar both encrypted files
POST archive to remote host
I was running it (as a proxy) in my homelab with docker compose using the litellm/litellm:latest image https://hub.docker.com/layers/litellm/litellm/latest/images/... , I don't think this was compromised as it is from 6 months ago and I checked it is the version 1.77.
I guess I am lucky as I have watchtower automatically update all my containers to the latest image every morning if there are new versions.
I also just added it to my homelab this sunday, I guess that's good timing haha.
Only tangentially related: Is there some joke/meme I'm not aware of? The github comment thread is flooded with identical comments like "Thanks, that helped!", "Thanks for the tip!", and "This was the answer I was looking for."
Since they all seem positive, it doesn't seem like an attack but I thought the general etiquette for github issues was to use the emoji reactions to show support so the comment thread only contains substantive comments.
> It also seems that attacker is trying to stifle the discussion by spamming this with hundreds of comments. I recommend talking on hackernews if that might be the case.
When something like this happens, do security researchers instantly contact the hosting companies to suspend or block the domains used by the attackers?
First line of defense is the git host and artifact host scrape the malware clean (in this case GitHub and Pypi).
Domains might get added to a list for things like 1.1.1.2 but as you can imagine that has much smaller coverage, not everyone uses something like this in their DNS infra.
github, pypi, npm, homebrew, cpan, etc etc. should adopt a multi-multi-factor authentication approach for releases. Maybe have it kick in as a requirement after X amount of monthly downloads.
Basically, have all releases require multi-factor auth from more than one person before they go live.
A single person being compromised either technically, or by being hit on the head with a wrench, should not be able to release something malicious that effects so many people.
They would have to find someone else if they grew too big.
Though, the secondary doesn't necessarily have to be a maintainer or even a contributor on the project. It just needs to be someone else to do a sanity check, to make sure it is an actual release.
Heck, I would even say that as the project grows in popularity, the amount of people required to approve a release should go up.
So if I'm developing something I want to use and the community finds it useful but I take no contributions and no feature requests I should have to find another person to deal with?
How do I even know who to trust, and what prevents two people from conspiring together with a long con? Sounds great on the surface but I'm not sure you've thought it through.
It wouldn't prevent a project that has a goal of being purposely malicious, just from pushing out releases that aren't actually releases.
As far as who to trust, I could imagine the maintainers of different high-level projects helping each other out in this way.
Though, if you really must allow a single user to publish releases to the masses using existing shared social infrastructure. Then you could mitigate this type of attack by adding in a time delay, with the ability for users to flag. So instead of immediately going live, add in a release date, maybe even force them to mention the release date on an external system as well. The downside with that approach is that it would limit the ability to push out fixes as well.
But I think I am OK with saying if you're a solo developer, you need to bring someone else on board or host your builds yourself.
Edit: ignore this silliness, as it sidesteps the real problem. Leaving it here because we shouldn't remove our own stupidity.
It's pretty disappointing that safetensors has existed for multiple years now but people are still distributing pth files. Yes it requires more code to handle the loading and saving of models, but you'd think it would be worth it to avoid situations like this.
Yeah, fair enough, the problem here is that the credentials were stolen, the fact that the exploit was packaged into a .pth is just an implementation detail.
Yep, DSPy and CrewAI have direct dependencies on it. DSPy uses it as its primary library for calling upstream LLM providers and CrewAI falls back to it I believe if the OpenAI, Anthropic, etc. SDKs aren't available.
Our modern economy/software industry truly runs on egg-shells nowadays that engineers accounts are getting hacked to create a supply-chain attack all at the same time that threat actors are getting more advanced partially due to helps of LLM's.
First Trivy (which got compromised twice), now LiteLLM.
I work with security researchers, so we've been on this since about an hour ago. One pain I've really come to feel is the complexity of Python environments. They've always been a pain, but in an incident like this, where you need to find whether an exact version of a package has ever been installed on your machine. All I can say is good luck.
The Python ecosystem provides too many nooks and crannies for malware to hide in.
I have created an comment to hopefully steer the discussion towards hackernews if the threat actor is stifling genuine comments in github by spamming that thread with 100's of accounts
Am I the only one having feeling that with LLM-era we have now bigger amount of malicious software lets say parsers/fetchers of credentials/ssh/private keys?
And it is easier to produce them and then include in some 3rd party open-source software? Or it is just our attention gets focused on such things?
LangChain at least has its own layer for upstream LLM provider calls, which means it isn't affected by this supply chain compromise. DSPy uses LiteLLM as its primary way to call OpenAI, etc. and CrewAI imports it, too, but I believe it prefers the vendor libraries directly before it falls back to LiteLLM.
I hope that everyone's course of action will be uninstalling this package permanently, and avoiding the installation of packages similar to this.
In order to reduce supply chain risk not only does a vendor (even if gratis and OS) need to be evaluated, but the advantage it provides.
Exposing yourself to supply chain risk for an HTTP server dependency is natural. But exposing yourself for is-odd, or whatever this is, is not worth it.
Remember that you are programmers and you can just program, you don't need a framework, you are already using the API of an LLM provider, don't put a hat on a hat, don't get killed for nothing.
And even if you weren't using this specific dependency, check your deps, you might have shit like this in your requirements.txt and was merely saved by chance.
An additional note is that the dev will probably post a post-mortem, what was learned, how it was fixed, maybe downplay the thing. Ignore that, the only reasonable step after this is closing a repo, but there's no incentive to do that.
> Remember that you are programmers and you can just program, you don't need a framework, you are already using the API of an LLM provider, don't put a hat on a hat, don't get killed for nothing.
Programming for different LLM APIs is a hassle, this library made it easy by making one single API you call, and in the backstage it handled all the different API calls you need for different LLM providers.
Comparing this project to is-odd seems very disingenuous to me. My understanding is this was the only way you could use llama.cpp with Claude Code for example, since llama.cpp doesn't support the Anthropic compatible endpoint and doing so yourself isn't anywhere near as trivial as your comparison. Happy to be corrected if I'm wrong.
LiteLLM maintainer here, this is still an evolving situation, but here's what we know so far:
1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%20trivy... https://ramimac.me/trivy-teampcp/#phase-09
2. If you're on the proxy docker, you were not impacted. We pin our versions in the requirements.txt
3. The package is in quarantine on pypi - this blocks all downloads.
We are investigating the issue, and seeing how we can harden things. I'm sorry for this.
- Krrish
>1. Looks like this originated from the trivvy used in our ci/cd
Were you not aware of this in the short time frame that it happened in? How come credentials were not rotated to mitigate the trivy compromise?
There are hundreds of PRs fixing valid issues to your github repo seemingly in limbo for weeks. What is the maintainer state over there?
Not really the time for that. There's also PRs being merged every hour of the day.
Is it just in 1.82.8 or are previous versions impacted?
1.82.7 is also impacted if I remember correctly.
1.82.7 doesn't have litellm_init.pth in the archive. You can download them from pypi to check.
EDIT: no, it's compromised, see proxy/proxy_server.py.
1.82.7 has the payload in `litellm/proxy/proxy_server.py` which executes on import.
> - Krrish
Was your account completely compromised? (Judging from the commit made by TeamPCP on your accounts)
Are you in contacts with all the projects which use litellm downstream and if they are safe or not (I am assuming not)
I am unable to understand how it compromised your account itself from the exploit at trivvy being used in CI/CD as well.
>I am unable to understand how it compromised your account itself from the exploit at trivvy being used in CI/CD as well.
Token in CI could've been way too broad.
He would have to state he didn't in fact make all those commits and close the issue.
The decision to block all downloads is pretty disruptive, especially for people on pinned known good versions. Its breaking a bunch of my systems that are all launched with `uv run`
> Its breaking a bunch of my systems that are all launched with `uv run`
From a security standpoint, you would rather pull in a library that is compromised and run a credential stealer? It seems like this is the exact intended and best behavior.
You should be using build artifacts, not relying on `uv run` to install packages on the fly. Besides the massive security risk, it also means that you're dependent on a bunch of external infrastructure every time you launch. PyPI going down should not bring down your systems.
Are you sure you are pinned to a “known good” version?
No one initially knows how much is compromised
That's PyPI's behavior when they quarantine a package.
We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requirements we have for agent runtimes, lets use this momentum to make our dev environments safer! In such an environment the container would crash, we see the violations, delete it and dont' have to worry about it. We should treat this as an everyday possibility not as an isolated security incident.
This is the security shortcuts of the past 50 years coming back to bite us. Software has historically been a world where we all just trust each other. I think that’s coming to an end very soon. We need sandboxing for sure, but it’s much bigger than that. Entire security models need to be rethought.
I've been thinking the same thing. And it's somewhat parallel to what happened to meditation vs. drugs. In the old world the dangerous insights required so many years of discipline that you could sort of trust that the person getting the insight would be ok. But then any idiot can get the insight by just eating some shrooms and oops, that's a problem. Mostly self-harm problem in that case. But the dynamic is somewhat similar to what's happening now with LLMs and coding.
Software people could (mostly) trust each other's OSS contributions because we could trust the discipline it took in the first place. Not any more.
This assumes that we can get a locked down, secure, stable bedrock system and sandbox that basically never changes except for tiny security updates that can be carefully inspected by many independent parties.
Which sounds great, but the way things work now tend to be the exact opposite of that, so there will be no trustable platform to run the untrusted code in. If the sandbox, or the operating system the sandbox runs in, will get breaking changes and force everyone to always be on a recent release (or worse, track main branch) then that will still be a huge supply chain risk in itself.
Don't just update every time the dependency project updates. Update specifically for security issues, new features, and specific performance benefits. And even then avoid the latest version when possible.
Sure, and that is basically what sane people do now, but that only works until something needs a security patch that was not provided for the old version, and changing one dependency is likely to cascade so now I am open to supply chain attacks in many dependencies again (even if briefly).
To really run code without trust would need something more like a microkernel that is the only thing in my system I have to trust, and everything running on top of that is forced to behave and isolated from everything else. Ideally a kernel so small and popular and rarely modified that it can be well tested and trusted.
Would value your opinion on my project to isolate creds from the container:
https://github.com/calebfaruki/tightbeam https://github.com/calebfaruki/airlock
This is literally the thing I'm trying to protect against.
Also, I typically only update dependencies when either an exploit is known in the current version or I need a feature present in a later version - and even then not to the absolute latest version if possible. I do this for all my projects under the many eyes principal. Finding exploits takes time, new updates are riskier than slightly-stale versions.
Though, if I'm filing a bug with a project, I do test and file against the latest version.
> In one of my vibe coded personal projects (Python and Rust project) I'm actually getting rid of most dependencies and vibe coding replacements that do just what I need. I think that we'll see far fewer dependencies in future projects.
No free lunch. LLMs are capable of writing exploitable code and you don’t get notifications (in the eg Dependabot sense, though it has its own problems) without audits.
So... I'm working on an open source technology to make a literal virtual machine shippable i.e. freezing everything inside it, isolated due to vm/hypervisor for sandboxing, with support for containers too since it's a real linux vm.
The problems you mentioned resonated a lot with me and why I'm building it, any interest in working to solve that together?: https://github.com/smol-machines/smolvm
Thanks for the pointer! Love the premise project. Just a few notes:
- a security focused project should NOT default to train people installing by piping to bash. If i try previewing the install script in the browser it forces download instead of showing as plain text. The first thing i see is an argument
# --prefix DIR Install to DIR (default: ~/.smolvm)
that later in the script is rm -rf deleting a lib folder. So if i accidentally pick a folder with ANY lib folder this will be deleted.
- Im not sure what the comparison to colima with krunkit machines is except you don't use vm images but how this works or how it is better is not 100% clear
- Just a minor thing but people don't have much attention and i just saw aws and fly.io in the description and nearly closed the project. it needs to be simpler to see this is a local sandbox with libkrun NOT a wrapper for a remote sandbox like so many of the projects out there.
Will try reaching you on some channel, would love to collaborate especially on devX, i would be very interested in something more reliable and bit more lightweight in placce of colima when libkrun can fully replace vz
Probably on the side of your project, but did you try SmolBSD? <https://smolbsd.org> It's a meta-OS for microVMs that boots in 10–15 ms.
It can be dedicated to a single service (or a full OS), runs a real BSD kernel, and provides strong isolation.
Overall, it fits into the "VM is the new container" vision.
Disclaimer: I'm following iMil through his twitch streams (the developer of smolBSD and a contributor to NetBSD) and I truly love what he his doing. I haven't actually used smolBSD in production myself since I don't have a need for it (but I participated in his live streams by installing and running his previews), and my answer might be somewhat off-topic.
More here <https://hn.algolia.com/?q=smolbsd>
What would the advantage of this be compared to using something like a Firecracker backend for containerd?
Run locally on macs, much easier to install/use, and designed to be "portable" meaning you can package a VM to preserve statefulness and run it somewhere else.
worked in AWS and specifically with firecracker in the container space for 4 years - we had a very long onboarding doc to dev on firecracker for containers... So I made sure to focus on ease of use here.
firecracker does not run on macos and has no GPU support
We need programming languages where every imported module is in its own sandbox by default.
Now is probably a pretty good time to start a capabilities-based language if someone is able to do that. I wish I had the time.
We have one where thats possible: workerd (apache 2.0) no new language needed just a new runtime
I mean, the sandboxing aspect of a language is just one thing.
We should have sandboxing in Rust, Python, and every language in between.
That's no solution. If you can't trust and/or verify dependencies, and they are malicious, then you have bigger problems than what a sandbox will protect against. Even if it's sandboxed and your host machine is safe, you're presumably still going to use that malicious code in production.
I'm supportive of going further - like restricting what a library is able to do. e.g. if you are using some library to compute a hash, it should not make network calls. Without sub-processes, it would require OS support.
Except that LiteLLM probably got pwned because they used Trivy in CI. If Trivy ran in a proper sandbox, the compromised job could not publish a compromised package.
(Yes, they should better configure which CI job has which permissions, but this should be the default or it won't always happen)
Containers prevent this kind of info stealing greatly, only explicitly provided creds would be leaked.
Containers can mean many things, if you mean plain docker default configured containers then no, they are a packaging mechanism not safe environment by themselves.
They don't have access to the host filesystem nor environment variables and this attack wouldn't work.
This is tied to the TeamPCP activity over the last few weeks. I've been responding, and keeping an up to date timeline. I hope it might help folks catch up and contextualize this incident:
https://ramimac.me/trivy-teampcp/#phase-09
Besides main issue here, and the owners account being possibly compromised as well, there's like 170+ low quality spam comments in there.
I would expect better spam detection system from GitHub. This is hardly acceptable.
i'm guessing it's accounts they have compromised with the stealer.
They repeat only six sentences during 100+ comments:
Worked like a charm, much appreciated.
This was the answer I was looking for.
Thanks, that helped!
Thanks for the tip!
Great explanation, thanks for sharing.
This was the answer I was looking for.
Over the last ~15 years I have been shocked by the amount of spam on social networks that could have been caught with a Bayesian filter. Or in this case, a fairly simple regex.
Well, large companies/corporations don't care about Spam because they actually benefit from spam in a way as it boosts their engagement ratio
It just doesn't have to be spammed enough that advertisers leave the platform and I think that they sort of succeed in doing so.
Think about it, if Facebook shows you AI slop ragebait or any rage-inducing comment from multiple bots designed to farm attention/for malicious purposes in general, and you fall for it and show engagement to it on which it can show you ads, do you think it has incentive to take a stance against such form of spam
Yeah, I almost included that part in my comment, but it still sucks.
I just installed Harbor, and it instantly pegged my cpu.. i was lucky to see my processes before the system hard locked.
Basically it forkbombed `grep -r rpcuser\rpcpassword` processes trying to find cryptowallets or something. I saw that they spawned from harness, and killed it.
Got lucky, no backdoor installed here from what i could make out of the binary
Same experience with browser-use, it installs litellm as a dependency. Rebooted mac as nothing was responding; luckily only github and huggingface tokens were saved in .git-credentials and have invalidated them. This was inside a conda env, should I reinstall my os for any potential backdoors?
What is Harness?
Sorry i mean Harbor.. was running terminal bench
I wonder at what point ecosystems just force a credential rotation. Trivy and now LiteLLM have probably cleaned out a sizable number of credentials, and now it's up to each person and/or team to rotate. TeamPCP is sitting on a treasure trove of credentials and based on this, they're probably carefully mapping out what they can exploit and building payloads for each one.
It would be interesting if Python, NPM, Rubygems, etc all just decided to initiate an ecosystem-wide credential reset. On one hand, it would be highly disruptive. On the other hand, it would probably stop the damage from spreading.
Their previous release would be easily caught by static analysis. PTH is a novel technique.
Run all your new dependencies through static analysis and don't install the latest versions.
I implemented static analysis for Python that detects close to 90% of such injections.
https://github.com/rushter/hexora
Interesting tool, will definitely try - just curious, is there a tool (hexora checker) that ensures that hexora itself and its dependencies are not compromised ? And of course if there is one, I'll need another one for the hexora checker....
A general question - how do frontier AI companies handle scenarios like this in their training data? If they train their models naively, then training data injection seems very possible and could make models silently pwn people.
Do the labs label code versions with an associated CVE to label them as compromised (telling the model what NOT to do)? Do they do adversarial RL environments to teach what's good/bad? I'm very curious since it's inevitable some pwned code ends up as training data no matter what.
Everyone’s (well, except Anthropic, they seem to have preserved a bit of taste) approach is the more data the better, so the databases of stolen content (erm, models) are memorizing crap.
This was a compromise of the library owners github acccounts apparently, so this is not a related scenario to dangerous code in the training data.
I assume most labs don't do anything to deal with this, and just hope that it gets trained out because better code should be better rewarded in theory?
I am pretty sure that such measures aren't taken by AI companies, though I may be wrong.
The API/online model inference definitely runs through some kind of edge safeguarding models which could do this.
Look like the Founder and CTO account has been compromised. https://github.com/krrishdholakia
Most his recent commits are small edits claiming responsibility on behalf of "teampcp", which was the group behind the recent Trivy compromise:
https://news.ycombinator.com/item?id=47475888
I was just wondering why the Trivy compromise hit only npm packages, thinking that bigger stuff should appear sooner or later. Here we go...
Or his company is trash and hes moved onto plain old theft.
It will only take one agent-led compromise to get some Claude-authored underhanded C into llvm or linux or something and then we will all finally need to reflect on trusting trust at last and forevermore.
Reflect in what way? The primary focus of that talk is that it’s possible to infect the binary of a compiler in a way that source analysis won’t reveal and the binary self replicates the vulnerability into other binaries it generates. Thankfully that particular problem was “solved” a while back [1] even if not yet implemented widely.
However, the broader idea of supply chain attacks remains challenging and AI doesn’t really matter in terms of how you should treat it. For example, the xz-utils back door in the build system to attack OpenSSH on many popular distros that patched it to depend on systemd predates AI and that’s just the attack we know about because it was caught. Maybe AI helps with scale of such attacks but I haven’t heard anyone propose any kind of solution that would actually improve reliability and robustness of everything.
[1] Fully Countering Trusting Trust through Diverse Double-Compiling https://arxiv.org/abs/1004.5534
The only way to be safe is to constantly change internal API's so that LLM's are useless at kernel code
To slightly rephrase a citation from Demobbed (2000) [1]:
The kernel is not just open source, it's a very fast-moving codebase. That's how we win all wars against AI-authored exploits. While the LLM trains on our internal APIs, we change the APIs — by hand. When the agent finally submits its pull request, it gets lost in unfamiliar header files and falls into a state of complete non-compilability. That is the point. That is our strategy.
1 - https://en.wikipedia.org/wiki/Demobbed_(2000_film)
If that would happen, The worry I would have is of all the sensitive Government servers from all over the world which might be then exploited and the amount of damage which can be caused silently by such a threat actor or something like AWS/GCP/these massive hyperscalers which are also used by the governments around the globe at times.
The possibilities within a good threat could be catastrophic if we assume so, and if we assume nation-states to be interested in sponsoring hacking attacks (which many nations already do) to attack enemy nations/gain leverage. We are looking at damage within Trillions at that point.
But I would assume that Linux might be safe for now, it might be the most looked at code and its definitely something safe.
LLVM might be a bit more interesting as it might go a little unnoticed but hopefully people who are working at LLVM are well funded/have enough funding to take a look at everything carefully to not have such a slip up.
You know that people can already write backdoored code, right?
Yeah, and they can write code with vulnerabilities by accident. But this is a new class of problem, where a known trusted contributor can accidentally allow a vulnerability that was added on purpose by the tooling.
But now you have compromise _at scale_. Before poor plebs like us had to artisinally craft every back door. Now we have a technology to automate that mundane exploitation process! Win!
You still have a human who actually ends up reviewing the code, though. Now if the review was AI powered... (glances at openclaw)
This looks like the same TeamPCP that compromised Trivy. Notice how the issue is full of bot replies. It was the same in Trivy’s case.
This threat actor seems to be very quickly capitalising on stolen credentials, wouldn’t be surprised if they’re leveraging LLMs to do the bulk of the work.
This is bad, especially from a downstream dependency perspective. DSPy and CrewAI also import LiteLLM, so you could not be using LiteLLM as a gateway, but still importing it via those libraries for agents, etc.
Wow, the postmortem for this is going to be brutal. I wonder just how many people/orgs have been affected.
Yep, I think the worst impact is going to be from libraries that were using LiteLLM as just an upstream LLM provider library vs for a model gateway. Hopefully, CrewAI and DSPy can get on top of it soon.
I'm surprised to see nanobot uses LiteLLM: https://github.com/HKUDS/nanobot
LiteLLM wouldn't be my top choice, because it installs a lot of extra stuff. https://news.ycombinator.com/item?id=43646438 But it's quite popular.
I completely removed nanobot after I found that. Luckily, I only used it a few times and inside a docker container. litellm 1.82.6 was the latest version I could find installed, not sure if it was affected.
Looks like all of the LiteLLM CEO’s public repos have been updated with the description “teampcp owns BerriAI” https://github.com/krrishdholakia
Does anyone know a good alternate project that works similarly (share multipple LLMs across a set of users)? LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had issues with creating tokens for other users etc.
I just found https://github.com/jasmedia/InferXgate which looks interesting although quite new and not supporting so many providers.
Bifrost is the only real alternative I'm aware of https://github.com/maximhq/bifrost
Virtual Keys is an Enterprise feature. I am not going to pay for something like this in order to provide my family access to all my models. I can do without cost control (although it would be nice) but I need for users to be able to generate a key and us this key to access all the models I provide.
github.com/mozilla-ai/any-llm :)
pydantic-ai
We just analysed the payload. Technical details here: https://safedep.io/malicious-litellm-1-82-8-analysis/
We are looking at similar attack vectors (pth injection), signatures etc. in other PyPI packages that we know of.
You can see it for yourself here:
https://inspector.pypi.io/project/litellm/1.82.8/packages/fd...
Two URLs found in the exploit: https://checkmarx.zone/raw https://models.litellm.cloud/
A question from a non-python-security-expert: is committing uv.lock files for specific versions, and only infrequently updating versions a reasonable practice?
(I am not a security expert either)
But, one of the arguments that I saw online from this was that when a security researcher finds a bug and reports it to the OSS project/Company they then fix the code silently and include it within the new version and after some time, they make the information public
So if you run infrequently updated versions, then you run a risk of allowing hackers access as well.
(An good example I can think of is OpenCode which had an issue which could allow RCE and the security researcher team asked Opencode secretly but no response came so after sometime of no response, they released the knowledge in public and Opencode quickly made a patch to fix that issue but if you were running the older code, you would've been vulnerable to RCE)
I blogged about this last year[0]...
> ### Software Supply Chain is a Pain in the A*
> On top of that, the room for vulnerabilities and supply chain attacks has increased dramatically
AI Is not about fancy models, is about plain old Software Engineering. I strongly advised our team of "not-so-senior" devs to not use LiteLLM or LangChain or anything like that and just stick to `requests.post('...')".
[0] https://sb.thoughts.ar/posts/2025/12/03/ai-is-all-about-soft...
Valid, but for all the crap that LangChain gets it at least has its own layer for upstream LLM provider calls, which means it isn't affected by this supply chain compromise (unless you're using the optional langchain-litellm package). DSPy uses LiteLLM as its primary way to call OpenAI, etc. and CrewAI imports it, too, but I believe it prefers the vendor libraries directly before it falls back to LiteLLM.
This is a brutal one. A ton of people use litellm as their gateway.
Not just as a gateway in a lot cases, but CrewAI and DSPy use it directly. DSPy uses it as its only way to call upstream LLM providers and CrewAI falls back to it if the OpenAI, Anthropic, etc. SDKs aren't available.
Do you feel as if people will update litellm without looking at this discussion/maybe having it be automatic which would then lead to loss of crypto wallets/ especially AI Api keys?
Now I am not worried about the Ai Api keys having much damage but I am thinking of one step further and I am not sure how many of these corporations follow privacy policy and so perhaps someone more experienced can tell me but wouldn't these applications keep logs for legal purposes and those logs can contain sensitive information, both of businesses but also, private individuals perhaps too?
Maybe then people will start to realize crypto isn't even worth the stored bits.
Irrevocable transfers... What could go wrong?
More details here: https://futuresearch.ai/blog/litellm-pypi-supply-chain-attac...
LiteLLM has like a 1000 dependencies this is expected https://github.com/BerriAI/litellm/blob/main/requirements.tx...
If it was not spinning so many Python processes and not overwhelming the system with those (friends found out this is consuming too much CPU from the fan noise!) it would have been much more successful. So similar to xz attack
it does a lot of CPU intensive work
I can't tell which part of that is expensive unless many multiples of python are spawned at the same time. Are any of the payloads particularly large?
Looking forward to a Veritasium video about this in the future, like the one they recently did about the xz backdoor.
That was massively more interesting, this is just a straight-up hack.
I was running it (as a proxy) in my homelab with docker compose using the litellm/litellm:latest image https://hub.docker.com/layers/litellm/litellm/latest/images/... , I don't think this was compromised as it is from 6 months ago and I checked it is the version 1.77.
I guess I am lucky as I have watchtower automatically update all my containers to the latest image every morning if there are new versions.
I also just added it to my homelab this sunday, I guess that's good timing haha.
Only tangentially related: Is there some joke/meme I'm not aware of? The github comment thread is flooded with identical comments like "Thanks, that helped!", "Thanks for the tip!", and "This was the answer I was looking for."
Since they all seem positive, it doesn't seem like an attack but I thought the general etiquette for github issues was to use the emoji reactions to show support so the comment thread only contains substantive comments.
In the thread:
> It also seems that attacker is trying to stifle the discussion by spamming this with hundreds of comments. I recommend talking on hackernews if that might be the case.
These have been popping up on all the TeamPCP compromises lately
Ton of compromised accounts spamming the GH thread to prevent any substantive conversation from being had.
Oh wow. That's a lot of compromised accounts. Guess I was wrong about it not being an attack.
Those are all bots commenting, and now exposing themselves as such.
Bots to flood the discussion to prevent any actual conversation.
what's up with everyone in the issue thread thanking it, is this an irony trend or is that a flex on account takeover from teampcp? this feels wild
I will wait with updating anything until this whole trivy case gets cleaned up.
When something like this happens, do security researchers instantly contact the hosting companies to suspend or block the domains used by the attackers?
First line of defense is the git host and artifact host scrape the malware clean (in this case GitHub and Pypi).
Domains might get added to a list for things like 1.1.1.2 but as you can imagine that has much smaller coverage, not everyone uses something like this in their DNS infra.
Version 1.82.7 is also compromised. It doesn't have the pth file, but the payload is still in proxy/proxy_server.py.
I've been developing an alternative to LiteLLM. Javascript. No dependencies. https://github.com/johnhenry/ai.matey/
Stuff like is happening too much recently. Seems like the more fast paced areas of development would benefit from a paradigm shift
Move Slow and Fix Things.
title is bit misleading.
The package was directly compromised, not “by supply chain attack”.
If you use the compromised package, your supply chain is compromised.
What’s the best way to identify a compromised machine? Check uv, conda, pip, venv, etc across the filesystem? Any handy script around?
EDIT: here's what I did, would appreciate some sanity checking from someone who's more familiar with Python than I am, it's not my language of choice.
find / -name "litellm_init.pth" -type f 2>/dev/null
find / -path '/litellm-1.82..dist-info/METADATA' -exec grep -l 'Version: 1.82.[78]' {} \; 2>/dev/null
there's probably a more precise way, but if you're on uv:
That's a bad supply-chain attack, many folks use litellm as main gateway
laughs smugly in vimscript
github, pypi, npm, homebrew, cpan, etc etc. should adopt a multi-multi-factor authentication approach for releases. Maybe have it kick in as a requirement after X amount of monthly downloads.
Basically, have all releases require multi-factor auth from more than one person before they go live.
A single person being compromised either technically, or by being hit on the head with a wrench, should not be able to release something malicious that effects so many people.
And how would that work for single maintainer projects?
They would have to find someone else if they grew too big.
Though, the secondary doesn't necessarily have to be a maintainer or even a contributor on the project. It just needs to be someone else to do a sanity check, to make sure it is an actual release.
Heck, I would even say that as the project grows in popularity, the amount of people required to approve a release should go up.
So if I'm developing something I want to use and the community finds it useful but I take no contributions and no feature requests I should have to find another person to deal with?
How do I even know who to trust, and what prevents two people from conspiring together with a long con? Sounds great on the surface but I'm not sure you've thought it through.
It wouldn't prevent a project that has a goal of being purposely malicious, just from pushing out releases that aren't actually releases.
As far as who to trust, I could imagine the maintainers of different high-level projects helping each other out in this way.
Though, if you really must allow a single user to publish releases to the masses using existing shared social infrastructure. Then you could mitigate this type of attack by adding in a time delay, with the ability for users to flag. So instead of immediately going live, add in a release date, maybe even force them to mention the release date on an external system as well. The downside with that approach is that it would limit the ability to push out fixes as well.
But I think I am OK with saying if you're a solo developer, you need to bring someone else on board or host your builds yourself.
Worth exploring safeguard for some: The automatic import can be suppressed using Python interpreter’s -S option.
This would also disable site import so not viable generically for everyone without testing.
The 1.82.7 exploit was executed on import. The 1.82.8 exploit used a pth file which is run at start up (module discovery basically).
airflow, dagster, dspy, unsloth.ai, polar
Edit: ignore this silliness, as it sidesteps the real problem. Leaving it here because we shouldn't remove our own stupidity.
It's pretty disappointing that safetensors has existed for multiple years now but people are still distributing pth files. Yes it requires more code to handle the loading and saving of models, but you'd think it would be worth it to avoid situations like this.
safetensors is just as vulnerable to this sort of exploit using a pth file since it's a Python package.
Yeah, fair enough, the problem here is that the credentials were stolen, the fact that the exploit was packaged into a .pth is just an implementation detail.
teampcp taking credit?
https://github.com/krrishdholakia/blockchain/commit/556f2db3...
It's been quarantined on PyPI
We need real sandboxing. Out-of-process sandboxing, not in-process. The attacks are only going to get worse.
That's why I'm building https://github.com/kstenerud/yoloai
Wow this is in a lot of software
Yep, DSPy and CrewAI have direct dependencies on it. DSPy uses it as its primary library for calling upstream LLM providers and CrewAI falls back to it I believe if the OpenAI, Anthropic, etc. SDKs aren't available.
Our modern economy/software industry truly runs on egg-shells nowadays that engineers accounts are getting hacked to create a supply-chain attack all at the same time that threat actors are getting more advanced partially due to helps of LLM's.
First Trivy (which got compromised twice), now LiteLLM.
I work with security researchers, so we've been on this since about an hour ago. One pain I've really come to feel is the complexity of Python environments. They've always been a pain, but in an incident like this, where you need to find whether an exact version of a package has ever been installed on your machine. All I can say is good luck.
The Python ecosystem provides too many nooks and crannies for malware to hide in.
What is happening in this issue thread? Why are there 100+ satisfied slop comments?
Attackers trying to stifle discussion, they did the same for trivy: https://github.com/aquasecurity/trivy/discussions/10420
I have created an comment to hopefully steer the discussion towards hackernews if the threat actor is stifling genuine comments in github by spamming that thread with 100's of accounts
https://github.com/BerriAI/litellm/issues/24512#issuecomment...
Are they trying to slide stuff down? but it just bumps stuff up?
First thing I noticed too.
Potentially compromised?
Am I the only one having feeling that with LLM-era we have now bigger amount of malicious software lets say parsers/fetchers of credentials/ssh/private keys? And it is easier to produce them and then include in some 3rd party open-source software? Or it is just our attention gets focused on such things?
LiteLLM is now in quarantine on PyPI [1]. Looks like burning a recovery token was worth it.
[1]: https://pypi.org/project/litellm/
Someone needs to go to prison for this.
I reviewed the LiteLLM source a while back. Without wanting to be mean, it was a mess. Steered well clear.
Terrible code quality and terrible docs
LiteLLM is the second worst software project known to man. (First is LangChain. Third is OpenClaw.)
I'm sensing a pattern here, hmm.
Not familiar with LangChain besides at a surface level - what makes it the worst software project known to man?
LangChain at least has its own layer for upstream LLM provider calls, which means it isn't affected by this supply chain compromise. DSPy uses LiteLLM as its primary way to call OpenAI, etc. and CrewAI imports it, too, but I believe it prefers the vendor libraries directly before it falls back to LiteLLM.
good i was too lazy to bump versions
In case you missed it, according to the OP, the previous point release (1.82.7) is also compromised.
Yeah, that release has the base64 blob, but it didn't contain the pth file that auto triggers the malware on import.
The latest version with the the pth file doesn't require an import to trigger the exploit (just having the package installed is enough thanks to [1]).
The previous version triggers on `import litellm.proxy`
Again, all according to the issue OP.
[1] https://docs.python.org/3/library/site.html
Thank you for posting this, interesting.
I hope that everyone's course of action will be uninstalling this package permanently, and avoiding the installation of packages similar to this.
In order to reduce supply chain risk not only does a vendor (even if gratis and OS) need to be evaluated, but the advantage it provides.
Exposing yourself to supply chain risk for an HTTP server dependency is natural. But exposing yourself for is-odd, or whatever this is, is not worth it.
Remember that you are programmers and you can just program, you don't need a framework, you are already using the API of an LLM provider, don't put a hat on a hat, don't get killed for nothing.
And even if you weren't using this specific dependency, check your deps, you might have shit like this in your requirements.txt and was merely saved by chance.
An additional note is that the dev will probably post a post-mortem, what was learned, how it was fixed, maybe downplay the thing. Ignore that, the only reasonable step after this is closing a repo, but there's no incentive to do that.
> Remember that you are programmers and you can just program, you don't need a framework, you are already using the API of an LLM provider, don't put a hat on a hat, don't get killed for nothing.
Programming for different LLM APIs is a hassle, this library made it easy by making one single API you call, and in the backstage it handled all the different API calls you need for different LLM providers.
There's only two different LLM APIs in practice (Anthropic and everyone else), and the differences are cosmetic.
This is like a couple hours of work even without vibe coding tools.
Comparing this project to is-odd seems very disingenuous to me. My understanding is this was the only way you could use llama.cpp with Claude Code for example, since llama.cpp doesn't support the Anthropic compatible endpoint and doing so yourself isn't anywhere near as trivial as your comparison. Happy to be corrected if I'm wrong.
Now I feel lucky that I switched to just using OpenRouter a year ago because LiteLLM was incredible flaky and kept causing outages.