> Half the kernel is still built by individuals: people using gmail.com, personal domains, or university emails. The "corporate takeover" narrative is overstated. Companies contribute heavily, but the kernel remains a genuinely collaborative project.
Isn't the assumption here flawed? Someone may be employed by a corporation but still use their gmail/personal domain/university domain. This needs to be cross-correlated against some secondary source of employment data to give a more accurate picture.
That's an assumption in the exact opposite direction. GP is assuming that if someone commits while employed by a company then that company paid completely for that commit, while you're assuming that in that case the company "probably" didn't pay for the whole commit.
Either way, the article's conclusion seems to be insufficiently supported.
I actually specifically didn’t state it one way or the other, just highlighting the potential for undercounting. I think there’s some potential for overcounting if you assume employer paid for commits, but less so if you constrain to those employed to work on the Linux kernel - I don’t know if many people would be working “spare time” on the same thing they’re getting paid for.
These smell like the kind of metrics that cause someone to feel informed and then to miss the forest for the trees. The kind of data for a "data driven" decision maker who will just invent a narrative to explain the numbers, and then do what they wanted to do all along.
> These smell like the kind of metrics that cause someone to feel informed and then to miss the forest for the trees. The kind of data for a "data driven" decision maker who will just invent a narrative to explain the numbers, and then do what they wanted to do all along.
> We need to increase reliability in the kernel, so the kernel team should fire the top 5 bug-introducers, to reduce the amount of bugs being introduced (https://pebblebed.com/blog/kernel-bugs-part2/05_author_analy...). Linus has got to go.
You've cut bugs being introduced while also reducing development costs by slashing team size. You deserve a promotion and an increase in equity.
117 people meet this criteria. And the impact is dramatic:
It’s strange to me to think of “bugfixes” in terms of a commodity. Different problem spaces between subsystems and thus different types of (and surfaces for) bugs; different contributor mixes; different number of eyes on them; different potential impacts…
> CAN bus drivers top the list [of bug lifetime by subsystem]. These are used in automotive and industrial systems. Critical infrastructure with few maintainers watching.
…or maybe higher-quality initial submissions, with most of the easy bugs already wrung out of them, so only subtle bugs remain (thus fewer to fix).
Or adequately vigilant maintainers but low diversity of systems running that code, thus fewer users/situations where the bugs manifest, so they go unreported. Or poorer telemetry so an ordinary rate of latent bugs but they go undetected.
Could be any, probably a little of all, can’t really tell from the analysis; and each cause would suggest a different response to improve quality.
I’d expect an LLM to use correct grammatical number in the first sentence that you quoted. As written, the demonstrative ‘this’ has singular number but ‘criteria’ plural.
This reads like Claude wrote it (more than ChatGPT.) Interesting data but I am unsure how actionable it is. Are they suggesting, for example, that specific commit messages get scanner more closely? Why is CAN more severe than Intel? (It does worry me. I feel like bugs, of any sort, in car systems are terrifying.)
I suspect the usage of the CAN driver in Linux is pretty low. The largest user of the Linux can driver is likely testing and diagnostics tooling for developing cars rather than the car themselves. Even when the car has a Linux computer, they often use multi CPU SOC's that run the real-time CAN logic separate from Linux, and only convey application logic into Linux.
I could also speculate that the overlap between Linux kernel developers and automotive and industrial embedded systems is pretty low. So the high bug severity in the CAN driver could be developers contributing patches from a very different programming background?
I'm not sure why this isn't included in the blog, but I was curious about the ratio between bugs and commits. Presented here are my calculations in order of total number of bugs:
Intel : 11.86%
[1] Independent : 2.27%
Red Hat : 9.74%
Linaro : 12.73%
Google : 12.78%
AMD : 9.70%
The above is based on the bug count table in the article.
This suggests that corporations are introducing significantly more bugs than independent developers. However, I have not done statistical testing on this nor have I recreated the numbers. If I had to speculate, I would assume that the analysis from the author was partly vibe-coded or they purposely left this analysis out due to fear of retaliation. Extending my speculation would also include that corporations are purposely introducing bugs out of malice such that there are backdoors available for them. The author mentions that there is no "corporate takeover" but perhaps there are more interesting conclusions to be found.
Ya, a total guess on my part is the corporations are adding more things like complete drivers and kernel modules where individuals may be adding more smaller fixes.
> Half the kernel is still built by individuals: people using gmail.com, personal domains, or university emails. The "corporate takeover" narrative is overstated. Companies contribute heavily, but the kernel remains a genuinely collaborative project.
Isn't the assumption here flawed? Someone may be employed by a corporation but still use their gmail/personal domain/university domain. This needs to be cross-correlated against some secondary source of employment data to give a more accurate picture.
I am a bit confused though here - should the kernel be written by unemployed people? Most contributors won’t have full-time funding.
That's an assumption in the exact opposite direction. GP is assuming that if someone commits while employed by a company then that company paid completely for that commit, while you're assuming that in that case the company "probably" didn't pay for the whole commit.
Either way, the article's conclusion seems to be insufficiently supported.
I actually specifically didn’t state it one way or the other, just highlighting the potential for undercounting. I think there’s some potential for overcounting if you assume employer paid for commits, but less so if you constrain to those employed to work on the Linux kernel - I don’t know if many people would be working “spare time” on the same thing they’re getting paid for.
Not happy with the lack of statistical testing, some of the smaller differences in % could probably be coincidence
These smell like the kind of metrics that cause someone to feel informed and then to miss the forest for the trees. The kind of data for a "data driven" decision maker who will just invent a narrative to explain the numbers, and then do what they wanted to do all along.
The map is not the territory.
> These smell like the kind of metrics that cause someone to feel informed and then to miss the forest for the trees. The kind of data for a "data driven" decision maker who will just invent a narrative to explain the numbers, and then do what they wanted to do all along.
We need to increase reliability in the kernel, so the kernel team should fire the top 5 bug-introducers, to reduce the amount of bugs being introduced (https://pebblebed.com/blog/kernel-bugs-part2/05_author_analy...). Linus has got to go.
> We need to increase reliability in the kernel, so the kernel team should fire the top 5 bug-introducers, to reduce the amount of bugs being introduced (https://pebblebed.com/blog/kernel-bugs-part2/05_author_analy...). Linus has got to go.
You've cut bugs being introduced while also reducing development costs by slashing team size. You deserve a promotion and an increase in equity.
The LLM-tone doesn’t help:
117 people meet this criteria. And the impact is dramatic:
It’s strange to me to think of “bugfixes” in terms of a commodity. Different problem spaces between subsystems and thus different types of (and surfaces for) bugs; different contributor mixes; different number of eyes on them; different potential impacts…
> CAN bus drivers top the list [of bug lifetime by subsystem]. These are used in automotive and industrial systems. Critical infrastructure with few maintainers watching.
…or maybe higher-quality initial submissions, with most of the easy bugs already wrung out of them, so only subtle bugs remain (thus fewer to fix).
Or adequately vigilant maintainers but low diversity of systems running that code, thus fewer users/situations where the bugs manifest, so they go unreported. Or poorer telemetry so an ordinary rate of latent bugs but they go undetected.
Could be any, probably a little of all, can’t really tell from the analysis; and each cause would suggest a different response to improve quality.
I’d expect an LLM to use correct grammatical number in the first sentence that you quoted. As written, the demonstrative ‘this’ has singular number but ‘criteria’ plural.
This reads like Claude wrote it (more than ChatGPT.) Interesting data but I am unsure how actionable it is. Are they suggesting, for example, that specific commit messages get scanner more closely? Why is CAN more severe than Intel? (It does worry me. I feel like bugs, of any sort, in car systems are terrifying.)
> CAN more severe than Intel
I suspect the usage of the CAN driver in Linux is pretty low. The largest user of the Linux can driver is likely testing and diagnostics tooling for developing cars rather than the car themselves. Even when the car has a Linux computer, they often use multi CPU SOC's that run the real-time CAN logic separate from Linux, and only convey application logic into Linux.
I could also speculate that the overlap between Linux kernel developers and automotive and industrial embedded systems is pretty low. So the high bug severity in the CAN driver could be developers contributing patches from a very different programming background?
> I could also speculate that the overlap between Linux kernel developers and automotive and industrial embedded systems is pretty low.
Agreed.
> So the high bug severity in the CAN driver could be developers contributing patches from a very different programming background?
Background and situation. Their mindset is "I need this to work, right now", not "I need this to work, and not break anything else, forever".
There are so many more embedded CAN systems beyond cars. Industrial battery management uses Linux and canbus, for example.
I'm not sure why this isn't included in the blog, but I was curious about the ratio between bugs and commits. Presented here are my calculations in order of total number of bugs:
Intel : 11.86%
[1] Independent : 2.27%
Red Hat : 9.74%
Linaro : 12.73%
Google : 12.78%
AMD : 9.70%
The above is based on the bug count table in the article.
[1] I combined the total bug count for independent and kernel.org because they are combined for the total contributions here, https://github.com/quguanni/kernel-archaeology/blob/main/scr...
This suggests that corporations are introducing significantly more bugs than independent developers. However, I have not done statistical testing on this nor have I recreated the numbers. If I had to speculate, I would assume that the analysis from the author was partly vibe-coded or they purposely left this analysis out due to fear of retaliation. Extending my speculation would also include that corporations are purposely introducing bugs out of malice such that there are backdoors available for them. The author mentions that there is no "corporate takeover" but perhaps there are more interesting conclusions to be found.
What about the complexity and type of the committed code?
Ya, a total guess on my part is the corporations are adding more things like complete drivers and kernel modules where individuals may be adding more smaller fixes.
You can feel IC fear of being roasted by linus in those numbers.
I'd also like to see this broken down for C vs Rust.
Bugs Georg, who is an outlier and should be excluded from the analysis.
Strange how someone in a cave with no internet can push 10,000 bugs a day.