All other arguments aside... Yes, people do review compiler output, all the time in fact!
When optimizing code it's not unusual to look at the assembly. It's not unusual to look for opportunities for autovectorization or to verify inlining or loop unrolling.
Compilers are, for the most part, deterministic. This means after people have reviewed the output, it's unlikely to change. It also means if they do change, only a few people are required to notice.
None of this applies to LLMs. They are worse than compilers, in regards to the quality and characteristics of their output, in every possible way.
Not really, people that argue about assembly on Godbolt tend to be the specimens that the site was created for, those that count CPU cycles per Assembly instrution and are religious about which programming language syntax generates the less amount of Assembly opcodes.
The rest of us uses it because it is cool way to share code snippets.
> Compilers have type systems, formal contracts about what code means before it runs.
This is a complete misunderstanding of what makes compilers trustworthy. Those are all properties of the language, not the compiler. The compiler is trustworthy to the extent that it is well built, internally. It is trustworthy to the extent that the mapping from source code to machine code is well defined, and implemented correctly.
You can have the best type system you want, but if the compiler is badly implemented, it won't be trustworthy. A perfect example is C - a language that barely has a type system, yet has some of the most trustworthy and optimized compilers. And it also has, or at least had, plenty of buggy compilers, typically for small embedded platforms with complicated mappings between C constructs and the limited CPU instruction set.
There are formally compilers (e.g. CompCert https://en.wikipedia.org/wiki/CompCert ) which are formally proven correct. I think eventually all production compilers will be formally verified.
Biggest problem right now is that we have teams pushing dozen of MD files, dozend of thousands of lines of English prose as if they were specs. There’s no way we can validate so much written prose because it’s plain English: perhaps one sentence is written in a way LLMs will read between the lines; perhaps line 100 and line 1000 are contradictory in such a subtle way that LLMs may not be aware of it.
In my company we have so much english prose committed to MD files that Im starting to think it’s all just snake oil. I cannot trust an engineer that writes “no bugs , please” and can go on with their lives.
That's why you should just subscribe to multiple LLM vendors. One model to write specs, one to write code against the specs and another to validate the code. Problem solved. (I have heard this proposed at work.)
First you need to write these specifications and if you say just tell the llm to write them - then how would it be different from just tell the llm to write the program?
I guess you can argue that these are two independent processes so you can combine them to get something more reliable than both - this might be a viable path. But from what I heard writing formal specifications is just really hard - I haven't seen anything practical in this area.
Several days ago I was toying with Rust trying to hack mmtc[0]. Mind you, I have absolutely no idea about Rust, but thanks to the verbosity and helpfulness of its compiler I was able to add a shuffle playlist feature - and am trying to figure out how to add an 'update database' feature too. At some point I even thought learning Rust could be easy...
This is a very bad comparison. You can see that immediately if you think to yourself "can I see a future in which we compile code with LLMs?". The answer is no. That's a terrible idea, and the fact that it's a terrible idea is obvious.
The formal foundations of compilers are completely different from the formal foundations of LLMs.
The former are deterministic, easy to formally verify, and extremely simple in nature. "Translate a for-loop into x86 instructions using a set of rules."
The latter is intrinsically statistical in nature. "Translate a human language prompt into functional code" has to infer the correct output statistically from similar, observed input->output relationships. There is no guarantee of consistency. Different builds of the model will see different input->output evidence, in different order, and parameter tuning will further change how it responds to those pieces of evidence. Evidence is incomplete. Local minima are inevitable. LLMs are lossy curve-fitters under the hood. Errors aren't an option, they're an inevitability.
I only skimmed this. Lots of “not to be read, but to be verified; process, not the artifact; not x but...”.
“AI-checks-AI pipelines as first-class CI infrastructure, not bolt-on curiosity”—what’s the contrast here? Is it serious aspiration, not unserious aspiration?
“Formal specification layers that agents execute against, not just prompts”—Okay.
It just looks like it is stating lots of problems with a x-not-y as if there is progress being made by way of insistence.
I am open to the idea of something like a small verification kernel that can be comprehended by “humans” which can check GenAI output. But right now we can contrast mature (decade+) compilers with GenAI like this.
- Compilers: You get the abstraction you asked for: it might not be “optimal” code, but it is code that works the way you wrote it
- GenAI: Here is 200KLOC, good luck, could be anything
Now you could reduce the space of those 200KLOC with tests and verification. But so far (based on this submission) it looks like this is at the handwaving stage.
Certainly you would need high-value tests if tests are the thing that is supposed to be the verification. Either something simple and expressive enough for “humans” to write or something that is both short and easy to read for “humans” (and generated by GenAI). Not some copy-paste smelling mockfest that looks like it is a pile of junk that has evolved over five years, each author pushing some junk on top while taking care to not make the whole pile tilt and collapse.
Just like LLMs, compilers are just another layer of abstraction and no they're not deterministic.
Just yesterday I've reported a codegen bug in MSVC. (Luckily they've fixed it very fast.)
Can you realise that it's an optimiser bug without inspecting the assembly? Hardly.
All the arguments people claim against LLMs are similarly applicable to compilers, but compilers are old technology and LLMs are new.
If you're an expert, just about every compiled function contains obvious inefficiencies, and a skilled assembly programmer can speed it up by in the ballpark of 3x. If we're talking about your average webapp, you can usually get 1000x better resource usage in most ways, including CPU, RAM, storage and so on.
And the output isn't deterministic either - the bugs no withstanding, code generation is highly chaotic, optimisations have non-local impacts and you can't easily predict optimised codegen output from source.
LLMs aren't much worse. They have non-deterministic output, but you can steer it - similarly to a compiler. An expert can use it to gain great speed and efficiency, but in the hands of someone not as capable, you can make something awful just as fast. Both tools are force multipliers.
All other arguments aside... Yes, people do review compiler output, all the time in fact!
When optimizing code it's not unusual to look at the assembly. It's not unusual to look for opportunities for autovectorization or to verify inlining or loop unrolling.
Compilers are, for the most part, deterministic. This means after people have reviewed the output, it's unlikely to change. It also means if they do change, only a few people are required to notice.
None of this applies to LLMs. They are worse than compilers, in regards to the quality and characteristics of their output, in every possible way.
If no one reviewed compiler output then https://godbolt.org/ wouldn't exist.
Same goes for linker output, especially in embedded dev.
Not really, people that argue about assembly on Godbolt tend to be the specimens that the site was created for, those that count CPU cycles per Assembly instrution and are religious about which programming language syntax generates the less amount of Assembly opcodes.
The rest of us uses it because it is cool way to share code snippets.
All other arguments aside... Yes, people do review compiler output, all the time in fact!
No. In reality, this is almost never done anymore.
We used to do it all the time back when performance mattered, but that was then.
HN readers don't have to like it, and obviously they (we) don't, but shooting the messenger won't help.
In my circles we're actually doing it even more because we can have an LLM take a look at the assembly.
Your circles are... rarefied.
> Compilers have type systems, formal contracts about what code means before it runs.
This is a complete misunderstanding of what makes compilers trustworthy. Those are all properties of the language, not the compiler. The compiler is trustworthy to the extent that it is well built, internally. It is trustworthy to the extent that the mapping from source code to machine code is well defined, and implemented correctly.
You can have the best type system you want, but if the compiler is badly implemented, it won't be trustworthy. A perfect example is C - a language that barely has a type system, yet has some of the most trustworthy and optimized compilers. And it also has, or at least had, plenty of buggy compilers, typically for small embedded platforms with complicated mappings between C constructs and the limited CPU instruction set.
There are formally compilers (e.g. CompCert https://en.wikipedia.org/wiki/CompCert ) which are formally proven correct. I think eventually all production compilers will be formally verified.
Biggest problem right now is that we have teams pushing dozen of MD files, dozend of thousands of lines of English prose as if they were specs. There’s no way we can validate so much written prose because it’s plain English: perhaps one sentence is written in a way LLMs will read between the lines; perhaps line 100 and line 1000 are contradictory in such a subtle way that LLMs may not be aware of it.
In my company we have so much english prose committed to MD files that Im starting to think it’s all just snake oil. I cannot trust an engineer that writes “no bugs , please” and can go on with their lives.
""" we need to build:
"""It is probably easier to just write that program.
yep. "Formal specification layers " aka code.
Talked with someone this morning who is using "formal methods" to validate their AI generated code.
They are using the same AI to generate the proofs.
That's why you should just subscribe to multiple LLM vendors. One model to write specs, one to write code against the specs and another to validate the code. Problem solved. (I have heard this proposed at work.)
Right, because to trust that those "formal specifications" are correct, you will have to write them by hand.
First you need to write these specifications and if you say just tell the llm to write them - then how would it be different from just tell the llm to write the program?
I guess you can argue that these are two independent processes so you can combine them to get something more reliable than both - this might be a viable path. But from what I heard writing formal specifications is just really hard - I haven't seen anything practical in this area.
Several days ago I was toying with Rust trying to hack mmtc[0]. Mind you, I have absolutely no idea about Rust, but thanks to the verbosity and helpfulness of its compiler I was able to add a shuffle playlist feature - and am trying to figure out how to add an 'update database' feature too. At some point I even thought learning Rust could be easy...
[0] https://github.com/figsoda/mmtc/
Some of us do spend hours on godbolt.org tweaking code like it was game character build.
This is a very bad comparison. You can see that immediately if you think to yourself "can I see a future in which we compile code with LLMs?". The answer is no. That's a terrible idea, and the fact that it's a terrible idea is obvious.
The formal foundations of compilers are completely different from the formal foundations of LLMs.
The former are deterministic, easy to formally verify, and extremely simple in nature. "Translate a for-loop into x86 instructions using a set of rules."
The latter is intrinsically statistical in nature. "Translate a human language prompt into functional code" has to infer the correct output statistically from similar, observed input->output relationships. There is no guarantee of consistency. Different builds of the model will see different input->output evidence, in different order, and parameter tuning will further change how it responds to those pieces of evidence. Evidence is incomplete. Local minima are inevitable. LLMs are lossy curve-fitters under the hood. Errors aren't an option, they're an inevitability.
This is why I use LLMs to write directly in Assembly, making it impossible for me to review it. (joke)
Compilers are (mostly) deterministic. LLMs are explicitly stochastic.
This is so dumb.
ah yes.
Lets indeed treat non-deterministic output exactly like we treat deterministic output.
We could make it deterministic
I only skimmed this. Lots of “not to be read, but to be verified; process, not the artifact; not x but...”.
“AI-checks-AI pipelines as first-class CI infrastructure, not bolt-on curiosity”—what’s the contrast here? Is it serious aspiration, not unserious aspiration?
“Formal specification layers that agents execute against, not just prompts”—Okay.
It just looks like it is stating lots of problems with a x-not-y as if there is progress being made by way of insistence.
I am open to the idea of something like a small verification kernel that can be comprehended by “humans” which can check GenAI output. But right now we can contrast mature (decade+) compilers with GenAI like this.
- Compilers: You get the abstraction you asked for: it might not be “optimal” code, but it is code that works the way you wrote it
- GenAI: Here is 200KLOC, good luck, could be anything
Now you could reduce the space of those 200KLOC with tests and verification. But so far (based on this submission) it looks like this is at the handwaving stage.
Certainly you would need high-value tests if tests are the thing that is supposed to be the verification. Either something simple and expressive enough for “humans” to write or something that is both short and easy to read for “humans” (and generated by GenAI). Not some copy-paste smelling mockfest that looks like it is a pile of junk that has evolved over five years, each author pushing some junk on top while taking care to not make the whole pile tilt and collapse.
this take is peak dunning-kruger:
https://github.com/llvm/llvm-project/tree/main/llvm/test/Cod...
Just like LLMs, compilers are just another layer of abstraction and no they're not deterministic.
Just yesterday I've reported a codegen bug in MSVC. (Luckily they've fixed it very fast.) Can you realise that it's an optimiser bug without inspecting the assembly? Hardly.
All the arguments people claim against LLMs are similarly applicable to compilers, but compilers are old technology and LLMs are new.
If you're an expert, just about every compiled function contains obvious inefficiencies, and a skilled assembly programmer can speed it up by in the ballpark of 3x. If we're talking about your average webapp, you can usually get 1000x better resource usage in most ways, including CPU, RAM, storage and so on.
And the output isn't deterministic either - the bugs no withstanding, code generation is highly chaotic, optimisations have non-local impacts and you can't easily predict optimised codegen output from source.
LLMs aren't much worse. They have non-deterministic output, but you can steer it - similarly to a compiler. An expert can use it to gain great speed and efficiency, but in the hands of someone not as capable, you can make something awful just as fast. Both tools are force multipliers.