Bootstapping is so complex, here's 181-step document to compile Linux: from "commented opcodes" in hex0 to assembler with labels, from simple C compiler to ~c89 compiler to tinycc ~c99 compiler, from simple shell to scheme interpreter to Fiwix unix-like kernel.
I have been reviewing stage0 and gave a presentation, 'reviewing live-bootstrap' about it at WHY2025, in which I also made some suggestions about improving stage0. I am now working on this.
What I refer to as stage0 is the first 19 steps of the mentioned description, which actually involves the execution of over 700 executables. (See presenration for T-diagran.) I believe that number can reduced by at least a half.
Please share me the youtube video regarding the presentation for bootstrapping as I find the idea of bootstrapping really pleasant and like I am most likely sure that although I am more an arch(cachy) personal user/debian for servers users. I feel like even these are adding reproducibility and I am sure that nixos can be reproducible https://news.ycombinator.com/item?id=38057591
Guix can definitely be bootstrapped but there are comments on this post where someone tries to reproduce nix with tcc etc.
Imagining the ability that nix can be completely bootstrapped makes me feel like it can be really good for these purposes
Like if this is reproducible and bootstrappble, then I am satisfied with the hash because other people can test it out to verify they get the same hash and I can download the iso's and then verify the hash myself to find complete safety.
Although I might still be trusting my linux iso but maybe we will have risc-v hardware where I can plug in an sd card or data or similar and it can give the hash of it for the extremely paranoid (maybe universities or very secure companies perhaps?)
After that (or quite frankly just the hash part for 99% people), I can then flash the iso to any and every hardware with complete verification
Although I feel like this still won't help if the firmware or the hardware itself can't be trusted. I mean even in corebooted chromebooks, the intel TPM only stops the ability of sending the bytecode over wire but it itself really can't be stopped. Maybe we will have risc-v machines but they would rather be really expensive.
I think tho librebooted thinkpads + nixos/guix can be really great for privacy perhaps. But what are your reasons why bootstrappability fascinates you (for me its this stacking aspect and then finally we get some useful os without having to trust anything but source which we can read/heck even audit ourselves or have the shared faith that the source code is audited by security people since its completely readable)
The idea of bootstrapping is that you start with nothing, or at least with almost nothing. In case of stage0 (the first stage of live-bootstrap) this a small program that converts a file with hexadecimal characters (in pairs) to a binary file. From there you have to build everything. To compile Rust, you need a Rust or C compiler, I understand. I have no idea how hard it would be to build a minimal Rust compiler that could compile itself, one that not necessary performs all the checks that the (full) Rust compiler would do.
I think that Rust is build as part of live-bootstrap, but at a much later stage when the latest version of the GNU C compiler has been build.
There was a project [0] to write a bare bones rust compiler (no borrow checking) in C that aims to be able to build rustc in order to simplify bootstrapping, but it seems to have stalled.
Related: pnut.sh https://github.com/udem-dlteam/pnut just released. It’s a self-compiling C->POSIX shell transpiler that outputs readable shell, aimed at live-bootstrap and reproducible build chains. The shell edition is auditable and can bootstrap a native compiler from just POSIX sh + coreutils. Very much in the spirit of stage0 / compile-from-nothing work.
I understand that you mean that SectorLisp is taken as the 'seed' that we have to trust that it is not malicious (or secretly executes a malicious program if its finds it). In the live-bootstrap project it is 'hex0' that is the seed, a simple program that converts hexadecimal character (by pairs) into bytes.
So technically someone can run a c compiler written in guile in sectorlisp?
I think that the reason stage0 and other projects work this way is that they found lisp and other projects to be very limiting and there were definitely some reasons that they do things the current way that are mentioned in their docs.
In fact the MES-compiler, the one that I want to replace, is written in Scheme. It also comes with a Scheme interpreter written in C. Before the MES-compiler can be executed the source has to be unzipped and unpacked with two programs that are also compiled from C sources. So, you already need a compiler that supports a sizable subset of C to compile all these programs in order to compile the MES-compiler, which is a rather compliant C-compiler, which is used to compile the Tiny C Compiler and a standard library (included in the MES sources). The unzip and untar programs are also needed to unpack/zip the archive with the Tiny C Compiler sources.
Super fascinating stuff! I love these kind of stuff. Genuinely reading comments like yours really cheer me up in a way
I had this thought once that there would be a single sd card or hard drive which would allow bootstrapping source and a hash and with systems like tor and torrenting in general combining with self hosted wikipedia perhaps and all the mirrors of open source software I use.
I had this idea that its now possible to have such hard drives if I so desire and I am not usually paranoid but in actuality, feats like bootstrappability remove paranoia because I feel like there must be other people who do this and who verify security and I can always go to them or they exist :)
Bootstrappability is really cool and awesome stuff in my opinion which is severely underrated by many and I am glad that people like you work in helping bootstrappability. Have a nice day!
Regarding the comment itself, I didn't know that this was the case of Mes as I had just thought the only thing it depends on is scheme but thanks for sharing information about this.
The thing about Mes is that it does riscv64 too. I don't know the current state of the support, but there is something there.
There are still many pieces of riscv64 missing in the whole thing (an equivalent to Fiwix being the most challenging right now), and realistically only x86 is currently viable. I think riscv64 is the next in line though.
Stage 3 or 4 support for the other targets is also planed. This will require some small adaptations to the compiler for 32 to 64 word size and Stack-C versions for those targets.
Bootstapping is so complex, here's 181-step document to compile Linux: from "commented opcodes" in hex0 to assembler with labels, from simple C compiler to ~c89 compiler to tinycc ~c99 compiler, from simple shell to scheme interpreter to Fiwix unix-like kernel.
https://github.com/fosslinux/live-bootstrap/blob/master/part...
I have been reviewing stage0 and gave a presentation, 'reviewing live-bootstrap' about it at WHY2025, in which I also made some suggestions about improving stage0. I am now working on this.
What I refer to as stage0 is the first 19 steps of the mentioned description, which actually involves the execution of over 700 executables. (See presenration for T-diagran.) I believe that number can reduced by at least a half.
Please share me the youtube video regarding the presentation for bootstrapping as I find the idea of bootstrapping really pleasant and like I am most likely sure that although I am more an arch(cachy) personal user/debian for servers users. I feel like even these are adding reproducibility and I am sure that nixos can be reproducible https://news.ycombinator.com/item?id=38057591
Guix can definitely be bootstrapped but there are comments on this post where someone tries to reproduce nix with tcc etc.
Imagining the ability that nix can be completely bootstrapped makes me feel like it can be really good for these purposes
Like if this is reproducible and bootstrappble, then I am satisfied with the hash because other people can test it out to verify they get the same hash and I can download the iso's and then verify the hash myself to find complete safety.
Although I might still be trusting my linux iso but maybe we will have risc-v hardware where I can plug in an sd card or data or similar and it can give the hash of it for the extremely paranoid (maybe universities or very secure companies perhaps?)
After that (or quite frankly just the hash part for 99% people), I can then flash the iso to any and every hardware with complete verification
Although I feel like this still won't help if the firmware or the hardware itself can't be trusted. I mean even in corebooted chromebooks, the intel TPM only stops the ability of sending the bytecode over wire but it itself really can't be stopped. Maybe we will have risc-v machines but they would rather be really expensive.
I think tho librebooted thinkpads + nixos/guix can be really great for privacy perhaps. But what are your reasons why bootstrappability fascinates you (for me its this stacking aspect and then finally we get some useful os without having to trust anything but source which we can read/heck even audit ourselves or have the shared faith that the source code is audited by security people since its completely readable)
At WHY2025: https://www.youtube.com/watch?v=akzyyO5wvm0 with (interactive) notes: https://www.iwriteiam.nl/WHY2025_talk.html
At Hackfest Enschede 2024: https://www.youtube.com/watch?v=eKXMwVqaauc with notes: https://www.iwriteiam.nl/Hackfest_2024.html
Are they gonna keep doing this with Rust etc.?
The idea of bootstrapping is that you start with nothing, or at least with almost nothing. In case of stage0 (the first stage of live-bootstrap) this a small program that converts a file with hexadecimal characters (in pairs) to a binary file. From there you have to build everything. To compile Rust, you need a Rust or C compiler, I understand. I have no idea how hard it would be to build a minimal Rust compiler that could compile itself, one that not necessary performs all the checks that the (full) Rust compiler would do.
I think that Rust is build as part of live-bootstrap, but at a much later stage when the latest version of the GNU C compiler has been build.
There was a project [0] to write a bare bones rust compiler (no borrow checking) in C that aims to be able to build rustc in order to simplify bootstrapping, but it seems to have stalled.
[0] https://notgull.net/announcing-dozer/
You would get a C++ compiler going and then build mrustc and then use that to build rustc and then use that to build the version of rustc you want.
Related: pnut.sh https://github.com/udem-dlteam/pnut just released. It’s a self-compiling C->POSIX shell transpiler that outputs readable shell, aimed at live-bootstrap and reproducible build chains. The shell edition is auditable and can bootstrap a native compiler from just POSIX sh + coreutils. Very much in the spirit of stage0 / compile-from-nothing work.
Eliminating Mes would be amazing for us in Stagex. Mes is the second longest build in the entire tree (after rust).
At this point I believe running Common Lisp/Scheme from SectorLisp wouldn't be that far off
I understand that you mean that SectorLisp is taken as the 'seed' that we have to trust that it is not malicious (or secretly executes a malicious program if its finds it). In the live-bootstrap project it is 'hex0' that is the seed, a simple program that converts hexadecimal character (by pairs) into bytes.
So technically someone can run a c compiler written in guile in sectorlisp?
I think that the reason stage0 and other projects work this way is that they found lisp and other projects to be very limiting and there were definitely some reasons that they do things the current way that are mentioned in their docs.
In fact the MES-compiler, the one that I want to replace, is written in Scheme. It also comes with a Scheme interpreter written in C. Before the MES-compiler can be executed the source has to be unzipped and unpacked with two programs that are also compiled from C sources. So, you already need a compiler that supports a sizable subset of C to compile all these programs in order to compile the MES-compiler, which is a rather compliant C-compiler, which is used to compile the Tiny C Compiler and a standard library (included in the MES sources). The unzip and untar programs are also needed to unpack/zip the archive with the Tiny C Compiler sources.
Once everything is unpacked, are the resulting compiler materials sufficient to compile and run those packing and unpacking programs?
Super fascinating stuff! I love these kind of stuff. Genuinely reading comments like yours really cheer me up in a way
I had this thought once that there would be a single sd card or hard drive which would allow bootstrapping source and a hash and with systems like tor and torrenting in general combining with self hosted wikipedia perhaps and all the mirrors of open source software I use.
I had this idea that its now possible to have such hard drives if I so desire and I am not usually paranoid but in actuality, feats like bootstrappability remove paranoia because I feel like there must be other people who do this and who verify security and I can always go to them or they exist :)
Bootstrappability is really cool and awesome stuff in my opinion which is severely underrated by many and I am glad that people like you work in helping bootstrappability. Have a nice day!
Regarding the comment itself, I didn't know that this was the case of Mes as I had just thought the only thing it depends on is scheme but thanks for sharing information about this.
don't threaten me with a good time
The thing about Mes is that it does riscv64 too. I don't know the current state of the support, but there is something there.
There are still many pieces of riscv64 missing in the whole thing (an equivalent to Fiwix being the most challenging right now), and realistically only x86 is currently viable. I think riscv64 is the next in line though.
Stage 3 or 4 support for the other targets is also planed. This will require some small adaptations to the compiler for 32 to 64 word size and Stack-C versions for those targets.