Does no one know what exit codes and stderr vs. stdout is anymore?
I work with old and new code bases used by many clients in complicated setups, but adding a warning to stderr while stdout was left untouched, and proper exit codes maintained, was hardly, if ever, a problem so far.
Of course, there's always some unpleasant exception, but it's rare.
And of course, I also understand that the author might have found themselves not only in one of those rare-ish instances, but also one where reasoning with the other side was fruitless.
I worked on a code base that ran on a variety of Unixes. One customer complained about file corruption from our software. Turns out that their Unix setup was... rather unique. Spawned programs were not given a stderr file handle! If you wrote to stderr, it scribbled on whatever file was open on file handle 2.
We, uh, reasoned with the other side - we told them to fix their stupid broken setup.
It's POSIX convention to write to stderr for anything that's not strict program output. I have seen 2>&1 far too often in scripts. I don't worry about it and happily write error messages to stderr whenever my scripts exit without a 0 status code.
Another thing that gets screwed up a lot is: Comand line usage help/information should be printed to stderr if it was invoked because the user passed an invalid option to the command line (in other words, the usage text counts as diagnostic[1] info), but it should be printed to stdout if the user invoked the application with -h, --help or similar. Reasons:
1. If you mess up the command line to the program in a script or pipe, and get a bunch of usage output in stdout, a downstream consumer of that stdout might think its legit program output and try to parse it.
2. If your user actually calls the program with -h or --help, they might want to |less through it to read it on a small terminal screen. Output that to stdout.
3. Generally, you can always tell if something is going wrong by grepping for errors or warnings a single stream (stderr), or by looking for a nonzero exit code.
But your general principle applies: Output expected by the user -> stdout. Output incidental to the program's operation or errors -> stderr.
Disagree. stdout is only reserved for actual processed command output. It may be empty, it may be invalid because the input was invalid (shit in, shit out), but it may never be things intended for a human to read.
If one wants to use a pager (like I sometimes do, though most of the time I just scroll up), they'll just use `foo 2>&1 | less`.
I guess this is a point where I disagree with GNU. Then again, the reasons why I disagree are subtle enough that I don't care too much... I think this particular thing doesn't matter very much in practice, so go do your thing GNU...
Indeed you're supposed to, but that way if someone calls exit(0), it looks like the program worked fine, when in fact they committed some debug code and made the program no longer run to completion. "Yay, done" was put in for the scripts to flag this sort of thing, presumably based on experience.
"People deliberately misuse the very mechanism that was designed to indicate successful completion, so we added another, flawed detection mechanism based on IO, because there's no way they'll do the thing that they should have learned in the first year of school, and just call exit with any other argument than 0 on irregular completion".
Gosh I thought the engineering culture was bad where I work.
Yea, whatever you do, don't solve the actual problem! This reminds me of solving a buffer overflow by just blindly increasing the size of the buffer until it no longer crashes.
Next thing you know, you'll have to skip a major version number in your product because a significant fraction of your user base string parse its name to determine what version it is.
I can say up until 2005 or so I was a real believer in printf() debugging but I deliberately switched to using a debugger as much as possible around that time. I found that no matter how hard people "try" if they modifying the code to do debugging there is some chance these get checked in -- whereas you can investigate many things with the debugger without checking anything it.
Some applications have more trouble with setup and teardown than others. Like I knew a professor who kept sending me C programs that would crash before main() and some systems have a lot of trouble with "crash on shutdown" which might be a problem (corrupted files) or a non-problem.
> I was a real believer in printf() debugging but I deliberately switched to using a debugger
This really does not need to be an either/or. They have different uses. You can stick in 20 printfs and get a quick feel for where the bug is far quicker than stepping through the code - especially if you set a breakpoint and hit run, only to realise that you've overshot. You can run the program 10 times with different parameters and compare the results with printf much more easily than you could with a debugger. But, once you've found the rough area, a debugger is much better for fine grained inspection, and especially interrogating state with carefully written watches.
I do get your point about the risk of leaving in some trace by accident. But it feels like overkill to throw away such a valuable tool just because of that.
We still seem to have fairly bad tooling for advanced debugging use cases.
There's no good reason you shouldn't be able to have an IDE maintain a text overlay of debugging points which is solely supplied as breakpoint scripts to the debugger instead.
Then that's utterly and horribly wrong? And in my decades of experience, I've fortunately only relatively rarely had to deal with that particular kind of atrocity.
But the whole idea is that a warning is a warning. Solving a warning can be deferred, and a warning doesn't cause execution to fail. Your warning was transmuting itself into an error. I feel like "All means are fair except solving the problem" is the wrong conclusion to draw here. If it should have been solved immediately, it should have been an error in the first place. (And then you should have politely bumped the version so that you don't immediately break the code of all your dependents.) If there is no need to solve it immediately, then "all means are fair" to convert it back to a warning as was originally intended.
The warning was only "transmuting" itself into an error because another team took a dependency in their test on the exact ordering of writes of certain data to a globally shared resource.
> Instead, they were quick to point out that it’s hard to know where these warnings could come from, and we cannot risk all those critical workflows failing when some case of misuse surfaces in a new context.
Ah yes, Schroedinger's workflow. So important any disruption is a disaster, and simultaneously so unimportant they couldn't possibly spend a single dime on the tools critical to the workflow.
I'm sorry, but "they committed some debug code that made the program exit(0)" for me is pretty much equivalent to "they committed some debug code that made the program not work correctly".
Does no one know what exit codes and stderr vs. stdout is anymore?
I work with old and new code bases used by many clients in complicated setups, but adding a warning to stderr while stdout was left untouched, and proper exit codes maintained, was hardly, if ever, a problem so far.
Of course, there's always some unpleasant exception, but it's rare.
And of course, I also understand that the author might have found themselves not only in one of those rare-ish instances, but also one where reasoning with the other side was fruitless.
Yes, this was also exactly what I have thought while reading TFA.
I worked on a code base that ran on a variety of Unixes. One customer complained about file corruption from our software. Turns out that their Unix setup was... rather unique. Spawned programs were not given a stderr file handle! If you wrote to stderr, it scribbled on whatever file was open on file handle 2.
We, uh, reasoned with the other side - we told them to fix their stupid broken setup.
It's POSIX convention to write to stderr for anything that's not strict program output. I have seen 2>&1 far too often in scripts. I don't worry about it and happily write error messages to stderr whenever my scripts exit without a 0 status code.
Another thing that gets screwed up a lot is: Comand line usage help/information should be printed to stderr if it was invoked because the user passed an invalid option to the command line (in other words, the usage text counts as diagnostic[1] info), but it should be printed to stdout if the user invoked the application with -h, --help or similar. Reasons:
1. If you mess up the command line to the program in a script or pipe, and get a bunch of usage output in stdout, a downstream consumer of that stdout might think its legit program output and try to parse it.
2. If your user actually calls the program with -h or --help, they might want to |less through it to read it on a small terminal screen. Output that to stdout.
3. Generally, you can always tell if something is going wrong by grepping for errors or warnings a single stream (stderr), or by looking for a nonzero exit code.
But your general principle applies: Output expected by the user -> stdout. Output incidental to the program's operation or errors -> stderr.
1: https://pubs.opengroup.org/onlinepubs/9799919799/functions/s...
Disagree. stdout is only reserved for actual processed command output. It may be empty, it may be invalid because the input was invalid (shit in, shit out), but it may never be things intended for a human to read.
If one wants to use a pager (like I sometimes do, though most of the time I just scroll up), they'll just use `foo 2>&1 | less`.
GNU offers guidance[1] to output --help to stdout, but it's not against the law to do it your way.
1: https://www.gnu.org/prep/standards/html_node/_002d_002dhelp....
I guess this is a point where I disagree with GNU. Then again, the reasons why I disagree are subtle enough that I don't care too much... I think this particular thing doesn't matter very much in practice, so go do your thing GNU...
Out of the problems we all agonize over, this is probably a 1 out of 10!
I see you've never tried to run command --help | less
I just said that I always do `command --help 2>&1 | less`?
Aren't you supposed to return a 0 status code when "yea done!" and some other status code when it wasn't done?
Indeed you're supposed to, but that way if someone calls exit(0), it looks like the program worked fine, when in fact they committed some debug code and made the program no longer run to completion. "Yay, done" was put in for the scripts to flag this sort of thing, presumably based on experience.
"People deliberately misuse the very mechanism that was designed to indicate successful completion, so we added another, flawed detection mechanism based on IO, because there's no way they'll do the thing that they should have learned in the first year of school, and just call exit with any other argument than 0 on irregular completion".
Gosh I thought the engineering culture was bad where I work.
Yea, whatever you do, don't solve the actual problem! This reminds me of solving a buffer overflow by just blindly increasing the size of the buffer until it no longer crashes.
Brother, have I got bad news for you about all the places outside your door.
Next thing you know, you'll have to skip a major version number in your product because a significant fraction of your user base string parse its name to determine what version it is.
I can say up until 2005 or so I was a real believer in printf() debugging but I deliberately switched to using a debugger as much as possible around that time. I found that no matter how hard people "try" if they modifying the code to do debugging there is some chance these get checked in -- whereas you can investigate many things with the debugger without checking anything it.
Some applications have more trouble with setup and teardown than others. Like I knew a professor who kept sending me C programs that would crash before main() and some systems have a lot of trouble with "crash on shutdown" which might be a problem (corrupted files) or a non-problem.
Just make it fprintf(stderr, ...) debugging instead.
> I was a real believer in printf() debugging but I deliberately switched to using a debugger
This really does not need to be an either/or. They have different uses. You can stick in 20 printfs and get a quick feel for where the bug is far quicker than stepping through the code - especially if you set a breakpoint and hit run, only to realise that you've overshot. You can run the program 10 times with different parameters and compare the results with printf much more easily than you could with a debugger. But, once you've found the rough area, a debugger is much better for fine grained inspection, and especially interrogating state with carefully written watches.
I do get your point about the risk of leaving in some trace by accident. But it feels like overkill to throw away such a valuable tool just because of that.
We still seem to have fairly bad tooling for advanced debugging use cases.
There's no good reason you shouldn't be able to have an IDE maintain a text overlay of debugging points which is solely supplied as breakpoint scripts to the debugger instead.
IDEs seem to conk out at click to set breakpoint.
Then that's utterly and horribly wrong? And in my decades of experience, I've fortunately only relatively rarely had to deal with that particular kind of atrocity.
Backwards compatibility is golden.
But the whole idea is that a warning is a warning. Solving a warning can be deferred, and a warning doesn't cause execution to fail. Your warning was transmuting itself into an error. I feel like "All means are fair except solving the problem" is the wrong conclusion to draw here. If it should have been solved immediately, it should have been an error in the first place. (And then you should have politely bumped the version so that you don't immediately break the code of all your dependents.) If there is no need to solve it immediately, then "all means are fair" to convert it back to a warning as was originally intended.
The warning was only "transmuting" itself into an error because another team took a dependency in their test on the exact ordering of writes of certain data to a globally shared resource.
I have seen this kind of thing go so many ways.
- sometimes you can get the status code, sometimes you can't.
- sometimes you can separate out stdout from stderr, sometimes you can't
- sometimes the program generating the error message identifies itself, sometimes it doesn't
- sometimes you don't know if you have a "good error" (ok to ignore) or a "bad error" (cannot ignore)
I am a fan of the HARD FAIL.
I think internal unit tests or things like that should hard fail, then get a human to either fix it, or put in a hard exception.
if it is user-facing... sigh
> Instead, they were quick to point out that it’s hard to know where these warnings could come from, and we cannot risk all those critical workflows failing when some case of misuse surfaces in a new context.
Ah yes, Schroedinger's workflow. So important any disruption is a disaster, and simultaneously so unimportant they couldn't possibly spend a single dime on the tools critical to the workflow.
Why not have the program touch a file on success?
Why not have the program exit with exit code 0 on success, and non-0 on failure, like almost every program in a POSIX environment ever?
see yosefk reply above
I'm sorry, but "they committed some debug code that made the program exit(0)" for me is pretty much equivalent to "they committed some debug code that made the program not work correctly".
Was this FOSS or commercial?
If it's commercial software, you're paid to make it work, no matter how stupid that may be.
If it's FOSS, you can tell the user to deal with it and close the ticket.