So is reviewing and verifying code. Maybe not twice as "hard" if you're skilled in such things. But most programmers I've worked with can't be bothered to write tests let alone verify correctness by other means (good tests, property tests, types, model checking, etc).
It's one thing to point out small trivialities like initialization and life time issues in a small piece of code. But it's quite another to prove they don't exist in a large code base.
Kernigan is a good source of quotes and thinking on programming.
I am fascinated by the prevalence of wanting "tests" from hacker news comments. Most of the code I have worked on in the past 20 years did not have tests. Most of it was shopping carts, custom data transformation code, orchestrating servers, plugin code functionality to change some aspect of a website.
Now, I have had to do some salesforce apex coding and the framework requires tests. So I write up some dummy data of a user and a lead and pass it through the code, but it feels of limited value, almost like just additional ceremony. Most of the bugs I see are from a misconception of different users about what a flag means. I can not think of a time a test caught something.
The organization is huge and people do not go and run all the code every time some other area of the system is changed. Maybe they should? But I doubt that would ever happen given the politics of the organization.
So I am curious, what are the kinds of tests do people write in other areas of the industry?
what are the kinds of tests do people write in other areas of the industry?
Aerospace here. Roughly this would be typical:
- comprehensive requirements on the software behavior, with tests to verify those requirements. Tests are automated as much as possible (e.g., scripts rather than manual testing)
- tests are generally run first in a test suite in a completely virtual software environment
- structural coverage analysis (depending on level of criticality) to show that all code in the subsystem was executed by the testing (or adequately explain why the testing can't hit that code)
- then once that passes, run the same tests in a hardware lab environment, testing the software as it runs on the the actual physical component that will be installed on the plane
- then test that actually on a plane, through a series of flight tests. (The flight testing would likely not be as entirely comprehensive as the previous steps)
A full round of testing is very time-consuming and expensive, and as much as possible should be caught and fixed in the virtual software tests before it even gets to the hardware lab, much less to the plane.
Per corporate policy -- in the name of safety and legal reasons -- the most extensive public uses of AI, like vibe coding and agentic systems, aren't really options. The most common usage I have seen is more like consulting AI as a fancy StackOverflow.
Will this change? I personally don't expect to ever see pure vibe coding, with code unseen and unreviewed, but I imagine AI coding uses will expand.
The interesting bit is this: how much of what you wrote over all those years actually did what you wanted it to do, no more, no less?
This is where testing gets interesting: I took some old code I wrote 30 years ago or so and decided to put it literally to the test. A couple of hundred lines from a library that has been in production without every showing a single bug over all that time. And yet: I'm almost ashamed at how many subtle little bugs I found. Things you'd most likely never see in practice, but still, they were there. And then I put a couple of those bugs together and suddenly realized that that particular chain must have happened in practice in some program built on top of this. And sure enough: fixing the bugs made the application built on top of this more robust.
After a couple of weeks of this I became convinced: testing is not optional, even for stuff that works. Ever since I've done my best to stop assuming that what I'm writing actually does what I want it to. It usually does, for the happy path. But there are so many other paths that with code of any complexity, even if you religiously avoid side effects you can still end up with issues that you overlook.
The value of tests is when the fail they show you of something you broke that you didn't realize. 80% (likely more, but I don't know how to measure) of the tests I write could safely be thrown away because they fail again - but I don't know which tests will fail and thus inform me that I broke things.
The system I'm working on has been in production for 12 years - we have added a lot of new features over those years. Many of those needed us to hook into existing code, tests help us know that we didn't break something that used to work.
Maybe that helps answer the question of why they are important to me. They might not be to your problems.
How do you know you haven't unknowingly broken something when you made a change?
I think if:
- the code base implements many code paths depending on options and user inputs and options such that a fix for code path A may break code path B
- it takes a great deal of time to run in production such that issues may only be caught weeks or months down the line when it becomes difficult to pinpoint their cause (not all software is real-time or web)
- any given developer does not have it all in their head such that they can anticipate issues codebase wide
then it becomes useful to have (automated) testing that checks a change in function A didn't break functionality in function B that relies on A in some way(s), that are just thorough enough that they catch edge cases, but don't take prod levels of resources to run.
Now I agree some things might not need testing beyond implementation. Things that don't depend on other program behavior, or that check their inputs thoroughly, and are never touched again once merged, don't really justify keeping unit tests around. But I'm not sure these are ever guarantees (especially the never touched again).
I think the whole concept of testing confuses a lot of people. I know I was (and still sometimes am) confused about the various "best practices" and opinions around testing. As as well as how/what/when to test.
For my projects, I mainly want to Get Shit Done. So I write tests for the major functional areas of the business logic, mainly because I want to know ASAP when I accidentally break something important. When a bug is found that a test didn't catch, that's usually an indicator that I forgot a test, or need to beef up that area of functional testing.
I do not bother with TDD, or tests that would only catch cosmetic issues, and I avoid writing tests that only actually test some major dependency (like an ORM).
If the organization you are in does not value testing, you are probably not going to change their mind. But if you have the freedom to write worthwhile tests for your contributions to the code, doing so will probably make you a better developer.
Tests are important. Writing good tests even moreso. However, you can't do either of you don't have good product requirements and communication.
Most software has "bugs" simply because people couldn't communicate how it should work.
I think most programmers are on top of actual runtime errors or compilation errors. It's straightforward on how to fix those. They are not on top of logic issues or unintended behavior because they aren't the product designer.
Programmers just cook the burger. If you order it well done, don't complain when it doesn't come out medium rare.
I do test manually in salesforce. Mainly its because you do not control everything and I find the best test is to log in as the user and go through the screens as they do. I built up some selenium scripts to do testing.
In old days, for the kinds of things I had to work on, I would test manually. Usually it is a piece of code that acts as glue to transform multiple data sources in different formats into a database to be used by another piece of code.
Or a aws lambda that had to ingest a json and make a determination about what to do, send an email, change a flag, that sort of thing.
Not saying mock testing is bad. Just seems like overkill for the kinds of things I worked on.
Not who you asked but I think it comes down to risk/reward. The consequences of some user finding a big in most websites is low, compared to the risk of an astronaut finding a bug the hard way whilst attempting re-entry.
There is genuinely a reasonable and rational argument to “testing requires more effort than fixing the issues as users find them” if the consequences are low. See video games being notorious for this.
So, industry is more important than language I’d say.
I don't see testing as a quality thing any more, I see it as a developer productivity thing.
If my project has tests I can work so much faster on it, because I can confidently add tests and refactor and know that I didn't break existing functionality.
You gotta pay that initial cost to to get the framework in place though. That takes early discipline.
Depends again on the type of work, and it absolutely has something to do with quality. You’re caring about not breaking existing functionality, you’re refactoring, you’re moving forward with confidence in your code.
That’s absolutely a quality thing. I can assure you that you could move a lot faster if you didn’t try and meet such standards, not that it’d be a good idea necessarily, but in isolation it proves the point.
Developer testing is checking whether the code does what the developer themself thinks it should. QA testing is checking whether the code does what the customers / users / rest of the world thinks it should.
It’s a lot faster and easier than it used to be. Things like xUnit in the .net world make setting up tests friction free to the point where I question a codebase that doesn’t have some kind of basic unit tests. It doesn’t make mock testing or integration testing easier but I would argue if you know the base code and logic is sound those tests are less relevant.
One thing I found is that if testing is easy, your code structure does change a bit to aid with a “test first” approach and I don’t hate it. I thought it made me slower but it doesn’t, it ensures that when all the ground work is finished, the gnarly part of wiring everything up goes much faster.
Yeah I've found the same, having good test discipline influences my code design in a positive way because code that's easier to test is also code that's easier to integrate and understand.
I consider a lot of my cleverness when writing code is aimed at achieving functionality, performance, conciseness, etc., while being easy to understand and self-debugged by design. The latter meaning:
• Types enforce safety
• Very tight dependencies
• A tight design (value, store, control flows) where most bugs are likely to be catastrophic, or at least highly visible, as apposed to silent. It works, or it doesn't.
• Use naming and code organization first, concise comments second, and a page or two of doc if all else fails, to make any non-intuitive optimization or operation intuitive again. (Clarify as needed: what it does, how it does it, and why that specific method was chosen.)
For me, a bug repellent design is more important than tests, during development. Except where there is truly messy functionality that no implementation can un-mess. Or results are not used in any way that creates implicit tests them. I.e. they are an intermediate solution to a downstream problem that is not part of the development context.
That seems so basic to writing the "most cleverest" code, that the quote makes little sense to me.
I spend a very small percentage of my time debugging, outside of the real-time development-time write, run, correct loop. If debugging ever took a significant fraction of time, it would give me some serious anxiety.
But as the article states, that may be the result of learning from my less manageable cleverness on ambitious personal projects, over and over, early on. Endless personal projects, in which any success immediately brings into view "the next exciting level" they could become, often requiring a redesign, are great for forcing growth.
Interesting, I actually find LLMs very useful at debugging. They are good at doing mindless grunt work and a great deal of debugging in my case is going through APIs and figuring out which of the many layers of abstraction ended up passing some wrong argument into a method call because of some misinterpretation of the documentation.
Claude Code can do this in the background tirelessly while I can personally focus more on tasks that aren't so "grindy".
They are good at purely mechanical debugging - throw them an error, they can figure out which line threw it, and therefore take a reasonable stab at how to fix it. Anything where the bug is actually in the code, sure, you'll get an answer. But they are terrible at weird runtime behaviors caused by unexpected data.
> In the age of LLMs, debugging is going to be the large part of time spent.
That seems a premature conclusion. LLMs excel at meeting the requirements of users having little if any interest in debugging. Users who have a low tolerance for bugs likewise have a low tolerance for coding LLMs.
I don't think so. I think reviewing (and learning) will be. I actually think that the motivation to become better will vanish. AI will produce applications as good as we have today, but will be incapable of delivering better because AI lacks the motivation.
In other words, the "cleverness" of AI will eventually be pinned. Therefore only a certain skill level will be required to debug the code. Debug and review. Which means innovation in the industry will slow to a crawl.
AI will never be able to get better either (once it plateaus) because nothing more clever will exist to train from.
Though it's a bit worse than that. AI is trained from lots of information and that means averages/medians. It can't discern good from bad. It doesn't understand what clever is. So it not only will plateau, but it will ultimately rest at a level that is below the best. It will be average and average right now is pretty bad.
Kind of explains in an epic manner why pipes were useful back then.
They are still useful, but computers are so much more powerful that
many techniques back then came primarily because their computers were
not as powerful. I still, oddly enough, prefer the hardware of the
1970s, 1980s, almost early 1990s too. Today's hardware is much better,
but I am nowhere as interested in it; it is now just like a common
tool.
The Unix command-line composition-via-pipes workflow has no particular connection to hardware capabilities whatsoever.
foo | bar | baz
is useful whether you have a 25Mhz uniprocessor or 1024 cores running at 8GHz.
It is true that the unix pipe model is explicitly designed around text as the data format, and it is true that there's lot of data on computers in 2026 that is not sensibly represented as text. But there's also lots that is, and so the pipe model continues to be valuable and powerful.
Reading this article seems outdated and therefore quaint in some areas now, the “we’ve all felt that moment of staring at a small bit of simple code that can’t possibly be failing and yet it does” - I so rarely experience this anymore as I’d have an LLM take a look and they tend to find these sort of “stupid” bugs very quickly. But my earlier days were full of these issues so it’s almost nostalgic for me now. Bugs nowadays are far more insidious when they crop up.
I’m not going to take bitter advice from someone who either hasn’t used them in a long time, or is terribly bad at using them. Especially as it seems like you hate them so much.
I don’t particularly like them or dislike them, they’re just tools. But saying they never work for bug fixing is just ridiculous. Feels more like you just wanted an excuse to get on your soapbox.
It's not that they can't fix bugs at all, but I find that if I've already attempted to debug something and hit a wall, they're rarely able to help further.
Just focusing on the outputs we can observe, LLMs clearly seem to be able to "think" correctly on some small problems that feel generalized from examples its been trained on (as opposed to pure regurgitation).
Objecting to this on some kind of philosophical grounds of "being able to generalize from existing patterns isn't the same as thinking" feels like a distinction without a difference. If LLMs were better at solving complex problems I would absolutely describe what they're doing as "thinking". They just aren't, in practice.
> Just focusing on the outputs we can observe, LLMs clearly seem to be able to "think" correctly on some small problems that feel generalized from examples its been trained on (as opposed to pure regurgitation).
"Seem". "Feel". That's the anthropomorphisation at work again.
These chatbots are called Large Language Models for a reason. Language is mere text, not thought.
If their sellers could get away with calling them Large Thought Models, they would. They can't, because these chatbots do not think.
> "Seem". "Feel". That's the anthropomorphisation at work again.
Those are descriptions of my thoughts. So no, not anthropomorphisation, unless you think I'm a bot.
> These chatbots are called Large Language Models for a reason. Language is mere text, not thought. If their sellers could get away with calling them Large Thought Models, they would. They can't, because these chatbots do not think.
They use the term "thinking" all the time.
----
I'm more than willing to listen to an argument that what LLMs are doing should not be considered thought, but "it doesn't have 'thought' in the name" ain't it.
> Those are descriptions of my thoughts. So no, not anthropomorphisation
The result of anthromorphisation. When we treat a machine as a machine, we less need to understand it by seems and feel.
> They use the term "thinking" all the time.
I find not. E.g. ChatGPT:
Short answer? Not like you do.
Longer, honest version: I don’t think in the human sense—no consciousness, no inner voice, no feelings, no awareness. I don’t wake up with ideas or sit there wondering about stuff. What I do have is the ability to recognize patterns in language and use them to generate responses that look like thinking.
Wouldn't it make intuitive sense for "writing new code to do a task" and "tracking down a problem debugging code" to be multiple different skills and not the one same skill? Wouldn't it make sense for the one you do more of to be the one you are better at, and not directly 'smart' related? Wouldn't it make sense if tooling could affect the ease of debugging?
"Understanding what code is doing" is a unitary skill (even if it has many different aspects to it, along with language and task specific details). There are plenty of people who are good (enough) at writing new code to do a task but do not understand how to debug; there is (I would suggest) nobody who is good at debugging code who is not also good at writing it.
If debugging is not a separate skill then there is either:
- the author wrote it (including 'debugging') until it worked properly. Therefore they were clever enough to write it that way.
- the author can't make it work (including 'debugging') and therefore they aren't clever enough to write it that way.
And there cannot be a state where they (are clever enough to write it but it doesn't work properly) and they (are not clever enough to debug it), because the fact that it doesn't work properly and they can't make it work properly refutes the claim that they were clever enough to write it that way, and it becomes the second state above. Which puts you on the side of what I'm saying?
"Works" is a less absolute term than you treat it as. If I'm hacking together a little automation script, if the pretty path gives the correct answer, then it "works" to some degree. If that script graduates to part of a company workflow, I'd probably need to fix up some of the corner cases. Like if I wrote it to take in tab-delimited CSV files, but it breaks horribly when encountering a comma-delimited CSV file, I should probably realize that and either guard against it, or add a fix to handle it appropriately.
Are you suggesting that if you wrote your CSV file parser as complex as you could manage, then it would be twice as hard to change the comma to a tab, and you wouldn't be able to do that? That's in line with the quote I'm questioning which you are defending. I might think that the corner cases were easier and less complex than the core.
I'd read the quote as saying "if you write a compression algorithm with the most obfuscated 'clever' macro-using compiler-quirk exploiting C code you can manage and it doesn't work properly, you won't have enough brain power left over to debug such code and make it work. Instead you should have written it with idiomatic C and boring for(){} loops and then you would have a better chance of debugging it until it works". I was questioning the quote and suggesting that if debugging is a separate skill, it must be one that can be developed and improved by practise and tooling.
The person above me suggested that coding and debugging are not separate skills, which rather throws the quote into confusion - if you can "write" it but it doesn't work, and you can't debug it, and "debugging" is the same skill, what are we talking about at all?
This seems to assume that debugging is required to get software to work properly. Sometimes that's true, sometimes it isn't.
And even when it is, sometimes the "not working properly, must debug" point occurs later in time (sometimes much later) from the "it appears to be working" point.
And I might say that the more clever (skilled, experienced, wise) a programmer is, the more likely they can write something 'clever' and make it work and not need to debug it. Then why should they follow Kernighan's quote to avoid working that way?
As I take the quote, it would have been about 1970's, 1980's C and would not have had the benefit of an IDE with "edit and continue" or a LISP or Prolog or Smalltalk interactive REPL with live edit and retry, or ELM's "time travelling debugger" or Git and all related tooling for tracking down changes and who made them, or more modern fuzzers and Valgrinds and static analyzers.
Making a case for writing non-surprising idiomatic code is one thing, but HN parroting the "debugging is twice as hard as coding" and downvoting someone who asks for evidence for this claim is cargo-culting. Why would it be twice as hard and not 1.2x as hard or the same hardness or 10x or 100x as hard? And why would the relationship be fixed, even if tooling and languages and the industry change? And what does it mean to say you can write a, say, compression algorithm "as cleverly as you can" but it's twice as hard as that to spot that you typoed a variable name or something?
The evidence is that many respected people with decades of experience generally agree with them. They are not scientific theories that require validating through testing, they are general advice that is usually true and good to keep in mind.
A good QA team will just run the main tests for you and break your soul while doing it. But the end result is a system that has been broken by humans and not theories on how you can break business logic.
The real question is whether “debugging” the LLM is going to be as effective as debugging the code.
IME it pays dividends but it can be really painful. I’ve run into a situation multiple times where I’m using Claude Code to write something, then a week later while working it’ll come up with something like “Oh wait! Half the binaries are in .Net and not Delphi, I can just decompile them with ilspy”, effectively showing the way to a better rewrite that works better with fewer bugs that gets done in a few hours because I’ve got more experience from the v1. Either way it’s tens of thousands of lines of code that I could never have completed myself in that amount of time (which, given problems of motivation, means “at all”).
You want them writing tests especially in critical sections, I'll push to 100% coverage. (Not all code goes there, but thing that MUST work or everything crumbles. Yeah I do it.)
There was one time I was doing the classic: Pull a bug find 2 more thing. And I just told the LLM. "100% test coverage on the thing giving me problems." it found 4 bugs, fixed them, and that functionality has been rock solid since.
100% coverage is not a normal tool. But when you need it. Man does it help.
Kernighan's Lever - https://linusakesson.net/programming/kernighans-lever/index....
This article is perennially posted here and is probably the best breakdown of this quote.
awesome link, bookmarked, thanks for sharing
So is reviewing and verifying code. Maybe not twice as "hard" if you're skilled in such things. But most programmers I've worked with can't be bothered to write tests let alone verify correctness by other means (good tests, property tests, types, model checking, etc).
It's one thing to point out small trivialities like initialization and life time issues in a small piece of code. But it's quite another to prove they don't exist in a large code base.
Kernigan is a good source of quotes and thinking on programming.
I am fascinated by the prevalence of wanting "tests" from hacker news comments. Most of the code I have worked on in the past 20 years did not have tests. Most of it was shopping carts, custom data transformation code, orchestrating servers, plugin code functionality to change some aspect of a website.
Now, I have had to do some salesforce apex coding and the framework requires tests. So I write up some dummy data of a user and a lead and pass it through the code, but it feels of limited value, almost like just additional ceremony. Most of the bugs I see are from a misconception of different users about what a flag means. I can not think of a time a test caught something.
The organization is huge and people do not go and run all the code every time some other area of the system is changed. Maybe they should? But I doubt that would ever happen given the politics of the organization.
So I am curious, what are the kinds of tests do people write in other areas of the industry?
what are the kinds of tests do people write in other areas of the industry?
Aerospace here. Roughly this would be typical:
- comprehensive requirements on the software behavior, with tests to verify those requirements. Tests are automated as much as possible (e.g., scripts rather than manual testing)
- tests are generally run first in a test suite in a completely virtual software environment
- structural coverage analysis (depending on level of criticality) to show that all code in the subsystem was executed by the testing (or adequately explain why the testing can't hit that code)
- then once that passes, run the same tests in a hardware lab environment, testing the software as it runs on the the actual physical component that will be installed on the plane
- then test that actually on a plane, through a series of flight tests. (The flight testing would likely not be as entirely comprehensive as the previous steps)
A full round of testing is very time-consuming and expensive, and as much as possible should be caught and fixed in the virtual software tests before it even gets to the hardware lab, much less to the plane.
I'll bite: How is your job improving or affected by this AI surge?
So far, very little.
Per corporate policy -- in the name of safety and legal reasons -- the most extensive public uses of AI, like vibe coding and agentic systems, aren't really options. The most common usage I have seen is more like consulting AI as a fancy StackOverflow.
Will this change? I personally don't expect to ever see pure vibe coding, with code unseen and unreviewed, but I imagine AI coding uses will expand.
The interesting bit is this: how much of what you wrote over all those years actually did what you wanted it to do, no more, no less?
This is where testing gets interesting: I took some old code I wrote 30 years ago or so and decided to put it literally to the test. A couple of hundred lines from a library that has been in production without every showing a single bug over all that time. And yet: I'm almost ashamed at how many subtle little bugs I found. Things you'd most likely never see in practice, but still, they were there. And then I put a couple of those bugs together and suddenly realized that that particular chain must have happened in practice in some program built on top of this. And sure enough: fixing the bugs made the application built on top of this more robust.
After a couple of weeks of this I became convinced: testing is not optional, even for stuff that works. Ever since I've done my best to stop assuming that what I'm writing actually does what I want it to. It usually does, for the happy path. But there are so many other paths that with code of any complexity, even if you religiously avoid side effects you can still end up with issues that you overlook.
The value of tests is when the fail they show you of something you broke that you didn't realize. 80% (likely more, but I don't know how to measure) of the tests I write could safely be thrown away because they fail again - but I don't know which tests will fail and thus inform me that I broke things.
The system I'm working on has been in production for 12 years - we have added a lot of new features over those years. Many of those needed us to hook into existing code, tests help us know that we didn't break something that used to work.
Maybe that helps answer the question of why they are important to me. They might not be to your problems.
How do you know you haven't unknowingly broken something when you made a change?
I think if: - the code base implements many code paths depending on options and user inputs and options such that a fix for code path A may break code path B - it takes a great deal of time to run in production such that issues may only be caught weeks or months down the line when it becomes difficult to pinpoint their cause (not all software is real-time or web) - any given developer does not have it all in their head such that they can anticipate issues codebase wide
then it becomes useful to have (automated) testing that checks a change in function A didn't break functionality in function B that relies on A in some way(s), that are just thorough enough that they catch edge cases, but don't take prod levels of resources to run.
Now I agree some things might not need testing beyond implementation. Things that don't depend on other program behavior, or that check their inputs thoroughly, and are never touched again once merged, don't really justify keeping unit tests around. But I'm not sure these are ever guarantees (especially the never touched again).
I think the whole concept of testing confuses a lot of people. I know I was (and still sometimes am) confused about the various "best practices" and opinions around testing. As as well as how/what/when to test.
For my projects, I mainly want to Get Shit Done. So I write tests for the major functional areas of the business logic, mainly because I want to know ASAP when I accidentally break something important. When a bug is found that a test didn't catch, that's usually an indicator that I forgot a test, or need to beef up that area of functional testing.
I do not bother with TDD, or tests that would only catch cosmetic issues, and I avoid writing tests that only actually test some major dependency (like an ORM).
If the organization you are in does not value testing, you are probably not going to change their mind. But if you have the freedom to write worthwhile tests for your contributions to the code, doing so will probably make you a better developer.
Yes
I worked for a company that had no tests.
I worked on the core software, new employee, the programmer who wrote the software gone...
Regularly released new features and found out, some days later, that I'd broken some peripheral, but important, business logic.
Drove me mad! I was not allowed to write tests, it was "unproductive"
Tests are important. Writing good tests even moreso. However, you can't do either of you don't have good product requirements and communication.
Most software has "bugs" simply because people couldn't communicate how it should work.
I think most programmers are on top of actual runtime errors or compilation errors. It's straightforward on how to fix those. They are not on top of logic issues or unintended behavior because they aren't the product designer.
Programmers just cook the burger. If you order it well done, don't complain when it doesn't come out medium rare.
Follow-up questions: Do you test manually? Why? Do you debug manually? Why?
You wanted examples: https://github.com/openjdk/jdk/tree/master/test/jdk/java/uti...
I do test manually in salesforce. Mainly its because you do not control everything and I find the best test is to log in as the user and go through the screens as they do. I built up some selenium scripts to do testing.
In old days, for the kinds of things I had to work on, I would test manually. Usually it is a piece of code that acts as glue to transform multiple data sources in different formats into a database to be used by another piece of code.
Or a aws lambda that had to ingest a json and make a determination about what to do, send an email, change a flag, that sort of thing.
Not saying mock testing is bad. Just seems like overkill for the kinds of things I worked on.
I haven't worked in a codebase in 20 years that didn't have some sort of tests.
Out of interest, what language ecosystems do you tend to work in?
My guess is that some languages - like Go - have a more robust testing culture than other languages like PHP.
Not who you asked but I think it comes down to risk/reward. The consequences of some user finding a big in most websites is low, compared to the risk of an astronaut finding a bug the hard way whilst attempting re-entry.
There is genuinely a reasonable and rational argument to “testing requires more effort than fixing the issues as users find them” if the consequences are low. See video games being notorious for this.
So, industry is more important than language I’d say.
I don't see testing as a quality thing any more, I see it as a developer productivity thing.
If my project has tests I can work so much faster on it, because I can confidently add tests and refactor and know that I didn't break existing functionality.
You gotta pay that initial cost to to get the framework in place though. That takes early discipline.
Depends again on the type of work, and it absolutely has something to do with quality. You’re caring about not breaking existing functionality, you’re refactoring, you’re moving forward with confidence in your code.
That’s absolutely a quality thing. I can assure you that you could move a lot faster if you didn’t try and meet such standards, not that it’d be a good idea necessarily, but in isolation it proves the point.
Developer testing is checking whether the code does what the developer themself thinks it should. QA testing is checking whether the code does what the customers / users / rest of the world thinks it should.
It’s a lot faster and easier than it used to be. Things like xUnit in the .net world make setting up tests friction free to the point where I question a codebase that doesn’t have some kind of basic unit tests. It doesn’t make mock testing or integration testing easier but I would argue if you know the base code and logic is sound those tests are less relevant.
One thing I found is that if testing is easy, your code structure does change a bit to aid with a “test first” approach and I don’t hate it. I thought it made me slower but it doesn’t, it ensures that when all the ground work is finished, the gnarly part of wiring everything up goes much faster.
Yeah I've found the same, having good test discipline influences my code design in a positive way because code that's easier to test is also code that's easier to integrate and understand.
Absolutely. Well said.
I consider a lot of my cleverness when writing code is aimed at achieving functionality, performance, conciseness, etc., while being easy to understand and self-debugged by design. The latter meaning:
• Types enforce safety
• Very tight dependencies
• A tight design (value, store, control flows) where most bugs are likely to be catastrophic, or at least highly visible, as apposed to silent. It works, or it doesn't.
• Use naming and code organization first, concise comments second, and a page or two of doc if all else fails, to make any non-intuitive optimization or operation intuitive again. (Clarify as needed: what it does, how it does it, and why that specific method was chosen.)
For me, a bug repellent design is more important than tests, during development. Except where there is truly messy functionality that no implementation can un-mess. Or results are not used in any way that creates implicit tests them. I.e. they are an intermediate solution to a downstream problem that is not part of the development context.
That seems so basic to writing the "most cleverest" code, that the quote makes little sense to me.
I spend a very small percentage of my time debugging, outside of the real-time development-time write, run, correct loop. If debugging ever took a significant fraction of time, it would give me some serious anxiety.
But as the article states, that may be the result of learning from my less manageable cleverness on ambitious personal projects, over and over, early on. Endless personal projects, in which any success immediately brings into view "the next exciting level" they could become, often requiring a redesign, are great for forcing growth.
In the age of LLMs, debugging is going to be the large part of time spent.
Interesting, I actually find LLMs very useful at debugging. They are good at doing mindless grunt work and a great deal of debugging in my case is going through APIs and figuring out which of the many layers of abstraction ended up passing some wrong argument into a method call because of some misinterpretation of the documentation.
Claude Code can do this in the background tirelessly while I can personally focus more on tasks that aren't so "grindy".
They are good at purely mechanical debugging - throw them an error, they can figure out which line threw it, and therefore take a reasonable stab at how to fix it. Anything where the bug is actually in the code, sure, you'll get an answer. But they are terrible at weird runtime behaviors caused by unexpected data.
> In the age of LLMs, debugging is going to be the large part of time spent.
That seems a premature conclusion. LLMs excel at meeting the requirements of users having little if any interest in debugging. Users who have a low tolerance for bugs likewise have a low tolerance for coding LLMs.
I don't think so. I think reviewing (and learning) will be. I actually think that the motivation to become better will vanish. AI will produce applications as good as we have today, but will be incapable of delivering better because AI lacks the motivation.
In other words, the "cleverness" of AI will eventually be pinned. Therefore only a certain skill level will be required to debug the code. Debug and review. Which means innovation in the industry will slow to a crawl.
AI will never be able to get better either (once it plateaus) because nothing more clever will exist to train from.
Though it's a bit worse than that. AI is trained from lots of information and that means averages/medians. It can't discern good from bad. It doesn't understand what clever is. So it not only will plateau, but it will ultimately rest at a level that is below the best. It will be average and average right now is pretty bad.
> In the age of LLMs, debugging is going to be the large part of time spent.
That seems a premature conclusion. LLMs are quite good as debugging and much faster than people.
One of the few living oldschool UNIX legends still alive.
Here is when he was young:
https://www.youtube.com/watch?v=tc4ROCJYbm0
Kind of explains in an epic manner why pipes were useful back then.
They are still useful, but computers are so much more powerful that many techniques back then came primarily because their computers were not as powerful. I still, oddly enough, prefer the hardware of the 1970s, 1980s, almost early 1990s too. Today's hardware is much better, but I am nowhere as interested in it; it is now just like a common tool.
The Unix command-line composition-via-pipes workflow has no particular connection to hardware capabilities whatsoever.
is useful whether you have a 25Mhz uniprocessor or 1024 cores running at 8GHz.It is true that the unix pipe model is explicitly designed around text as the data format, and it is true that there's lot of data on computers in 2026 that is not sensibly represented as text. But there's also lots that is, and so the pipe model continues to be valuable and powerful.
There is an interesting parallel between 'foo | bar | baz' and 'baz(bar(foo()))'. They all transform and are composable.
Reading this article seems outdated and therefore quaint in some areas now, the “we’ve all felt that moment of staring at a small bit of simple code that can’t possibly be failing and yet it does” - I so rarely experience this anymore as I’d have an LLM take a look and they tend to find these sort of “stupid” bugs very quickly. But my earlier days were full of these issues so it’s almost nostalgic for me now. Bugs nowadays are far more insidious when they crop up.
Trying to get LLMs to understand bugs that I myself am stuck on has had an approximately 0% success rate for me.
They're energetic "interns" that can churn out a lot of stuff fast but seem to struggle a lot with critical thinking.
I’m not going to take bitter advice from someone who either hasn’t used them in a long time, or is terribly bad at using them. Especially as it seems like you hate them so much.
I don’t particularly like them or dislike them, they’re just tools. But saying they never work for bug fixing is just ridiculous. Feels more like you just wanted an excuse to get on your soapbox.
It's not that they can't fix bugs at all, but I find that if I've already attempted to debug something and hit a wall, they're rarely able to help further.
I guess ymmv, but it found a bug for me instantly that I was trying to find all of last night.
> I’m not going to take bitter advice from someone...
He didn't give advice. He reported his personal experience and conclusion.
Oh look we have a pedant. Your point doesn’t even help them though, advice was being charitable.
> seem to struggle a lot with critical thinking.
It is an illusion arising from anthropomorphisation. They aren't thinking at all. They are just parotting the output of thinking that has long gone.
This feels too strong IMO.
Just focusing on the outputs we can observe, LLMs clearly seem to be able to "think" correctly on some small problems that feel generalized from examples its been trained on (as opposed to pure regurgitation).
Objecting to this on some kind of philosophical grounds of "being able to generalize from existing patterns isn't the same as thinking" feels like a distinction without a difference. If LLMs were better at solving complex problems I would absolutely describe what they're doing as "thinking". They just aren't, in practice.
> Just focusing on the outputs we can observe, LLMs clearly seem to be able to "think" correctly on some small problems that feel generalized from examples its been trained on (as opposed to pure regurgitation).
"Seem". "Feel". That's the anthropomorphisation at work again.
These chatbots are called Large Language Models for a reason. Language is mere text, not thought.
If their sellers could get away with calling them Large Thought Models, they would. They can't, because these chatbots do not think.
> "Seem". "Feel". That's the anthropomorphisation at work again.
Those are descriptions of my thoughts. So no, not anthropomorphisation, unless you think I'm a bot.
> These chatbots are called Large Language Models for a reason. Language is mere text, not thought. If their sellers could get away with calling them Large Thought Models, they would. They can't, because these chatbots do not think.
They use the term "thinking" all the time.
----
I'm more than willing to listen to an argument that what LLMs are doing should not be considered thought, but "it doesn't have 'thought' in the name" ain't it.
> Those are descriptions of my thoughts. So no, not anthropomorphisation
The result of anthromorphisation. When we treat a machine as a machine, we less need to understand it by seems and feel.
> They use the term "thinking" all the time.
I find not. E.g. ChatGPT:
Short answer? Not like you do.
Longer, honest version: I don’t think in the human sense—no consciousness, no inner voice, no feelings, no awareness. I don’t wake up with ideas or sit there wondering about stuff. What I do have is the ability to recognize patterns in language and use them to generate responses that look like thinking.
This thread is the most pathetic internet argument I’ve read in quite a while. Look at yourselves, both of you.
Is there any evidence for these assertions?
Wouldn't it make intuitive sense for "writing new code to do a task" and "tracking down a problem debugging code" to be multiple different skills and not the one same skill? Wouldn't it make sense for the one you do more of to be the one you are better at, and not directly 'smart' related? Wouldn't it make sense if tooling could affect the ease of debugging?
"Understanding what code is doing" is a unitary skill (even if it has many different aspects to it, along with language and task specific details). There are plenty of people who are good (enough) at writing new code to do a task but do not understand how to debug; there is (I would suggest) nobody who is good at debugging code who is not also good at writing it.
If debugging is not a separate skill then there is either:
- the author wrote it (including 'debugging') until it worked properly. Therefore they were clever enough to write it that way.
- the author can't make it work (including 'debugging') and therefore they aren't clever enough to write it that way.
And there cannot be a state where they (are clever enough to write it but it doesn't work properly) and they (are not clever enough to debug it), because the fact that it doesn't work properly and they can't make it work properly refutes the claim that they were clever enough to write it that way, and it becomes the second state above. Which puts you on the side of what I'm saying?
"Works" is a less absolute term than you treat it as. If I'm hacking together a little automation script, if the pretty path gives the correct answer, then it "works" to some degree. If that script graduates to part of a company workflow, I'd probably need to fix up some of the corner cases. Like if I wrote it to take in tab-delimited CSV files, but it breaks horribly when encountering a comma-delimited CSV file, I should probably realize that and either guard against it, or add a fix to handle it appropriately.
Are you suggesting that if you wrote your CSV file parser as complex as you could manage, then it would be twice as hard to change the comma to a tab, and you wouldn't be able to do that? That's in line with the quote I'm questioning which you are defending. I might think that the corner cases were easier and less complex than the core.
I'd read the quote as saying "if you write a compression algorithm with the most obfuscated 'clever' macro-using compiler-quirk exploiting C code you can manage and it doesn't work properly, you won't have enough brain power left over to debug such code and make it work. Instead you should have written it with idiomatic C and boring for(){} loops and then you would have a better chance of debugging it until it works". I was questioning the quote and suggesting that if debugging is a separate skill, it must be one that can be developed and improved by practise and tooling.
The person above me suggested that coding and debugging are not separate skills, which rather throws the quote into confusion - if you can "write" it but it doesn't work, and you can't debug it, and "debugging" is the same skill, what are we talking about at all?
This seems to assume that debugging is required to get software to work properly. Sometimes that's true, sometimes it isn't.
And even when it is, sometimes the "not working properly, must debug" point occurs later in time (sometimes much later) from the "it appears to be working" point.
And I might say that the more clever (skilled, experienced, wise) a programmer is, the more likely they can write something 'clever' and make it work and not need to debug it. Then why should they follow Kernighan's quote to avoid working that way?
As I take the quote, it would have been about 1970's, 1980's C and would not have had the benefit of an IDE with "edit and continue" or a LISP or Prolog or Smalltalk interactive REPL with live edit and retry, or ELM's "time travelling debugger" or Git and all related tooling for tracking down changes and who made them, or more modern fuzzers and Valgrinds and static analyzers.
Making a case for writing non-surprising idiomatic code is one thing, but HN parroting the "debugging is twice as hard as coding" and downvoting someone who asks for evidence for this claim is cargo-culting. Why would it be twice as hard and not 1.2x as hard or the same hardness or 10x or 100x as hard? And why would the relationship be fixed, even if tooling and languages and the industry change? And what does it mean to say you can write a, say, compression algorithm "as cleverly as you can" but it's twice as hard as that to spot that you typoed a variable name or something?
The evidence is that many respected people with decades of experience generally agree with them. They are not scientific theories that require validating through testing, they are general advice that is usually true and good to keep in mind.
A good QA team will just run the main tests for you and break your soul while doing it. But the end result is a system that has been broken by humans and not theories on how you can break business logic.
Isn't it so interesting that AI will completely destroy the motivation to be more clever?
... and thereby also destroy the source of its future training data.
Not at at all ... intelligent.
The real question is whether “debugging” the LLM is going to be as effective as debugging the code.
IME it pays dividends but it can be really painful. I’ve run into a situation multiple times where I’m using Claude Code to write something, then a week later while working it’ll come up with something like “Oh wait! Half the binaries are in .Net and not Delphi, I can just decompile them with ilspy”, effectively showing the way to a better rewrite that works better with fewer bugs that gets done in a few hours because I’ve got more experience from the v1. Either way it’s tens of thousands of lines of code that I could never have completed myself in that amount of time (which, given problems of motivation, means “at all”).
LLMs are where you need the most tests.
You want them writing tests especially in critical sections, I'll push to 100% coverage. (Not all code goes there, but thing that MUST work or everything crumbles. Yeah I do it.)
There was one time I was doing the classic: Pull a bug find 2 more thing. And I just told the LLM. "100% test coverage on the thing giving me problems." it found 4 bugs, fixed them, and that functionality has been rock solid since.
100% coverage is not a normal tool. But when you need it. Man does it help.
> You want them writing tests especially in critical sections, I'll push to 100% coverage.
But how do you know if you got it?
I've seen no LLM that can even verify execution pathway coverage.