There seems to be a debugging pattern that arises when problems are in proximity to a poorly understood, highly complex part of the system: We tend to think that's where the problem probably is. And we can lose an inordinate amount of time looking for it there. It's like an inverse streetlight effect.
I've seen coding LLMs do it too. I have a well-tested, but complex, subsystem that constantly draws their attention when something non-obvious elsewhere goes wrong.
There seems to be a debugging pattern that arises when problems are in proximity to a poorly understood, highly complex part of the system: We tend to think that's where the problem probably is. And we can lose an inordinate amount of time looking for it there. It's like an inverse streetlight effect.
I've seen coding LLMs do it too. I have a well-tested, but complex, subsystem that constantly draws their attention when something non-obvious elsewhere goes wrong.
Love the punchline