IDK if the author's 'metacognition' needs to be a feature of the LLM itself.
I could imagine a harness that 1) reads LLM output 2) uses a research sub-agent to attempt to verify any factual claims 2) rephrase the main agent's output such that it conveys uncertainty if the factual claim cannot be independently verified
Unproductive tangent: Why do we call it "hallucinationing" instead of "bullshitting" when that is so clearly what it is?
If I'm talking to a guy that says, "I have a really fast metabolism, that's why I can eat whatever I want", he's not hallucinating - he's full of shit.
IDK if the author's 'metacognition' needs to be a feature of the LLM itself.
I could imagine a harness that 1) reads LLM output 2) uses a research sub-agent to attempt to verify any factual claims 2) rephrase the main agent's output such that it conveys uncertainty if the factual claim cannot be independently verified
Unproductive tangent: Why do we call it "hallucinationing" instead of "bullshitting" when that is so clearly what it is?
If I'm talking to a guy that says, "I have a really fast metabolism, that's why I can eat whatever I want", he's not hallucinating - he's full of shit.