Abstract:

Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts have mostly been empirical so far, which cannot answer the fundamental question whether it can be completely eliminated. In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs. Specifically, we define a formal world where hallucina- tion is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all of the computable functions and will therefore always hal- lucinate. Since the formal world is a part of the real world which is much more complicated, hallucinations are also inevitable for real world LLMs. Furthermore, for real world LLMs constrained by provable time complexity, we describe the hallucination-prone tasks and empirically validate our claims. Finally, using the formal world framework, we discuss the possible mechanisms and efficacies of existing hallucination mitigators as well as the practical implications on the safe deployment of LLMs.

  • eldrichhydralisk@lemmy.sdf.org
    link
    fedilink
    English
    arrow-up
    5
    ·
    8 months ago

    Which is exactly what the paper recommends! As long as you have something that isn’t an LLM in the pipeline to vet the output and you’re aware is the tech’s limitations, they can be useful tools. But some of those limitations might be a more solid barrier than some sales departments would like us to believe.

    • Daxtron2@startrek.website
      link
      fedilink
      arrow-up
      1
      ·
      8 months ago

      Yep! Figured id share for the 90% who won’t read it though haha. Sales teams truly are the worst when it comes to that