Reposted by Daniel Abraham
A thing I learned not too long ago is that when an LLM selects a token that leads it down an incorrect path (factually), it has no way to back up, or even know that it should back up, it's just on to the next likely bit. Literally every LLM is just posting their way through it. What an image.
