This is why every AI fail will be met with "what was the prompt?" This is why devs think that asking their LLM to not hallucinate will prevent it from hallucinating.
The prompt is a wish, and the perfect wish can do anything.
Except, of course, within the bounds of the spell's power.
arstechnica.com
“Do not hallucinate”: Testers find prompts meant to keep Apple Intelligence on the rails
Long lists of instructions show how Apple is trying to navigate AI pitfalls.