Why your LLM product hallucinates the one thing...

A woman forwards a conversation with her boyfriend to my AI bot. The model detects danger signals (emotional abuse, isolation tactics) and responds with a crisis hotline number. Caring. Responsible. One problem: it's a children's hotline. The model hallucinated a crisis contact for an adult in distress. The prompt says "DO NOT invent contact information." Doesn't matter. The model's drive to be helpful is stronger than any instruction. This is not a prompting problem. This is an architecture pro