Apple's On-Device Model is Terrible for Chat But Surprisingly Good at Structured Output and Tool Calling

I've spent weeks stress-testing Apple's on-device model — the ~3B parameter one that runs on the Neural Engine of any Apple Silicon Mac. To test it thoroughly, I built Think Local , a macOS app that exercises every capability of the model: chat, image generation, structured output, tool calling, and parameter comparison. My conclusion: As a chatbot, the model is terrible. As a structured output and tool calling engine, it's surprisingly good. This distinction matters because it completely change