Building a Real-Time Screen Reader on macOS That Actually Works

The models that failed, the bugs that took weeks, and the architecture that survived. What I Was Trying to Do I needed to read specific UI panel content from a screen in real-time on macOS with Apple Silicon. Detect panel boundaries from a screenshot, extract text via OCR, accumulate that text across scrolling viewports, and render a transparent overlay on top showing the results. All of it running locally -- no cloud APIs, no network round-trips, no usage limits. A self-contained screen reading