Tokenizer-Aware Markdown Chunking That Doesn't Shred Tables

Book: RAG Pocket Guide: Retrieval, Chunking, and Reranking Patterns for Production My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub Picture a support engineer asking a doc bot why a webhook retry is failing. The retriever returns half of a markdown table: the header row, two body rows, then a hard cut at exactly 512 tokens. The remaining four rows, including the one that lists 429 as the retry trigger, go to the