66 Tokens Make a Diffusion Language Model Look Easy

A diffusion language model generates text by starting from masked or otherwise corrupted tokens and iteratively restoring them. In this MacBook Air M2 demo, that idea shows up in its smallest, most hackable form: a toy character-level model that learns to recover missing characters from Karpathy's tiny Shakespeare dataset . The GitHub project Encrux/simple_dlm really is small and direct. The author reports a roughly 7.5 million parameter model, a 66-token vocabulary made of 65 characters plus [M