Testing Long-Horizon Coherence in MusicGen: A Real-Data Mech Interp Pipeline

Testing Long-Horizon Coherence in MusicGen: A Real-Data Mech Interp Pipeline I have been working on a mechanistic interpretability experiment for music generation models. The big motivating question is simple: Do autoregressive music models have internal features that track long-horizon musical structure, or are they mostly stitching together locally plausible audio? More concretely, I want to test whether a model like MusicGen has residual-stream or sparse-autoencoder features that say somethin