How to tell if a page uses JavaScript rendering (and what to do about it)

You write a scraper, test your selectors in the browser, and everything looks right. Then you run the spider and get back nothing. This is the most common point of confusion for developers new to web scraping: the browser shows you the data, your scraper does not find it, and the two are looking at completely different things. The browser executes JavaScript before you see anything on screen, but your scraper, unless you specifically configure it to do otherwise, does not. It sees the raw HTML t