… or config files floating around in your FHS dirs

Mostly true, but unless you’re doing an impermanence setup, programs still poop random files all over the place. Especially in home and other various places like /var/lib /var/cache.