A part of this too is age related. Young children and pregnant women that can transmit injested lead to their babies are much more at risk. The danger to a developing brain is more than to a developed brain. People work around lead in many industrial jobs safely.
Because lead can do no harm unless it's injested, actions like sanding wall unnecessarily are more dangerous than leaving it in place. If you watched the home improvement shows from Massachusetts years ago, you might remember that they only had to strip door and window frames and "mouthable" surfaces. This was to keep kids from injesting lead by chewing on these surfaces. As I understand it, even these restrictions have been relaxed a little.
No, I would not get excited about driving a nail into the wall, but I would if there was a lot of peeling and cracking paint. There are ways to safely add new wall surfaces and, in effect, bury the lead if it appears to be a problem.