The main issue is black lung disease. If you breath fibers they get into your lungs and can't get out. Eventually causing black lung disease.
I'm not an m.d., but I don't think you need to run out and buy that burial plot yet. Hope you didn't eat lunch without washing but probably not a problem cause the asbestos is embedded in the tile.
The main concern is the airborne fibers. It used to be common to wrap and unwrap pipes with asbestos insulation especially from WWII and on. Nobody worried about it and it used to snow asbestos on people upon removal and no one knew about the disease.
Many people were exposed to it and breathed it regularly. Still not everyone got the disease. My uncle did say that the newsletters in the dearly departed column from his plumbing, pipefitters, and steamfitters union says more and more people die from it though.
The EPA tends to go overboard but they would rather be safe than sorry. I've handled plenty of asbestos before (unaware of risks) and I'm still kicking. Grew up with shower doors made of sheet asbestos too.