the best thing is to get a box cutter and work on one end and get a "tail" loose so you can slowly and carefully just pull it down all the way. sometimes this works and sometimes it keeps breaking. Wish that I could give you the magic solution, but not one.
Next time when its removed and all clean, apply bleach to the intersection to kill the mold and mildew. Then let it dry over night. Then with a blow dryer, go along the intersection and dry it again and warm it up a bit.
Then apply new 100% silicon White Caulk over the area. Cut the nipple correctly and apply it smoothly over the intersection and then after its done, smooth the caulk out with your finger that's dipped in rubbing alcohol. This should work for 3 to 4 years when you have to do it again.