You'll probably get both 'sides' of an answer on your question. My answer would have recommended removal. With that, I'll explain to you why I would make this recommendation. I'll leave it to 'others' to give their reasons for not removing them.
During the installation process, the shingles are most likely 'broken' as nails and such are driven into them underneath your insulation and siding. It's certainly possible that over time, the shingles begin to 'fall off' behind all that and possibly create a 'bulge' in the siding as they slide down. I understand that the siding is nailed to the house but do realize that with a little pressure behind the siding, the nails can work their way loose and the siding can start to bulge.
Also, as time goes on, you never know what laws will be enacted. Seller Disclosure Forms are being requested of a Seller these days more and more because a Buyer wants to know some 'history' about the house. IF there's a question about Asbestos or Asbestos Shingles, and it requires you to divulge that information, you will need to do so. ASBESTOS just conjures up aaaaall kinds of thoughts in peoples' minds and usually ends up in a non-sale. Granted, if the Buyer only educated themselves about the hazards of the material (or non-hazards), they'll be able to make a more informed decision. In fact, if you ever choose to sell your home and ARE asked, make sure you 'attach' some useful information to help the Buyer know what they're getting. The information you provide may be enough to keep them 'interested' in your house.
Thirdly, it's certainly cheaper to not have to remove and dispose of a non-friable asbestos product. Depending on where you live, it could cost anywheres from $35-$150 per square (10' x 10') to have the stuff removed and properly disposed of. If the vinyl job called for 1800 sq. ft. of vinyl (18 squares), then it could cost $630 - $2700 to have the 'old stuff' removed. As an aside, in my particular jurisdiction, which includes my Township, County, and State Governments I, as the Homeowner of a Single-family Home, am 'allowed' to remove and dispose of the shingles w/o any special Permits or Professionals. Granted, it may be different where you live as to what you're allowed, and not allowed, to do.
I realize the job at your house is finished. There's no sense in trying to believe you made a 'wrong decision'. Like I said, you'll probably get an explanation or 2 that it's OK to keep the shingles on. I only made my recommendation based on what I gave you, nothing more. I do wish you well.
Jay J -Moderator
PS: I'm curious - Did the Sider give you an 'explanation' as to why to keep the shingles on the house? And what's giving you 2nd thoughts about having left them on? I'm curious why they weren't removed ...