The test for asbestos is rather inexpensive at less than 40 dollars per sample on the east coast. A small 1 inch square will surfice. Results would be in a couple days.
If asbestos, then removal is expensive. If it is asbestos, sounds like your panels are "friable". Thus asbestos fibers are exposed to the air due to crumbling of the panel. In this case, it would be wise not to spend prolonged time within the shed and definitely, don't do the removal yourself without wetting the material down significantly. Get the experts here. Asbestosis is nothing to full around with.