There are several "abatement" companies out there that get rid of old gas station storage tanks, oil furnace tanks, etc. They are experts at removing underground stuff and replacing the hole with useable compacted fill. I would try that route first.
Second, you might try calling around for some pool installers to ask them what they think.
I wouldn't recommend just dumping the pool full of dirt, because I suspect you would find voids and other problems down the road, such as a saturated soil condition as the pool fills up with water again.
Lastly, it would be interesting to see if there is a way to do this without totalling out your pool, so you can keep the value of the pool when you go to sell the house whenever that happens. Besides, you may decide later that you wish you still had it!