a razor blade and some Goof Off will go a long ways towards removing the old stuff. Read the label on the Goof Off, and use plenty of ventilation, DON'T SMOKE!
Look at glass scrapers which are little plastic handles you put a razor blade into. They work really well.
As far as caulk, go with a latex caulk, and buy the best (longest warranty) you can find. GE makes a good one. You can buy clear, or white, or it even comes in designer colors if you need it to.
Lastly, buy a "Dripless" caulking gun. They cost more, (about $15) but they make all the difference. When you stop squeezing the handle, they let the pressure off the tube, unlike the classic style that keeps pushing on the ratchet.
Keep your fingers clean and wet, and keep a wet rag handy to help wipe. Just press your finger tightly into the corner to give yourself that clean edge, and wipe off your finger OFTEN.
Only apply as much caulk as will barely scrape onto your finger when you wipe. People tend to apply more than they need.