I'm going to assume they are varnished and probably need stripped. BIX Orange is an outstanding stripper, it does all the hard work for you; has a thicker, gel-like texture and can be sprayed on. I've used several different strippers on furniture, my local antique dealer/furniture restorer recommended this one, it's great. Just remember not to let it set till it begins to dry (instructions on can). If allowed to dry out, it's makes the job more difficult. **Be sure to get some stripping gloves (they're orange color, steer clear of yellow household latex gloves--stripper will chew right throught those puppies.) And wear safety goggles, long sleeves and pants (a tiny splatter on skin really burns). You will also need some #0000 steel wool and mineral spirits for clean-up. After you're down to bare wood, sand with a medium grit, repeat with a fine grit, re-stain and seal/varnish.
Stain: if you want to match the color you already have, that can be a bit tricky with window sills. (I usually take a sample piece of wood to the paint store with me that I would like to match) What you see on the can is not necessarily what you will get: i.e. a "colonial maple" colored stain applied to pine will appear much different than "colonial maple" applied to oak. This is due to variances in natural wood color, porousity, grain, etc. Since you can't take a piece of the sill in to the store to have the stain color matched, try to determine what the wood is (i.e. oak), get yourself a scrap piece (lumber yards have them), take it to your paint store and have them apply several different stain choices that you can take home and compare to your present shade. Or, if you're in the mood for a change of color, just take the wood scrap in and ask the store to apply a few that you like, so you can see the end result. Again I use MinWax products, or Old Masters, but that's just personal preference.
I like MinWax sealers/varnish, but since you have the sun's interference to consider, there might be a finishing product designed more specifically for that on the market.