If the trim gets enough sun, it will break down over time. Polyurethane has a limited amount of UV protectant in it. This is one of the main reasons many folks resort to painting their window trim. (Curtains will help too.)
You didn't do anything wrong. 2 coats would be a minimum. If you wait too long, the UV will 'attack' the wood itself. Get yourself some Liquid Sandpaper, and read the can BEFORE you leave the store. Then re-coat. Sorry, unless you take measure to reduce exposure to sunlight, it will continue to break down.
My best to ya and hope this helps.
PS: We have south-facing windows ourselves. The 1st floor has a porch overhang. We get no summer sun and little winter sun. The 2nd floor is completely exposed. All windows have curtains and/or blinds. They are vinyl, replacement windows and 'E' rated too. You may need to look at the type of windows you have to see if they will help reduce your problem.