You might as well look at the fascia as well because in all likelyhood, you are going to have to remove the fascia in order to work on the soffit.
Steve is right about some installers that cover the damage up but, I learned from my dad's company this very important lesson, never cover or nail into rot if you plan on being in business for any lenth of time! As you can see, it causes blow off. It is also a bad reflection on the company.
You might want to contact the siding company to see if you can work out an agreement with them. You fix the wood, they fix the trim!
The one thing that I would recommend in the mean time if you choose to put off this repair, would be to install wire screening to keep the pest out. You should use a layer of insect screening to keep the bees out and a layer of 1/4" or 1/2 " wire to keep the squrrels, bats and othe pest away.