As you said, the house is 20 years old, so anything directly related to the hole has already happened, as far as building dead loads are concerned. (A dead load is the load placed on a structural element by the other permanent elements of the structure.)
With an inch and a half of good wood above and below, you have the most important part of the joist still intact. (There aren't any knots in the wood below the hole or either side of it within a foot of the center of the hole, are there?)
If you are really concerned about this, you can look at installing some straps on the bottom of the truss to assist in the tension side. The top of the joist is not really a concern, it would take a lot more compressive force than you have there to make the top "chord" collapse. For straps, I would probably go with nail on straps 1 1/2" by 4', and simply nail it tightly (in tension) to the bottom of the joist. If it really bothers you, you could also consider adding a small load-bearing wall under the long side of the joist (under the joist on the side of the hole where the joist is the longest) and make a closet or something downstairs.
Mostly, I don't think its a huge issue, the house has been that way for 20 years...