Our vent terminates outside with a louver-type closure. With the slightest 'breeze' from the dryer, they open up to let out the air.
The louvers catch a bit of the excess lint such that every few weeks, when I'm outside and think of it, I have to pull some of it off of the louver hinges. No big deal; it's not like bugs/animals can get in. I guess my point is if you have a lint trap in the dryer, it may have a tear in it. Pull it out and gently push on it to see if it's separated from the frame, especially on the edges. If it is, find a dealer that sells your brand and give them the dryer's model # so one can be ordered. (Maybe it's in stock.)
It's not abnormal to get 'some' lint in the garden. If it's excessive (and you can check what other family members' dryers 'dump' in the yard is a lot less),then the trap probably isn't sitting properly in the dryer. Clean out the 'holder' with your hand as much as you can. If you have a small vacume hose, go ahead and stick it in the holder too but NOT too far or you might not get it out. Personally, I wouldn't recommend putting any additional obstructions/vents/screens anywhere on the dryer or ductwork because you may create back-pressure, thus, possibly causing the dryer to automatically shut down, overheat, and clog up with MORE lint.
What you CAN do is buy some 1/4" type mesh or screen from the Home Center and rig up a 'screen' outside and AWAY from the exit point of your vent. Don't attach it to the point/wall where the vent EXITS the house (thus causing back-pressure). Instead, set it up as a free-standing screen or suspended one. Set it up so it can easily be moved to be cleaned. As LJ suggested, at worst, relocate the vent on the outside if necessary.