If you live in a climate where there's freezing and thawing, any trapped water will damage your block, or simply break the caulk and leak over time from such forces. This also goes to say that if there is any water seeping out on a warm winter day, when you wake up the next morning, you may see ice formations on the front of the wall!
IMO, the best way to deal with this is to do it right. It's worth it because in due time, you'll end up redoing the top rows anyway. Then, 1) you may not find a matching block any more, and 2) it will cost more to fix then it would cost you in manual labor and some rock today. I also suggest you line the dirt-side with filter fabric before back-filling with gravel. This prevents the dirt from 'polluting' the stone as water runs out of it and into the stone. The best thing to do is wrap the newly-graveled area COMPLETELY in the stone to about 6" from the top of the wall (and with a slight incline towards the front of the wall). This way, you can throw dirt on top of the wrapped gravel and grow grass, and the dirt won't pollute your gravel. (This is how it's done.) I also suggest you run gravel alllll the way to the top, ONLY right up against the wall for a width of about 2".) This way, any water that runs towards the wall or collects on TOP of your dirt just behind the wall has a place to DIRECTLY go to the bottom just behind the wall. (It's a drainage thing.) Hopefully, you did follow the directions when it came to draining the water away from the BASE of the wall.
If you'd like more on Retaining Walls, check out the link in Retaining Walls, and click on Retaining Walls. It's great stuff. (It will basically support what I've been saying.) My best to ya and hope this helps.