DON'T use weed block. Use either Filter Fabric or Landscape Fabric. Be sure to lay that in your 'trench' first, then the gravel (which you want to TAMP), then your pipe. THEN, wrap it all up with the remaining fabric. So, when you cut and lay your fabric, you need enough 'excess' to wrap the thing up. Don't worry about over-using the fabric. The key is to keep the dirt out and let the water in.
Also, you didn't mention anything about back-filling with gravel. You're on a borderline situation, and if your pool is 'above' your wall, I'd definitely back-fill w/gravel. It, too, should be tamped. (If you are backfilling w/gravel), then install it as you build the wall. Be SURE to, FIRST, lay some fabric on the dirt-side BEFORE you start laying in the gravel. Here, too, just like you're doing w/the drain trench, you DON'T want any dirt to 'pollute' your gravel. AT a height of 4', you'd want the gravel to be about 18" wide. That will give you about a 70 degree angle on your dirt slope.
You're doing just fine. The Fabric can be bought at Gravel Yards or Stone Quarries or the like. I don't know if it's sold in the Home Centers because I usually get it at the Yard when I get the stone. Visit Allan Block for DIY info. There are a few PDF downloads to view. You can use this info as a guide. (There's info on when to backfill w/gravel too.)
Perhaps others will have more info. My best to ya and hope this helps.