I have recently cleaned the brick on my fireplace. TSP didn't do the job. I had to use Muriatic Acid. Go buy the Muriatic Acid, it is available at your local Home Center. Also, you'll need a plastic spray bottle, safety goggles, chemical resistant gloves, a breathing mask with filters that keep out the muratic acid. Mix about 1 part acid with 8 parts water, put it in the spray bottle and spray it on the brick. Be sure to wear your safety equipment! Let it sit on the brick for a couple of minutes and then wash it off with water.
If this doesn't work try mixing 1 part acid with 5 parts water and repeat the process.
WARNING: Muriatic acid is very dangerous and so are the fumes it produces. Be sure to use your safety equipment. If it gets on your skin, wash it off with water right away.