Beg to differ. I've done this few times and if you are careful you can get it off without destroying the wall behind it. Get a stiff scraper and carefully tap it between the wall and the brick, holding the blade parallel to the wall. Once you get the hang of it they'll come off pretty easy. Scrape off as much of the adhesive as you can then you will need to skim the wall with drwyall compound to smooth it. Sand it, prime then paint.