Hi, J. There's a lot of things that it could be and tile grout is one of them. In any case, tile stores sell grout saws that are especially made for cutting the grout out from between tiles. They are pretty small so this will be a lot of work. The saws I saw were under $10 but I don't remember the exact price.