Hi Gurdeep: We too had a mouse problem about a year ago. This prompted a bit of research on our part to determine why they were with us and what we could do to get them out.
It seems mice are a teritorrial lot, with distinct plots of real estate they call their own. When real estate is scarce (as is often the case during mild winters when their numbers proliferate in the spring), they may inhabit areas they would normally avoid, like the inside of your house. If you don't have an effective deterrent (like a cat or dog) and this occurs, they will stay put until you destroy them (it seems they will not become dissuaded and go elsewhere once they have settled in). If they proliferate in your house, then your problems proliferate, as ours did.
Other than a hunter tom-cat, the only way we found to remove them was to trap them. I used the standard 4" mousetrap, bated with peanut butter (they love the stuff). In the course of 4 months I caught about 8 mice of various sizes. Once smashed in the trap, I put the carcuses out on the curb, where many of the local carnivorous birds found their meals. By carefully keeping doors and windows shut, we have not had any mice infestations since then.