Q: What reasons, if any, based upon which you can refuse to follow a particular person in salaah?
A: Only such a person should be appointed as Imaam whom people are pleased with. But their disapproval should be based on valid Islamic grounds, not on personal motives or unsubstantiated claims. If the Imaam falls under the category of 'faasiq', he should be replaced, but as long as he leads the congregation, people should continue to offer salaah behind him. It is better to perform Salaah behind a faasiq than performing it alone. A faasiq is one who commits a major sin openly at least twice, or commits minor sins on a continuous basis without taubah.
If a person holds a kufr belief, such as beliefs of Qadianis, Bahaais, Shias, etc. then he cannot be appointed as Imaam for salaah. In fact salaah behind him is not valid for that man is not a Muslim. The only valid reason for refusing to perform salaah behind an Imaam is if he maintains views and beliefs that are clear-cut kufr (disbelief, apostasy).
And Allah Knows Best