>>143278653 Reasons:
1) bad experiences with people with similar faces (Your brain recognises the similarity and thinks you could repeat the same bad experiences)
2) According to the studies of Objective personality, there is a statistical correlation (No causation, only statistical observation) between the face of a person and his personality. Your mind could recognize this and deduce someone is a shitty person from the face, even if it is not true
3) Faces express emotions, you can recognize fake smiles, intimidatory or hateful expressions.