The possibility that one's appearance or behavior will be misused to confirm another person's oversimplified, prejudiced attitude is referred to as:

Stereotype threat

Explanation:

This can be described as stereotype threat. Stereotype threat is a predicament where people fear that they might be at the risk of conformity. That is the fear that they may conform to stereotypes about the social groups they belong. This is a psychological threat and it comes up when the person is in a situation where there is a negative stereotype about their group.

From this question, just because two people are in a particular group one person's attitude and appearance is being used to judge the other person.