A question to gays, could you explain this logic to a hetero?

I made a suggestion to one of my buddies about a place to go for holiday saying, "you'd love it - it's like ...everybody's gay there, and those who aren't gay wish they were." to which he replied, well that would suck because if everyone were gay, then there would be nothing special about being gay. This baffled me a bit because I figured that I would easily trade whatever characteristic made me "special" in order to be discriminated against less. If someone's going to dislike me, I'd prefer it be for my personality, not for some label I've acquired involuntarily.