It boils down to people in my opinion. A twat will be a twat and is a twat. Irrelevant of culture lifestyle or race, in my opinion. I have a few traveller mates from my rave days who I consider some of my closest friends, but then I've met a couple who have been utter shites, just like I've met people at work who have been shites, on the bus, in the bank etc.
Tarring all with the same brush will just create more problems, if you're constantly being told you're a thriving p***y b*****d by nearly everyone around you. After enough time, you'll get fed up trying to prove them wrong. What's the