Do vulgar and rude characters disturb you

At 16 I was criticized for the vulgarity of my comics, and I realized that the criticism was right, so I stopped. At that time I made my character speak in a vulgar way just because they were teenagers and most of teenager are vulgar in their speech. Is just age. I can't remember if I did that for the sake of realism, or just to be 'edgy'.

Is a while I'm writing 'Rough Dream'. I already tell you that my main source of inspiration is 'Ashita no Joe' (But the story is totally different, is just inspiring) alongside with real wrestling matches. And when I was reading 'Ashita no joe', to me it was hard to relate with Joe Yabuki for a good part of the story, because his being a total asshole. I kept reading the story because I could relate well with the other characters and the story was good, but I costantly hoped about somebody giving an hard lesson to Joe.
Brian is an asshole even worse than Joe (also considering that he is older, and hadn't a so difficult past, a part from learning disorders+separated parents), and he is very rude and mocking in his speech. He has good quality, but they come out slowly in the story. My worry is that a character like that would make the reader escape. The other characters aren't that rude (some are normal, some are even well mannered), but Brian is both the protagonist and the main character. Also one of the comments I received on a old drawing of Brian was 'I would avoid a person like that', and I have to admit that I also would stay away from a person that talks and act like Brian, even knowing his good qualities (and supporting what he does in the last drawing I did).
Do you think that if the story is good, Brian's rudeness will be not a problem? (and maybe there will be possibilities that you will end to like him), or I should tone him down? The problem is that, toning him down would make me fell like I'm lying and I would rather prefer to write a good story and to make all RD characters interesting, relatable and believable. And in the end of the story, readers should also understand, like or appreciate Brian.