I don't think it was a racial thing with A-Gon. He didn't hit as many home runs as expected, he was a member of a 'dream team' that collapsed horrifically in his only full season with the team, and he also had a knack for saying the wrong thing.
The tag that everyone remembers about 2011 is 'chicken and beer', and that had nothing to do with A-Gon, it was supposedly led by Beckett, Lackey and Lester-kind of a redneck thing, you might say.
Beckett and Lackey in particular were guys who at various times were not well liked at all by Sox fans-in spite of their key roles in championships.