I'm a Colts fan, so I have an excuse to hate the Patriots. But I think the casual fan hates the Patriots for 2 reasons. One is that the constantly win, yet have no true superstar (with the exception of Brady, and even some idiots still question if hes really that good. Apparently he needs to win 8 Super Bowls.) The other reason is that teams that always win, regardless of the sport, are usually hated outside of their own fan base. Duke, Notre Dame, USC, Yankees, and Lakers fans can probably tell you about that. So I think thats why most of America hates the Patriots.
I should however admit that while I hate them because they're my team's rival, I do respect the Patriots a lot. I think its impressive how they win without a whole bunch of "superstars". I also find it strange that everyone complains about players playing individiually and for themselves and how no one ever plays as a team, yet the Patriots are a perfect example of team play (this is also applies to the Spurs), yet those same people don't acknowledge it.