It's probably a mix of "you bought foreign!?" and resentment that they're paying so much more for gas than we are.
Meanwhile, I think I made a hater out of the driver of a Ford pickup truck the other day.... we were beside each other as the light turned green, and his tires actually squealed as we both started off, but my Fit's tiny 1.5 L engine easily pulled ahead of him as the two lanes merged into one.
Okay maybe there's one other reason people hate the Fit--the owners are sometimes smug bastards