An article in Crain's Chicago Business posits the existence of the "Mary Poppins Effect," in which people who dress up a little when they ride their bikes, as opposed to wearing bike or athletic gear, experience better treatment from drivers. "It helps drivers realize bicyclists are people too," Dottie Brackett of Let's Go Ride a Bike is quoted as saying. Other items that may help include helmets and child trailers (even if they aren't being used to cart infants around).
The implication is that drivers think cyclists who look like messengers or Lance Armstrong are menaces on the road, and feel entitled to treat them as such. In a discussion about the article on the Chainlink, a locally-based site for cyclists (disclaimer: I contributed a comment to the thread), a user named Carly noted, "I've been riding a tricycle for the past month because Im [sic] recovering from an accident, and the wide berth of my vehicle has never angered a motorist. When I rode my agile little single speed though, I was constantly harassed."
It's no surprise that what you wear can affect how others treat you. But it's one thing to be judged by a snobby salesperson or maitre d'. It's another when it's someone behind the wheel of a large, heavy, fast-moving vehicle.