I'm not aware of many people who hate libertarians except other libertarians. For the most part we're irrelevant to everyone else, and when they do react it's dismissively and based on a caricature of what they've been told libertarianism is.

When you do get a more passionate reaction, it's probably because a lot of libertarians are insufferable human beings and horrible communicators. For example, there was recently a thread on Facebook I saw about some proposed rent control in NY, and the same libertarian in one post who explained how thoroughly politicians, property developers, and landlords collude to make housing more scarce and more expensive to enrich themselves, and how perverse incentives lead even those who aren't involved to build more luxury housing than units normal people could afford, started calling people socialists when they pointed out that if those people were enriching themselves at the expense of everyone else, it makes no sense to claim a law putting a roof on rent hikes would necessarily tank the market.

Basically libertarians point to the crony class, tell everyone else how this class is robbing them blind, and then when the people being lectured, as any normal and reasonable people would, react by asking things like, "well then they probably could afford to pay their employees more, so why is that so bad," or, "if they're so undeservedly rich why the hell can't I take some back by force," or, "then why the hell is it such an imposition that I get more than three paid days off a year," etc., libertarians then turn around and call them socialists, tell them to learn Econ 101, and walk away feeling smug. They insist everyone analyze every situation as if we are currently living in a pure free market and anything at issue is the first and only proposed deviation from the purity that's ever been brought forth.

For all practical purposes the libertarian position is that you're getting robbed blind by the state and its cronies, and any response from you that isn't perfectly in line with the strictest interpretation of the non aggression principle and pure free market economic theory qualifies you as evil and a socialist leftist traitor to the cause. Basically, as long as the mafioso that benefits from the protection racket and the person who breaks yours knees if you don't pay are different people, you can't do anything about it.

That doesn't sell well.