Apart from the things already mentioned, I'd say some people thing that anime fans run around with their arms behind their backs like in naruto and contantly scream some japanese words, which might happen at a con (never been to one, so I can't say for sure). Another stereotype is the "weeb". I mean somehow it happened that everyone who likes anime is referred to weeb, while a weeb is a person absolutely and irrationally obsessed with Japanese culture to the point where they don't see any negatives and romanticise everything. While weebs commonly like anime, not everyone who likes anime, even the more hardcore fans, are weebs. But somehow that's the image many people have.
I think Anime in general has become more accepted with younger people, so the bad stereotypes often come from older people who have less experience with this. (Ofc, there are school students who dislike people for liking anime and stuff, but it has become, depending where you live, less frequent).
A lot of the negtive images these people have, partly have their origins with the media. Whenever there is an incident involving violence or murder or something like that, the media search something to blame for that, like video games, anime, the kind of music they listen to, the meal they ate on the 12th of may five years ago or whatever. And if this person had a lot of anime-merch or was some akward person who only ever talked about anime, it apparently has to be anime's fault. Some less professional media outlets, the only one in english I know is the Daily Mail, like to depict anime lovers as possible murderer and psychpaths...
Frankly, When I meet people my age, they either have seen some anime, especially as children, or just don't care. I mean, I don't go up to people and start to talk about random shit, my favourite waifu and stuff, but if I know them somewhat better and think they like me somewhat I usually mention it - I think we can break some sterotypes this way.