I think non anime fans have only seen a couple of animes, then don't like any anime based off a couple animes they saw. I think this is mostly cause they have only heard of mainstream animes like Naruto, DBZ, one piece, or SAO. Which I would agree with them if those were the only animes I watched, I would hate anime also. Theres a lot of adult themed animes, and animes that adults could watch and like; like cowboy bebop, one punch man, jojo's bizarre adventure.
Or like my sister who likes to watch animes that are for girls, like Lucky Star, melancholy of Haruhi Suzumiya, or fruits basket
Recommending only main stream anime just seems like a bad idea for some people.