The first thing I look for to on a new anime is the animation. Since it's the first thing that we can see and judge, I consider it quite important for my enjoyment. If it's something too bad or simple, I don't expect to get anything that good.
But the thing that can take me off completely from a show it's the story and character development. I love to watch a good show, which has complex concepts and develop them, as well as characters who grow along the series.
When a director only makes its job to say it was done, and basically calls the watcher a stupid dumbass, I just drop it. It doesn't deserve my time.