I sorta do the same with magic, almost never use damaging spells. I mostly stick to illusion, alteration, and conjuration. Love playing as an assassin who uses dark magic and poison to kill off enemies with little to no effort.
Speaking of conjuration, I love the ritual stone power, possibly my favorite one since it lets you revive everything asides from dragons lol. Nothing like walking into a bandit camp with an army of giants and mammoths.