Management, at least as far as I can tell, is mostly a matter of being able to stay very organized. In an art department, it tends to be mostly a matter of scheduling and making sure the artists in your team are meeting those schedule requirements. The higher up the ladder you go, the more people you have to manage and be held accountable for. If Art Peon #5 missed his deadline, the CEO is going to look at the Art Director as to why this happend.

Personally, I don't see myself being in an Art Director position in my career. Maybe a Senior Artist or a Lead Artist. I could see manageing a small sub-team, but not the whole department.