if you're using specular, you'll also need a specular map so that it's only specular where the hair is, similar to the thransparancy map.

Also, for accurate shadows, you'll need to use raytracing.