None of the higher subd levels gives any better results, which means that its not a problem with the mesh itself but the normal maps. I shouldn't have to up my polycount by 4X or 16X to get decent looking normal maps anyways.

Does it matter if I was working at the highest level in ZBrush? Doesn't the high res detail get transferred down to the lower subD levels as I add it in?