For me I would try to give it some colour in those areas (I.e not totaly black in the areas) just to see if that works, as you've got no displacement there (as you said) so its just ignoring those areas and keeping them as they were.