With the diffuse map I'd be tempted to use a combination of the bump (due to less colour leaving the deeper recesses) and a desaturated version of your colour map. I'd experiment with either the level settings, brightness and contrast to achieve a nice balance before any extra painting on the map.

Sorry to hear about the texture swimming, that's a real pain. Hope you get it sorted without to much aggravation.