I have tried to composite a modified shadow layer with the color pass. It seems to look better than the previous, although, a lot of Photoshop work!