Try to create the AO shader in a different way. For example:
1. Go to Windows->Rendering Editors->Hypershade
2. Create a Surface Shader
3. Go to Textures (left bar) and create mib_amb_occlusion node.
4. Connect out-value of AO node with out color of Surface Shader.
5. Increase the samples of your AO node to 64 (or higher). Select the objects you want to be rendered in AO and assign the new shader.
6. Render in Mental Ray.
Any questions?^^
Of course, AO is just a method to improve your final composition.