If you wish to render paint effects with Mental Ray, you need to convert trees, grass, whatever you have created into polygons first. You can do this by selecting your paint effects stroke and going

modify -> convet -> paint effects to polygons

You may need to limit the amount of polys it uses though, as converting paint effects can result in very heavy meshes.

LisaG