Mental ray is a rendering engine.
When you want to render the image, every 3D software will pass the information of the scene to one of these engines. Usually it's a group of subroutines inside the software itself (like Maya software rendering, the standard engine of Maya), but you can also tell your software to "translate" the information for another engine that is a separate software. This is the case for Mental ray.
When you render with it, Maya converts all the data in the scene to the appropriate format and pass this to Mental ray, which produce the image or the animation.
Why using an external engine? Usually because they produce much better images than internal engines (or sometimes because they are faster).
The strength point of Mental ray is global illumination, with radiosity and hdri lighting.