Mmm, honestly that just looks like you have the mesh set to display the smooth preview subdivisions, uncheck it and tone down the subdiv levels. If you actually had smoothed it, it wouldn't look so low res in the maya software render.

And you don't have mental ray in the drop down because you probably don't have it loaded.