Yeah that isn't a lot of RAM at all, you're running an OS, running Maya then rendering with Mental Ray, sooner or later something is going to crash. Also, you should look into command line rendering.