Okay I got it figured out. (1) for single frame renders in maya 2009 under the render tab open render current frame options uncheck auto memory limit click on calculate This selects the soft memory limit for a single process based on the memory in your system. (2) for batch renders In my case I left everything default (ie auto memory limit checked) and batch render works great. In fact it renders significantly faster then a current frame render.