haven't used xp64bit (heard it doesn't play well with maya) but is there a folder program files and program files (x86) that you can install to?
if there is make sure you install to program files
i think you can change the limit of memory you assign to a render in the render options
render > render current frame > 
render > batch render > 
in the box titled Memory
is that what you're looking for?
also, when i render i notice the ram doesn't shoot up but the processor speed does, perhaps your renders aren't memory hungry as you expect? *shrug*
that's a "Ch" pronounced as a "K"
Computer skills I should have:
Maya, XSI, Photoshop, AfterEffects, Motion, Illustrator, Flash, Swift3D