I used an evga 8800 gtx on a 32bit xp system for a very long time running maya 7 and 8. So I am not sure what the problem is. I can say that if you are running the 32bit version of windows 7 the extra 2gb of memory will have marginal effect. The most memory a program can use in a 32bit os is 2gb (well 3gb in theory but extended memory has been wonky in my experience). That being said you should now have a full 2gb available to maya and the remainder for the os and other apps.

You might want to check to see what you have running in the background. I have a friend that has an 8800 and plays wow a lot and he was also running unfolding at home and seti at home in the background, both of which make heavy use of the GPU in the graphics card and was causing his system to hang whenever he played wow. You could have something similar going on.

Also, just another note - In my experience I have found the nvideo drivers to be more stable then ATI drivers with Maya, but that does not mean they are always perfect, for instance the nvidia 260.99 drivers deprecated support for high quality rendering and view port 2.0 support in Maya. The new 266.58 drivers seem to have fixed that. The point being, that you have to be careful with the version of your video drivers because technically Maya does not guarantee non-workstation cards to work 100% with Maya and the drivers some times will break compatibility. You might want to use google to find the version of the drivers that will give you the best compatibility with your 8800 and the version of Maya you are using.