My guess is that it's the final gather that is killing your render time. you could render an occlusion pass and comp it instead of FG.
