This is the effect of supersampling on the noise. I increased the scattering of the light (by 2 times) and added 2 samples (5 per pixel) supersampling.

Of course, it took way longer to render, and the quality is about the same as if I had increased the shadow rays to 81. But I'm sure the 81 samples would take longer to render.