I think it's because hardware rendering struggles on shadows and you've got quite a few try rendering without any shadow and see what times you get.

Edit
Forget that
I thought the image above was the one by dragonfx. woops