Just an update to what I put earlier. I did a full scene optimization (with all the trimmings) and that has seamed to have done the trick.

So now it's up to me to crank up the settings as much as possible and keep an eye on it before I trust it to do it on it's own again.

Any ideas why that could have solved the problem as I am still curious about the whole thing.