Ok, I found out that 'Shake' option for the camera was causing the issue. The displacement mapping doesn't have artifacts anymore. However I'm not sure why the shake option would do this.