Thanks for the comments. I've had some limited success after some research. Things I've changed are:

I am now using a 32bit displacement map exported as tiff and then converted to a .map file for mental ray to read. This works much better I think.

Imported a smaller mesh and using 2048x2048 map still gives me higher defintion. I think this was the main problem. Here are some images.

mf3