Yay, I found out something!

After connecting the height field to the shader group (if I remember correctly), I check this setting in the objects attribute editor.

"Feature displacement"

This way, you can create a displacement on a single polygon. And the result is fantastic!

Ooops, seems like Kbrown already said so....

Btw, the manual "states" that the extra sample rate should be set lower than the initial sample rate.