disp takes time to render with or without raytracing.
but, depending on your machine and how large your disp maps are, it should not take that long for a disp map to render.
i have a model with 5 disp maps on him and it takes about 1 min to see some test renders @640 resolution, and im running a 1.0 ghz PIII machine.
it really depends on how much disp you're rendering. you can lower the disp on shots away from the cam, and crank them back up for close up shots.