depends on if you're using raytraced shadows or depth map shadows... unless you know how to tweak the settings, default depth map shadows tend to be a little blocky and jagged. raytraced shadows are usually super-perfect, which takes away from any realism, and also requires turning on raytracing while rendering (any surface that has the default setting of .5 for reflections will reflect, so watch out for that too) will slow down your render times depending on the amount of objects you have in your scene.

either way, yes. shadows will add to your render time.