You probably don't have enough CVs on your profile curve, or if you do, they're not spaced evenly so the extrude gets all wonky. Go to "rebuild curves" and play with the span settings. Or, if you want start from scratch without having to rebuild, when using the CV tool, snap to grid while drawing the curve and place a CV at every "grid box" so the points are even. Also, use "profile normal" when extruding like Mike suggested.