If you want to delete it imediately after creating it you can hit backspace, if it's an already existing line right click on the object and select edge mode by clicking on edge at the top of the marking menu, then select the line you want to delete and go to the edit polygons tab and select delete edge.
Hope that helps.