press shift and select other vertex you want and they will get added to your selection, but if while pressing shift you go over the ones that are already selected they will get deselected.

also, by pressing ctrl+shift and clicking, it doesn't matter if you go over the ones that are already selected they won't get deselected.