I get that frequently too. The way I "fix" it is to go back into component mode, select everything, delesect everything, and it usually gets deselected the second time around. Definitely annoying though.