when I tried to delete the group it deleted all the objects

I'll try the middle mouse drag

edit: Middle mouse drag worked great. I had to change from sort alphabetically to sort by hierarchy