Might need to unparent, ungroup everything, and then just give it one group. When you have multiple objects parented, grouped, etc, things can get funky sometimes.