i'm afraid that's exactly what you will have to do, unless you do it in post and just fade it out in aftereffects or shake etc. The other option i guess is to do a set driven key or connect them all to one attribute (so they all share the same value) and just key one of them.

Alan