i think that might give you double transform prblems down the line.
the way you did it was right before.

you just needed then to point constrain the new group to the hand manip [u can point or parent constrain it]