You can try leaving the global stitch history on there, but it'll probably be a bit sluggish.

Otherwise, you'd need to paint the weights of the joints being used so that the patches don't come apart during animation. Kinda tedious.