I was able to replicate the issue I think, and interestingly it goes away if just rewind the playback. Give that a try.

Proxies are typically used to create lower res stand-in geometry to speed up simulation time, but you can use the same thing to get your transformations working. Simply create another object the same size and shape as your finger, move it into place and then create a parent constraint from the original geometry to keep it in the same place when the finger is animated. There should be some info online for doing the parent constraint.