If you set the "envelope" setting on the ffd that is assigned to the lattice (check for the ffd node in the attributes editor) to 0 that should do what you're asking.

or

You can deplicate your model and have a version affected by the lattice and one that isn't, and then create a blend shape to animate between the 2.