Post up a picture... Its sounds like some of the points on the brows where tweaked or moved when making the shapes of the mouth for example.
Is the distrotion really bad? Because if you think about a face while talking a lot of points move all over.
I am enough of an artist to draw freely upon my imagination, knowledge is limited, imagination encircles the world. (Albert Einstein)
https://www.artstation.com/kurtb