80 track points seems like alot to me, i would usually track about 7 points minimum. They might be unnessesary which would throw off the camera solve. can you screen shot some of the tracks?

do you have access to another matchmoving app like pftrack or something?