It almost looks as though your cloth object is not tesselated enough maybe? so it would then break, and loose the constraign? ..hmmm, not sure here....