Ah... ok... I didnt see the last post before writing...

The hole for the head/neck is to huge. When its too big in relation to your model, you will get this kind of result because of the gravity.

And what it this.. does he have boobs or is it breastmuscles ?? hehe. Still need to play with the collision offset and so on. BTW.. you dont actually need to make the cloth relax before you stop it - if youre happier with an earlier result...

But comming along real nicely. Sure hope you can find the collision offset in the channel box.