gster gave you a pretty good suggestion. You could just mask off an area for the shirt and inflate, or extract to a low thickness. If you maintain your nude base mesh, you can just paint weights on that and apply a wrap deformer on the clothes if need be.

With that, you may have to worry about the skin poking through the shirt. A few ways to get around that though. You could select all of the torso faces and apply a transparent material to them, you could duplicate the mesh, delete the torso faces and apply a wrap deformer to the new mesh and the shirt mesh. Then hide the driving smooth bound mesh.
Otherwise you can find a script to convert a wrap's weights to a smooth bind. It shouldn't be that hard so I'd imagine there are some out there.

there are a lot of ways to go, just choose one and see how it goes for you.