I managed to get the interaction that I am seeking with 2 ncloth both set to be quite rigid, altho I cannot seem to get it to work with a Passive collider tho....

Tabletennisball bouncing correctly - YouTube