Since version 2015(?) Maya has had a UV tile mode implemented on file nodes so the manual method isn't needed. I've never used the layered texture in a UV tile setup, I just tried it and it didn't work. The method I've used is, after setting the appropriate translate frame values and unchecking wrap U/V as you did, I plug each texture into the default color swatch (under the texture's color balance section) of the previous. The default color is what fills the empty space when a texture doesn't wrap along U/V. A couple downsides to this, the swatches render in 0-1 space so the textures outside of that won't have have very descriptive thumbnails and the UV texture editor will not display all those textures at the same time unlike with the "official" UV tile mode.