actually I was just taking a closer look at your render and it may not be the matte at all. make sure in your global renders in the paint effects section turn on oversampling on paint effects on. This should fix some of the antialiasing happening.