I would just create a bump map that whay you can have a lot more controle over where you want the weathered parts to be and look.
I am enough of an artist to draw freely upon my imagination, knowledge is limited, imagination encircles the world. (Albert Einstein)
https://www.artstation.com/kurtb