just extract the faces or walls of the building, and do the booleans, then combine it back into the object..