i do believe its because your model is open if you get what i mean, because when i modelled a building i did it in pretty much the same way as you, try closing off any open faces, edges etc
of take a poly square, and extrude it to get the shape of your current building and boolean again, see if that works