yes model the building
u want to duplicate it like 2x so u have 3 copies
then create geometry (hires) use sclupt and push and pull verts to make multiple pieces that will come off
the usin that and ur dups
doo booleans intercept and difference
so now u have a whole house, a house with bits missing and the chunks that have the same thick ness as the walls
use shatter effect on the chunks

add gravity, newton and animate this to act when u want
use a radial field to 'blast' that section away or fall
animate visibilities of the pieces of geo

hope this helps