First, thanks for the tris tip. That makes sense.

2nd: in deleting those extra loops I'll have to be careful that edge loops can still be added properly.

3rd: I don't have a poly count limit to worry about except for my hardware, which according to your specs should be more than adequate...

Athlon 64 X2 @ 2 GHz
3 Gb RAM
Quadro FX 1700 (softmoded GeForce )

I'll save as, and then delete those faces you're talking about, see if my loops still hold, and then I'll add windows, and then add the deformation details normal for a castle.