This usually happens regardless of how big the pic is. The easiest way to go is to simply create image planes and map your references on there. But, if you want to keep your current files, you can put them in a layer and make them invisible when not in use.
"The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man."
George Bernard Shaw - Man and Superman