Well, what I ended up doing was keframing it off when it was on the opposite side, but as you said, it IS annoying.