Thanks for verifying this. Unfortunately connecting the circle to the curve creates double transforms. I've tried selecting them both and rotating them together, but that didn't work.
Interestingly though, on a whim I put an orient constraint from the curve to the circle and that does seem to keep it in check. Not sure what the difference is.

However, I'm still not sure if that will work for me, as I'm basically simulating a dynamic curve, so the orient constraint won't be updating any rotation values.

I wonder if I can somehow query the relative rotation of the first two CVs? I'm trying to remember if there is a way to constrain a transform (like a locator) to a CV? The opposite of a cluster deformer.