I'm not at home, so I can't try it at the moment, but I believe you can group the lattice to the broom and that should do it.

I'll check when I get home.

Just remember, you'll need to select the group for everything to move together, so you might want to parent the group node to a more visible curve for easier selection.