I remember trying that very same tutorial some time ago and having the exact same problem. I found the problem was the same for the lights and then the camera as well. From memory, the tutorial gets you to have the camera follow the rocket as well.

I spent ages back then playing around (basically ended up doing option 1 above). I have just spent some time playing with the old file and here are the things I had to do:

I basically aimed the light at the rocket manually,

then selected the rocket first and then the light,

selected Constrain/Aim options and set the following:

Aim Vector: 0 0 -1 (x,y,z)
Up Vector: 0 1 0 (x,y,z)
World Up Vector 0 1 0 (x,y,z).

Maybe this will work for you. I tried it this afternoon on all lights and camera and they followed the rocket perfectly. To make sure of this I selected each light and camera individually and set the perspective view to Look Through Selected, then played the anim.

Why these settings worked is beyond me. I even tried reading the help files!! but got lost in the jargon.