When you activate elytra mid-dive, the dive's directional momentum is carried onto the elytra gliding. If you dive, activate elytra, and turn 90º to the right, you should be going straight but instead you go to the left a bit as well. It appears using a rocket cancels it during the duration of the rocket's power, but the directional force returns as soon as the rocket gives out.