So to further that thought, if you do stray away from your line (on the waste side, of course

) I find that it is better to move the blade back and try to skim the line again, moving very slowly into the cut, to establish the correct track again.
Doesn't always make a bad situation better, but it avoids the tendency to push sideways.