If you dismount while the horse is still running you and the horse will stop dead like you would expect. However when you mount the horse again it continues running at the same speed as before you dismounted. Ideally the speed of the horse would be forgotten when you dismount and when you mount again the horse stays still.

Horses in vanilla Minecraft have no acceleration mechanic, like any entity not experiencing gravity other than minecarts, so it saving last speed is useful for leaving your horse ready to take off on mount. This should be an option, else the controls mod meant for combat is no longer ideal for initial movement compared to vanilla movement.