What I am getting is something that looks like if I was trying to be on two places at once. One frame renders from original location, another from destination location. The cycle never ends, the only way to stop it is to relog. Then I am back in original position, still mounted.

What version?

I tried to narrow it down, I used it with a custom-made pack and on server.

Now, a clean setup: a forge server forge-1.12.1-14.22.1.2478-universal.jar

Mod:
"version": "4.0.15",
"mcversion": "1.12.1",

Behavior: It is impossible to teleport when on a horse. There is no cycling back and forth, so it may have been artifact from other mod.

Is the lack of possibility to teleport mounted intended behavior? If yes, this ticket can be closed.

Okay, I looked into the code quickly. The code says player should dismount.

Dismount doesn't happen somehow.