After using the conductor's whistle for X number of times, the delay seems to go on for an unnaturally long period of time (at least an hour and counting as of writing this).

I think I had the same problem. It suddenly stopped working for me, even logging out and in the server again did not help. Fixed itself some time after that.

Was a player vs world tick time issue