I temp banned myself with the API (using BmAPI) with a timestamp set. In chat I saw "[...] has been temporarily banned for 23 hours by Console for test", and I got kicked with the message "You have been permanently banned."

You must handle kicking the player if they are online.

https://github.com/BanManagement/BanManager/blob/master/src/main/java/me/confuser/banmanager/BmAPI.java#L82

The kick message will have come from the BanSync scheduler. You need to handle the kicking yourself if you wish to set a different message.