1.) Vesion: 5.0.4-SNAPSHOT.
2.) Spigot 1.14.4.
3.) The items are added to the inventory with a slight delay on player join, so it causes that you can see, for a moment, that the items are missing

Uhhh 5.0.5-SNAPSHOT doesn't exist?

Also, the delay is intentional, tweaking the items-Delay in the items.yml will make the delay longer or shorter. So simply setting it to zero will make the delay disappear.

@RockinChaos Sorry, I was wrong, it's 5.0.4-SNAPSHOT, however I've set items-Delay to 0 but that's the same

Okay so, I have done some digging and there are a couple of things.
1.) After testing, setting the items-Delay to zero does, in fact, give the items instantly (code-wise which I was able to confirm with timings and a awesome debug window), there will always be a minor delay that the players will see because that is the delay which the client takes to render the updated inventory (this has to do with the Mojang side of things). (It is also why if you relog it will not flash an empty inventory since items already exist and it was already pre-rendered).

2.) The ONLY time there will be a bigger delay is if you have Clear-Items ENABLED in the config.yml for any of the triggers such as join or world-change. This is because of the Delay-Tick, which is only active if ItemJoin is actually clearing the player's inventory. So what happens is if the Delay-Tick is a greater value than the items-Delay the items-Delay will be equal to the Delay-Tick + 1 tick. That way ItemJoin doesn't clear its own items haha. (As I do not have your config.yml I cannot see if this applies in your case but I thought it was some helpful info).

@RockinChaos that option for me is disabled (as default), I think that is because on player join there is a scheduled task that runs, and even if with 0 delay, makes the code executing the next tick

Heeeyy there! Nice eye, I actually thought the tasks were (not) being ran async but it turns out when taking another looks at it they are being run async which is actually causing it to run 10 ticks later (about half a second). There is a big difference when removing the task and letting it run. Looks like I have a job to do ;)

EDIT: Should be resolved in the latest snapshot; http://ci.craftationgaming.com/job/ItemJoin/288/
If the items-Delay is set to zero and there is no "clear delay" then the code will no longer run through a scheduled async task.

This bug has been officially fixed in the release of ItemJoin v5.0.4.
You can grab it here; https://www.spigotmc.org/resources/itemjoin.12661/download?version=295178

Please read the changelog as the documentation has not been updated yet;
https://github.com/RockinChaos/ItemJoin/wiki/Recent-Changes