In main.cfg, I have the following:

toolleveling {
...
# The amount of modifiers new tools have. [range: 0 ~ 9, default: 0]
I:ExtraModifiers=0
...
}

This works for all regular tools except for arrows, they start with 3 modifiers.

Just noticed that this is the case because Arrows don't get levels. Will probably be changed with the update then.