I'm using BetterDexRewards and I tried to delete the default config file to use the custom rewards that I've created, but it's being recreated at every server restart and that causes unwanted reminders to be sent to all the players in the server very often.

I apologize in advance if my English isn't the best, as I'm not a native speaker.

This is intended behaviour. It will only generate when a file with that name does not exist, so just edit the existing files.