If Configuration is loaded and you run a datagen, then the jvm will not exit.

I am not 100% sure that this issue occurs with Configuration, as I have not tested it. I encountered this issue when using Azurelib, and the comment at the top of the file indicated it used a fork of this mod, so I'm opening the issue here as well.

The reason it's not exiting is due to the "Auto-Sync thread" thread:
https://github.com/Toma1O6/Configuration/blob/b440d1af2cc113891904cc54e26d04a6328499a0/common/src/main/java/dev/toma/configuration/config/io/FileWatchManager.java#L34-L28

I believe this is because the SERVER_STOPPED event is not emitted when exiting from the datagen:

The solution to this would be to either use some other method of cleaning up the executor and/or marking the thread as a daemon thread, allowing the jvm to exit even if it has not been properly cleaned up. The preferred option would be both.

Please see AzureDoom/AzureLib#62 for more info.

Marking the thread as daemon thread makes the most sense, not sure why it was not done in the first place to be honest. However it also seems to be partially Fabric fault as I have not noticed any other datagen having issues with it

However it also seems to be partially Fabric fault as I have not noticed any other datagen having issues with it

This is not the fault of fabric at all.
According to FabricMC/fabric-loom#1145,

Neither of these [ServerLifecycleEvents.SERVER_STARTING and ServerLifecycleEvents.SERVER_STOPPING] should be fired during data generation.

the use of ServerLifecycleEvents.SERVER_STOPPING is incorrect, as it is not expected to be fired during datagen.

Since the thread is now a daemon thread this issue should be resolved. As it affected only datageneration, it does not have large impact on the userbase and therefore this change should be sufficient.
Fix will be available in versions 3.0 and newer which should be probably released within this week