Hi,
currently, the fml_cache_annotations is only in the main jar, but it contains the cache for all the jars. This means that when the json class scanning is used, the cache is read, and FML finds references to all mod annotations, but then the other jars are scanned the old way (because the have no annotation cache) and will also report their mod class. Because of this, FML will think that the modid is duplicate, and will not continue loading.
EDIT: I just heard back from Lex, he saif it's your problem

That's because the module jar's are generated from the one source set, This is definitely a bug on PR's end.

This should be fixed now if it's still relevant. Versions for MC 1.15+ have source sets properly split and each module in a separate jar file.