What do you think about addin biome Control?

There is WaterControlExtreme for that, but that does not allowe to configure per fluid, but only in General. So i can not say Lava and water is infinite on Dimension X, but Lava is finite on Z, while water is not.

So i'd like to use your mod, but on overworld i would like to have water infinite in oceans, rivers and beaches.

Does using InfiniteFluids and WaterControlExtreme together not already do what you want? Make sure that anything you want to be "sometimes infinite" is set to infinite in InfiniteFluids, and it should just work. Let me know whether this works for you.