I'm running this serverside only on Limitless 2.0 (https://www.curseforge.com/minecraft/modpacks/tnp-limitless-2). I did some testing before I started editing the config to include all of the modded foods. Wheat and potatoes is not giving me any experience. I know that it's supposed to just update your EXP without any orbs dropping. I verified by dumping my entire experience off into a global xp block. Everything is set by default in config except for turning on curiosity mode (again as part of testing). I wanted to be able to verify the potato/wheat blocks matched the configs. They do.

Looking through the server log. I am seeing this.

https://gist.github.com/mrdeadlocked/a95ca6a90b1ab859f640873735ce67b9

According to log there's problem with other mod:
at enemeez.simplefarming.events.harvest.standard.CropHarvest.onCropHarvest(CropHarvest.java:76)
Try disabling simplefarming mod and see what'll happen.

So, Simple Farming implements it's own "Simple Harvest" feature that needs to be disabled. Open simplefarming.toml and disable it:

	#Indicates whether right-click harvesting crops should be enabled. Respective parts of the right-clicking function can be enabled/disabled below.
	"Enable right-click harvesting for crops" = false

Now open XPFromHarvest.toml and add all desired from Simple Farming crops into "Crops List" (eg."Block{simplefarming:barley_crop}[age=7]"):

#List of crops to process with their ages
"Crops List" = ["Block{minecraft:potatoes}[age=7]", "Block{minecraft:carrots}[age=7]", "Block{minecraft:wheat}[age=7]", "Block{minecraft:beetroots}[age=3]", "Block{minecraft:nether_wart}[age=3]", "Block{simplefarming:barley_crop}[age=7]"]

To see what actual values you need to add you can use "Curiosity Mode" in XPFH, sneak right clicking on desired crop will send info about it into chat, which you can then just copy/paste from MC log:

[Render thread/INFO] [minecraft/NewChatGui]: [CHAT] Block{simplefarming:barley_crop}[age=7]

Also contact modpack developers so they can fix it in next release.

I did what you suggested and its working partially. My test was just vanilla wheat in my field. My console still gives this error occasionally and I get no exp. It seems to be only if I have an empty hand that I dont get exp.

Server thread/FATAL 11:02:53
Error executing task on Server
Console 11:02:53
java.util.ConcurrentModificationException: null
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909) ~[?:1.8.0_211] {}
at java.util.ArrayList$Itr.next(ArrayList.java:859) ~[?:1.8.0_211] {}
at com.realgecko.xpfromharvest.SimpleHarvestHandler.handleHarvest(SimpleHarvestHandler.java:44) ~[?:?] {re:classloading}
at com.realgecko.xpfromharvest.SimpleHarvestHandler.handleRightClick(SimpleHarvestHandler.java:36) ~[?:?] {re:classloading}
at net.minecraftforge.eventbus.ASMEventHandler_1697_SimpleHarvestHandler_handleRightClick_RightClickBlock.invoke(.dynamic) ~[?:?] {}
at net.minecraftforge.eventbus.ASMEventHandler.invoke(ASMEventHandler.java:80) ~[eventbus-2.2.0-service.jar:?] {}
at net.minecraftforge.eventbus.EventBus.post(EventBus.java:258) ~[eventbus-2.2.0-service.jar:?] {}
at net.minecraftforge.common.ForgeHooks.onRightClickBlock(ForgeHooks.java:785) ~[?:?] {re:classloading}
at net.minecraft.server.management.PlayerInteractionManager.func_219441_a(PlayerInteractionManager.java:315) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,pl:mixin:APP:kubejs.mixins.json:PlayerInteractionManagerMixin,pl:mixin:A}
at net.minecraft.network.play.ServerPlayNetHandler.func_184337_a(ServerPlayNetHandler.java:853) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,xf:fml:astralsorcery:reach_set_server_entity_interact,xf:fml:astralsorcery:allow_entity_interaction,xf:fml:randompatches:RandomPatches ServerPlayNetHandler Transformer,xf:fml:backpacked:creative_inventory_action_fix,xf:fml:materialmaster:server_play_net_handler_patch,pl:mixin:APP:mixins.dcintegration.json:MixinNetHandlerPlayServer,pl:mixin:A}
at net.minecraft.network.play.client.CPlayerTryUseItemOnBlockPacket.func_148833_a(SourceFile:36) ~[?:?] {re:classloading}
at net.minecraft.network.play.client.CPlayerTryUseItemOnBlockPacket.func_148833_a(SourceFile:10) ~[?:?] {re:classloading}
at net.minecraft.network.PacketThreadUtil.func_225383_a(SourceFile:21) ~[?:?] {re:classloading}
at net.minecraft.util.concurrent.TickDelayedTask.run(SourceFile:18) ~[?:?] {re:classloading}
at net.minecraft.util.concurrent.ThreadTaskExecutor.func_213166_h(SourceFile:144) [?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
at net.minecraft.util.concurrent.RecursiveEventLoop.func_213166_h(SourceFile:23) [?:?] {re:mixin,re:classloading}
at net.minecraft.server.MinecraftServer.func_213166_h(MinecraftServer.java:731) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.server.MinecraftServer.func_213166_h(MinecraftServer.java:141) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.util.concurrent.ThreadTaskExecutor.func_213168_p(SourceFile:118) [?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
at net.minecraft.server.MinecraftServer.func_213205_aW(MinecraftServer.java:714) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.server.MinecraftServer.func_213168_p(MinecraftServer.java:708) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.util.concurrent.ThreadTaskExecutor.func_213161_c(SourceFile:127) [?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
at net.minecraft.server.MinecraftServer.func_213202_o(MinecraftServer.java:694) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:641) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_211] {}

I see more than one mod implementing right click harvesting in this modpack. In order for XPFH to handle simple harvest you need to disable it in all other mods. Quick look at config files reveals it in:

simplefarming.toml
pickletweaks-common.toml
pamhc2crops.toml
quark-common.toml

I wonder how this modpack even works with all this handlers doing same job enabled.

Alright. I disabled all of the simpleharvest/rightclick to harvest options in those configs. I am still getting issues where with your mods right click to harvest enabled we get console spam and it doesn't always harvest.

java.util.ConcurrentModificationException: null
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911) ~[?:1.8.0_265] {}
at java.util.ArrayList$Itr.next(ArrayList.java:861) ~[?:1.8.0_265] {}
at com.realgecko.xpfromharvest.SimpleHarvestHandler.handleHarvest(SimpleHarvestHandler.java:44) ~[?:?] {re:classloading}
at com.realgecko.xpfromharvest.SimpleHarvestHandler.handleRightClick(SimpleHarvestHandler.java:36) ~[?:?] {re:classloading}
at net.minecraftforge.eventbus.ASMEventHandler_1693_SimpleHarvestHandler_handleRightClick_RightClickBlock.invoke(.dynamic) ~[?:?] {}
at net.minecraftforge.eventbus.ASMEventHandler.invoke(ASMEventHandler.java:80) ~[eventbus-2.2.0-service.jar:?] {}
at net.minecraftforge.eventbus.EventBus.post(EventBus.java:258) ~[eventbus-2.2.0-service.jar:?] {}
at net.minecraftforge.common.ForgeHooks.onRightClickBlock(ForgeHooks.java:785) ~[?:?] {re:classloading}
at net.minecraft.server.management.PlayerInteractionManager.func_219441_a(PlayerInteractionManager.java:315) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,pl:mixin:APP:kubejs.mixins.json:PlayerInteractionManagerMixin,pl:mixin:A}
at net.minecraft.network.play.ServerPlayNetHandler.func_184337_a(ServerPlayNetHandler.java:853) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,xf:fml:astralsorcery:reach_set_server_entity_interact,xf:fml:astralsorcery:allow_entity_interaction,xf:fml:randompatches:RandomPatches ServerPlayNetHandler Transformer,xf:fml:backpacked:creative_inventory_action_fix,xf:fml:materialmaster:server_play_net_handler_patch,pl:mixin:APP:mixins.dcintegration.json:MixinNetHandlerPlayServer,pl:mixin:A}
at net.minecraft.network.play.client.CPlayerTryUseItemOnBlockPacket.func_148833_a(SourceFile:36) ~[?:?] {re:classloading}
at net.minecraft.network.play.client.CPlayerTryUseItemOnBlockPacket.func_148833_a(SourceFile:10) ~[?:?] {re:classloading}
at net.minecraft.network.PacketThreadUtil.func_225383_a(SourceFile:21) ~[?:?] {re:classloading}
at net.minecraft.util.concurrent.TickDelayedTask.run(SourceFile:18) ~[?:?] {re:classloading}
at net.minecraft.util.concurrent.ThreadTaskExecutor.func_213166_h(SourceFile:144) [?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
at net.minecraft.util.concurrent.RecursiveEventLoop.func_213166_h(SourceFile:23) [?:?] {re:mixin,re:classloading}
at net.minecraft.server.MinecraftServer.func_213166_h(MinecraftServer.java:731) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.server.MinecraftServer.func_213166_h(MinecraftServer.java:141) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.util.concurrent.ThreadTaskExecutor.func_213168_p(SourceFile:118) [?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
at net.minecraft.server.MinecraftServer.func_213205_aW(MinecraftServer.java:714) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.server.MinecraftServer.func_213168_p(MinecraftServer.java:708) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.util.concurrent.ThreadTaskExecutor.func_213161_c(SourceFile:127) [?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
at net.minecraft.server.MinecraftServer.func_213202_o(MinecraftServer.java:694) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:641) [?:?] {re:classloading,pl:accesstransformer:B,re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,pl:mixin:A,pl:runtimedistcleaner:A}
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_265] {}
Server thread/INFO

I appreciate the help you've given. I might just forgo using this mod for now. Maybe its some weird interaction with another mod. I just liked the idea of XP from farming.

Looks like there're too many right click handlers in this modpack. You can still use this mod without quick harvesting as breaking mature crops (like you do in unmodded MC) will still give you XP.