Don't really know if this is a problem or just have to do with my ownstupidity. Everything else works when I link it to the keybinds but XaeroWorldMap. It says it's bound and all but nothing happens when I try to use it. Tried with "toggle" as well just in case that was needed (like I said, kinda stupid).

menu.json (only added the map this time and still using stone as icon)

{
"main": {
"0": {
"title": "Open Map",
"icon": {
"name": "minecraft:stone",
"damage": 0
},
"action": {
"key": {
"key": "gui.xaero_open_map",
"toggle": false
}
}
}
},
"Mapping": {}
}

Thanks for the report!
Making stuff like this would require a major rewrite of the mod, which I simply do not have time to do.