Giving the permission bm.timelimit.playerBans.X where X is the name of the timeLimits: does not work.
The time limit works as intended, but anyone who inherits that moderator group, is also limited to time restraints even though they have the bm.timelimit.playerBans.X and bm.timelimit.playerMutes.X

It's permission based. If you have group inheritance, you must either give the user the permission to bypass the limit, e.g. bm.timelimit.playerBans.bypass or negate the permission in question.