I took this on one of my creative servers which had around 400 players and noticed that PEX was showing up in my timings taking a lot of resources, particularly on the login stuff. Here's a timings report: https://timings.aikar.co/v2/?id=3e96d048cd454019883fcba06de3dbd9&start=1469497988&end=1469498242&section=lagsummary
I see that pex is using recalc as a way to bulk modify but I'm guessing recalcing on login and join events is causing issues here?
Here's my permissions file shortened: http://hastebin.com/cuvuwemoga.sm

Yeah, this is kinda unavoidable with PEX 1.x. Performance is improved in 2.x, but not all features are implemented and commands work differently.

Should I stop using inheritance as a temporary solution? or will that not change anything?

That probably won't change anything.