We should cache list of items/blocks instead of loading them each time we access it.

This will mainly apply to the whitelists and blacklists and should improve a bit performances.
These only need to be refreshed when the config is reloaded/updated.