I have to set it so that everyone can interact with blocks in my claim instead of just my party. If I don't, then my party can't interact with blocks at all. So I either have nobody interact with my base, or let everyone, including enemies, interact with my base. Im not sure how to fix this either.

There are separate options for all players and just your party members, so I have no idea what your problem is.