Capabilities since 1.14.4 have the ability to be cached.
That means the owner of the Capabilities needs to invalidate them.
Forge doesn't do that automatically for you.

This basically allows a drawer slave to be used even if it isn't connected anymore, if someone decides to use the cache.
A simple fix:
On TileEntity unload call the LazyOptional.invalidate function which triggers all listeners that cached the capability.

Thanks supporting this should help optimizing inventory accessing blocks a lot.

Added invalidation as a part of this PR: #1016