I'm going to preface this by saying I'm not an expert on optimization so correct me if I'm wrong. However, browsing spark profiler, accessories and tclayer are always among the top laggiest mods.

For Accessories, it seems LogUtils.getLogger() in AccessoryAttributeBuilder.java is causing a lot of lag. I'm honestly not sure why as it seems to be static and following best standards, but maybe it's being re-initiated every single tick? Either way, that much for logging seems overkill.

For tclayer, it seems to be the java.util collect in ImmutableDelegatingMap.java used by isEquipped in LivingEntityTrinketComponent.java, defined in WrappedTrinketComponent.java
I have no idea how this one works, but since its an API method, many other mods which uses it also becomes laggier.

@Boden-C The logger lag was fixed with update beta 43 for 1.20.1

The TCLayer code for 1.20.1 is a direct copy of the Trinkets API method found here so idk besides reworking such code to be more performant though in 1.21.1 and up the code uses native accessories method which is I believe faster but idk as I have yet to find other reports of issues.

Closing this as mostly resolved as stated a version was pushed to fix the issue and the lag from the method within TCLayer is a duplicate of the original code was removed in 1.21.1 but kept for compat reasons in 1.20.1

It seems modrinth did not upload the beta 43 version so download from curseforge https://www.curseforge.com/minecraft/mc-mods/accessories/files/6025902

The highest for fabric is only beta 41