Loaded into a new world and as soon as I picked up a gem, I was given the "Captured" achievement, which, as far as I know, should only be triggered when getting a spawn egg.

You're right! Just had a fresh new world and the first item in my inventory triggered the achievement.

Created a fresh instance of Minecraft 1.21.1, NeoForge 21.1.93, Apotheosis 8.0.2 for testing. Received the "Captured!" advancement as soon as I get anything in my inventory, not just gems.