There might be an easy way to fix this but Idk it. All villagers in the game are a lot smaller than me but everything else works. I made sure they weren't just children or teenagers. They are adults. This is on single player.

I think it's based on rng, but you can change their heights with a villager editor.