I just freed two Allays in 1.19 (using Independent Pets 0.8.1-1.19 with Fabric Loader 0.14.6), and the second they were free, they just ran right off and didn't return.

After searching for them for a minute, I looked into the NBT data and found they had traveled hundreds of blocks away from where they were free / where I was located.

Neither Allay had the tag "AllowedToFollow," but nor were Allays on the blocklist. So depending on the fallback behavior of NbtCompound.getBoolean(key_name), I could totally imagine that (!((Follower) tameable).isFollowing()) will evaluate to True by default.

Have you validated Allay behavior with IndyPets? If so, I could totally imagine that something else is afoot, but IndyPets was myt first most likely culprit.

I haven't actually looked at Allays at all until now, however, I'm certain that this mod couldn't mess with the Allay behavior since every Mixin in this mod only affects TameableEntitys and Allays are definitely not TameableEntitys (I checked the code).

To me this seems very much like a Vanilla bug.
Bees also have the high tendency to fly off into the distance when not caged, seemingly no matter how many flowers or bee's nests there are close by. There was this "horses sloowly slide towards a certain fixed point miles away" issue in the past as well, so I wouldn't be surprised if Allays had some similar issue.
On the other hand, I couldn't find an issue report for this.

Some time later I now did some testing with the Allay and also freed them from a Pillager Outpost and while one of them did kinda flee the scene by flying up a hill, they didn't really bugger off for good, never to be seen again, so I have to ask:

Are you sure what you're seeing is actually a bug?

Because Allays do not follow the player until the player gives them an item.
By default they wander around and they are not exactly slow either, so when you say "they just ran right off", how far are we actually speaking?

P.S.
Gonna close this issue for now, since I'm almost certain it's not caused by IndyPets, do let me know however if you find something!