On the curseforge page it states
"Can specify which container to deliver by shift right clicking golem and then container with an empty hand"
I had to search through the issues on here to determine that you needed wheat in your hand to order them.

Can you reference the specific GitHub issue that states the thing about wheat?

@astynax777 I found the issues you mentioned. I made a pull request (#71) to update the Read Me on this GitHub repository. Unfortunately, as a lowly community member, I don't think I can do anything about the CurseForge page. Still, it's a start.