I built a farm where I fenced the farm in, but put the container in the corner (I used both barrels and chest) in 1.19 Fabric. After a short period of time, the golem would race back and forth along the edge taking about .5 seconds to go corner to corner and would do this constantly. The only way to stop this was to put the container in the center of the farm.