This is due to a longstanding bug in vanilla where observers fire when a structure is placed. The only real workaround it to add more potions to the dispenser or not use observers

Added more potions to dispenser. I could have changed the trap to not include observers, but then the best player detector I could do would be a puffer-fish/tripwire trap, which would require the player to be very close to it.