baritone sometimes breaks blocks that are in a schematic and it doesn't place them back. Or it sometimes goes in an infinite loop breaking and replacing the same few blocks

Place-break loops, improper handling of certain blocks, bad world scanning and generally bad building order are all known problems of the builder.