Unlike crimson and warped stems, their corresponding small stems can be used as fuel:

This is because they are not in the minecraft:non_flammable_wood item tag.