When I set hit/expertise soft cap for melee character and the gear itself just doesn't have stats to reforge around, instead of going AS CLOSE to the caps as possible, the addon just outright ignores those and selects the second/third priorities outright altogether OR even removes that small amount the gear already has.

In case of Feral Druid (tank), if I can't reach the expertise rating I set as "at least", it tries to reforge the remaining expertise on gear all the way to Dodge rating, instead of keeping some.

Is that the normal behavior or I should tweak something within the addon?

I noticed this with hit rating when my guild mates gear was not capable of hitting cap; even with reforging all possible items to add hit. I'd just say the addon should tell you that it's not possible. The workaround is to just set a lower value until you get better gear. I think the solution would be just a note/warning when this happens. Since this addon is getting popular and this confused my guild mate when it told him to remove all his hit rating.

this confused my guild mate when it told him to remove all his hit rating.

Exactly what happened! Super weird function imo.

I can confirm this, but I don't know what leads into this and it rarely works as expected. I usually try to re-equip my gear, reload, restart the game and sometimes it works.
As stated above, it just plain ignores the set caps and doesn't work towards them. Currently, although my Mastery is set to -1 and my Crit to 3 (Fire Mage), it suggests removing my Mastery > Crit reforge on the Volcano card.

https://imgur.com/a/H5FoBqt

As you can see, I set my hit cap to 615 (Level 87 Mobs) and it works (before suggesting this strange Mastery > Crit reforge). I only did this due to nonsense suggestions when going for the usual hitcap (1742).
On my Rogue it works most of the time (Expertise 781, Hit 1332), but even there sometimes it goes crazy.

Edit: Desparately trying something different, I switched to Sub Rogue Profile (on my Mage), hit reforge and it did some reforging (into Expertise etc.). So nothing unexpected. I switched back to Fire Mage 15 % Haste, calculated, and it didn't do anything. It does show a Lua error, though: Interface/AddOns/ReforgeLite/ReforgeEngine.lua:466: script ran too long, which seems to be one part of the problem. I set the slider a bit more towards 20, then it went without an error but this time again disregarded my spell hit cap (1742). Setting my spell hit value to 5 (greater than everything else) manually seems to solve the problem.