If you have an External Storage, Importer or Exporter and break it, any Filter inside will be destroyed instead of dropped.

• Minecraft: 1.12.2
• Forge: 14.23.1.2582
• Refined Storage: 1.5.31

Edit: As a suggestion, using the Filter inside any of these blocks should also create a shadow instead of moving the item inside. The downside is that you would not need to create another Filter again, only resetting the same one, but the recipe cost should be increased I suppose.

Related to how this works if you put an Filter inside any of these blocks and replace with any other item, the real item will be spent instead of creating a shadow. If you click to remove it, the item is lost.

That is intended.