There should be a way of disabling applied_to
When disabled it would do the damage boosting regardless of what is in your hand.
This way it would be possible to implement a global damage boost to any and all damage by the player based on a skill.

This is a pretty trivial thing to add. i'll get it in the next update.