Native Libs only resolve from certain specific locations.
Something like the Assembly Resolve Manager has where it allows for specifying custom locations to search would be useful.

A wrapper using AddDllDirectory and RemoveDllDirectory might be viable.