It seems that the map currently generate cache for all chunk within render distance. Is there a way to shrink radius to not make it too over powered?

Otherwise I would suggest it as an option.
Roughly 2-4 chunk radius above ground, configurable.
Single or adjacent chunks while underground.

If implemented, how will it effect performance?

It may affect only of the first map caching. Further, data takes from the cache, including data that was cached on the disk. In addition, map creates region images, these stored to the disk too, and loads from disk next times. And all steps does in the separate threads. Only one delay that you can see, it a black square at first run of the map in the new world or the first visit to the new dimension.

Now chunks data updates depends on View distance.

I've made the map updates starts from player's position and limited by player's view distance. Now the map updates almost immediate.