targettarget portrait Sometimes lost
targettarget portrait Model updating Slow

Not sure if this is needed, because you have not provided more information, but the slow updates are due to the fact, that events are not fired by the game for the targettarget unit. Thus oUF updates all registered elements every 0.5 seconds by default, which you can override by setting the .onUpdateFrequency options on the frame holding the unit. Because updating the 3d portrait model every 0.5 seconds resets the scene, we only update it if either the GUID or the visibility state of the unit has changed.

The "sometimes lost" case should be addressed by the PR linked above. Keep in mind that the aforementioned limitations still apply and the model is only updated if we get a portrait event and the targettarget unit is the same as the event unit.

TL;DR
Updating the portrait on *target units is unreliable and there is little we can do about it.