CommandHelper gets all keys from Redis on get_values() rather than just those requested
LadyCailinBot opened this issue ยท 4 comments
CMDHELPER-2688 - Reported by spudwaffle
When using a Redis data source, CommandHelper sends {{KEYS }} to the Redis server on {{get_values('key1.key2')}} rather than {{KEYS storage.key1.key2.}}. This can take upwards of 1 second when the Redis database has many keys, causing lag.
Comment by spudwaffle
This bug also affects optimized calls of get_value. The optimizer groups get_value calls into one get_values call which only decreases performance because of this bug.
Comment by LadyCailin
Currently, the abstract model doesn't distinguish these cases. I need to rework the abstract model allow for cases where this optimization can be done, and then implement it correctly for things like redis (and sqlite/mysql too). Given the sheer number of data source backends (which tends to be ini), and the fact that I have to cater to the lowest common denominator, my abstract model defined fairly little at first, though it's becoming obvious that I can and should optimize for some of the more advanced data storage models.
Comment by LadyCailin
Ok, this should be improved now. It should use something more specific than KEYS *. Let me know if you see a performance increase. (I also optimized SQLite data sources in the same way.)