Commit Graph

12 Commits

Author SHA1 Message Date
Mitchell Hashimoto 4576eaa966 helper/schema: replace config/lang 2016-02-03 13:24:04 -05:00
Sander van Harmelen ef4726bd50 Change Set internals and make (extreme) performance improvements
Changing the Set internals makes a lot of sense as it saves doing
conversions in multiple places and gives a central place to alter
the key when a item is computed.

This will have no side effects other then that the ordering is now
based on strings instead on integers, so the order will be different.
This will however have no effect on existing configs as these will
use the individual codes/keys and not the ordering to determine if
there is a diff or not.

Lastly (but I think also most importantly) there is a fix in this PR
that makes diffing sets extremely more performand. Before a full diff
required reading the complete Set for every single parameter/attribute
you wanted to diff, while now it only gets that specific parameter.

We have a use case where we have a Set that has 18 parameters and the
set consist of about 600 items (don't ask 😉). So when doing a diff
it would take 100% CPU of all cores and stay that way for almost an
hour before being able to complete the diff.

Debugging this we learned that for retrieving every single parameter
it made over 52.000 calls to `func (c *ResourceConfig) get(..)`. In
this function a slice is created and used only for the duration of the
call, so the time needed to create all needed slices and on the other
hand the time the garbage collector needed to clean them up again caused
the system to cripple itself. Next to that there are also some expensive
reflect calls in this function which also claimed a fair amount of CPU
time.

After this fix the number of calls needed to get a single parameter
dropped from 52.000+ to only 2! 😃
2015-11-22 14:21:28 +01:00
Panagiotis Moustafellos e4845f75cc removed extra parentheses 2015-10-08 15:48:04 +03:00
Mitchell Hashimoto fa934d96d0 helper/schema: FieldReaderConfig detects computed maps 2015-04-21 22:07:52 +02:00
Mitchell Hashimoto 2feaebdca5 config: substring containing computed value replaces element 2015-02-27 21:51:14 -08:00
Paul Hinze 5d4e69cc80 helper/schema: apply schema defaults at the field level when reading from config
We were waiting until the higher-level (m schemaMap) diffString method
to apply defaults, which was messing with set hashcode evaluation for
cases when a field with a default is included in the hash function.

fixes #824
2015-01-27 18:18:57 -06:00
Mitchell Hashimoto 1fcd24cf67 helper/schema: add float type 2015-01-10 16:04:01 -08:00
Mitchell Hashimoto 361d00347a helper/schema: refactor tests for fieldreader to be common 2015-01-10 12:42:15 -08:00
Mitchell Hashimoto b4bf813151 helper/schema: too big to fail 2015-01-08 18:02:19 -08:00
Mitchell Hashimoto 7e379cb1a1 helper/schema: field readers no longer take a schema as arg 2015-01-03 12:13:46 -05:00
Mitchell Hashimoto 73726e83b2 helper/schema: DiffFieldReader for reading data from a diff 2014-12-20 02:17:35 +05:30
Mitchell Hashimoto 9447973015 helper/schema: ConfigFieldReader and generic helpers 2014-12-19 22:52:53 +05:30