Commit Graph

79 Commits

Author SHA1 Message Date
Radek Simko 4c6ceef9b8 helper/schema: Allow identification of a new resource in update func 2015-12-27 14:01:03 +01:00
Sander van Harmelen ef4726bd50 Change Set internals and make (extreme) performance improvements
Changing the Set internals makes a lot of sense as it saves doing
conversions in multiple places and gives a central place to alter
the key when a item is computed.

This will have no side effects other then that the ordering is now
based on strings instead on integers, so the order will be different.
This will however have no effect on existing configs as these will
use the individual codes/keys and not the ordering to determine if
there is a diff or not.

Lastly (but I think also most importantly) there is a fix in this PR
that makes diffing sets extremely more performand. Before a full diff
required reading the complete Set for every single parameter/attribute
you wanted to diff, while now it only gets that specific parameter.

We have a use case where we have a Set that has 18 parameters and the
set consist of about 600 items (don't ask 😉). So when doing a diff
it would take 100% CPU of all cores and stay that way for almost an
hour before being able to complete the diff.

Debugging this we learned that for retrieving every single parameter
it made over 52.000 calls to `func (c *ResourceConfig) get(..)`. In
this function a slice is created and used only for the duration of the
call, so the time needed to create all needed slices and on the other
hand the time the garbage collector needed to clean them up again caused
the system to cripple itself. Next to that there are also some expensive
reflect calls in this function which also claimed a fair amount of CPU
time.

After this fix the number of calls needed to get a single parameter
dropped from 52.000+ to only 2! 😃
2015-11-22 14:21:28 +01:00
Mitchell Hashimoto 58a8776c41 helper/schema: test real nil pointer to ResourceData.Set 2015-03-02 23:37:43 -08:00
Mitchell Hashimoto c030148259 helper/schema: allow pointer values to ResourceData.Set 2015-03-02 21:06:14 -08:00
Mitchell Hashimoto 7d32c8946a helper/schema: GetOk now only returns true if set to non-zero value 2015-02-17 16:55:39 -08:00
Mitchell Hashimoto e04def93e6 Merge pull request #991 from hashicorp/b-autoscale-lc-update
providers/aws: allow in-place update of launch configuration
2015-02-17 16:29:56 -08:00
Mitchell Hashimoto 5a64d0900b providers/aws: test for allowing in-place lC update 2015-02-17 16:12:02 -08:00
Mitchell Hashimoto 66f7731995 helper/schema: GetChange shouldn't return true when no change 2015-02-17 15:43:19 -08:00
Mitchell Hashimoto 2212d6895d helper/schema: diff with set going to 0 elements removes it from state 2015-02-17 11:38:56 -08:00
Mitchell Hashimoto 87948b68fc helper/schema: use interface for equality check
/cc @svanharmelen
2015-01-16 09:32:15 -08:00
Sander van Harmelen c7550595a3 Fixing two related bugs in HasChange and GetChange
This was actually quite nasty as the first bug covered the second one…

The first bug is with HasChange. This function uses reflect.DeepEqual
to check if two instances are the same/have the same content. This
works fine for all types except for Set’s as they contain a function.
And reflect.DeepEqual will only say the functions are equal if they are
both nil (which they aren’t in a Set). So in effect it means that
currently HasChange will always say true for Set’s, even when they are
actually being equal.

As soon as you fix this problem, you will notice the second one (which
the added test is written for). Without saying you want the exact diff,
you will end up with a merged value which will (in most cases) be the
same.

Run all unit tests and a good part of the acc tests to verify this
works as expected and all look good.
2015-01-16 14:13:40 +01:00
Mitchell Hashimoto 4d067f4d6d helper/schema: don't put things into the state that don't exist or are
computed [GH-805]
2015-01-15 10:35:44 -08:00
Mitchell Hashimoto db02541d31 helper/schema: fix failing tests
/cc @svanharmelen - I think some logic changed after my refactor. I now
return Exists: true when Computed: true but the value might be blank to
note that the FieldReader FOUND a value, its just unknown. I think
before it didn't do that so the logic for GetOk has to be "does it exist
and is it _not_ computed"

Seems weird because I just realized there is no way to get the OLD value
of something if it is being computed now, but I looked and there are
tests that verify this and they're like... test #5 of Get. So, they're
not new meaning that must've been expected behavior? Hm. Let me know if
you find any other issues from acceptance tests
2015-01-14 15:38:18 -08:00
Mitchell Hashimoto 3cbcafe989 helper/schema: remove unused field 2015-01-10 12:50:53 -08:00
Mitchell Hashimoto 9ab128899a helper/schema: make the getSource enum easier 2015-01-10 12:25:34 -08:00
Mitchell Hashimoto d89446391a helper/schema: make the get API cleaner 2015-01-10 12:22:05 -08:00
Mitchell Hashimoto 3c1b55a75f helper/schema: use the field reader/writer for state 2015-01-10 12:18:32 -08:00
Mitchell Hashimoto e77b2b17c4 helper/schema: remove unused method 2015-01-10 11:46:09 -08:00
Mitchell Hashimoto 03c6453a72 helper/schema: FieldWriter, replace Set 2015-01-10 11:44:26 -08:00
Mitchell Hashimoto e57f3f69b1 helper/schema: empty maps, support reading objects directly 2015-01-09 15:07:02 -08:00
Mitchell Hashimoto f0af1c36f5 helper/schema: nested resource fields should be zero-valued on get 2015-01-09 11:51:29 -08:00
Mitchell Hashimoto 942a988ac2 helper/schema: zero value of a set should be a set 2015-01-08 18:48:03 -08:00
Mitchell Hashimoto b4bf813151 helper/schema: too big to fail 2015-01-08 18:02:19 -08:00
Sander van Harmelen 40fa6c3aaa Fixed a type and reordered the case statements
Making in more inline with the existing code…
2014-12-15 23:02:16 +01:00
Sander van Harmelen 83c760fcb3 core: refactoring the way sets work internally v2
This is a refactored solution for PR #616. Functionally this is still
the same change, but it’s implemented a lot cleaner with less code and
less changes to existing parts of TF.
2014-12-12 23:21:20 +01:00
Mitchell Hashimoto 61811fce23 helper/schema: remove debug info 2014-10-21 10:54:44 -07:00
Mitchell Hashimoto f63a5d24e9 helper/schema: diffs for sets should include the full set [GH-457]
Prior to this, the diff only contained changed set elements. The issue
with this is that `getSet`, the internal function that reads a set from
the ResourceData, expects that each level (state, config, diff, etc.)
has the _full set_ information. This change was done to fix merging
issues.

Because of this, we need to make sure the full set is visible in the
diff.
2014-10-21 10:49:53 -07:00
Mitchell Hashimoto d1324678dd helper/schema: setting empty map works [GH-464] 2014-10-21 00:17:17 -07:00
Mitchell Hashimoto 17d1bb7dde Revert "Merge pull request #422 from svanharmelen/f-shaving-off-some-cpu-cycles"
This reverts commit 428cce2478, reversing
changes made to 517c5b2ea8.
2014-10-20 23:14:14 -07:00
Mitchell Hashimoto 428cce2478 Merge pull request #422 from svanharmelen/f-shaving-off-some-cpu-cycles
helper/schema: Shaving off some unneeded calls, saving CPU cycles
2014-10-20 23:13:18 -07:00
Mitchell Hashimoto 5390357e45 helper/schema: sets properly take into account the diff 2014-10-20 15:32:30 -07:00
Sander van Harmelen e003334429 Shaving off some unneeded calls, saving CPU cycles
I know it’s very unlikely that a user will notice the difference, but
why range through the list, generate the set and calculate the
hashcode, only to find out that indexMap == nil (e.g. don’t do anything
with the generated hashcode).

As indexMap is only needed when len(parts) > 0, why not only create and
fill it (in one go) when len(parts) > 0?
2014-10-16 11:54:36 +02:00
Mitchell Hashimoto 59349cca11 helper/schema: sets must be treated atomically within ResourceData
This fixes a seemingly minor issue (GH-255) around plans showing changes
when in fact there are none. But in reality this turned out to uncover a
really terrible bug.

The effect of what was happening was that multiple items in a set were
being merged. Now, they were being merged in the right order, so if you
didn't have rich types (lists in a set) then you never saw the effect
since the later value would overwrite the earlier. But with lists (such
as in security groups), you would end up with the lists merging. So, if
you had one ingress rule with CIDR blocks and one with SGs, then after
the merge both ingress rules would have BOTH CIDR and SGs, resulting in
an incorrect plan (GH-255).

This fixes the issue by introducing a `getSourceExact` bitflag to the
ResourceData source. When this is set, ALL data must come from this
level, instead of merging lower levels. In the case of sets and diffs,
this is exactly what you want: "Get me the set 'foo' from the config and
the config ONLY (not the state or diff or w/e)".

Andddddd its fixed.

GH-255
2014-10-11 10:40:54 -07:00
Mitchell Hashimoto 3a107d2e50 helper/schema: set the field to empty if it is a list and computed 2014-10-10 15:58:38 -07:00
Mitchell Hashimoto e5d64318bf helper/schema: don't say that a set is computed if it exists 2014-10-10 15:46:24 -07:00
Mitchell Hashimoto e0994c717d helper/schema: fix issues around computed sets 2014-10-10 09:13:04 -07:00
Mitchell Hashimoto dcaf653d6f helper/schema: properly detect that a list is computed 2014-10-09 19:09:06 -07:00
Mitchell Hashimoto 00bdef3093 providers/aws: test tag removal 2014-10-08 18:21:21 -07:00
Mitchell Hashimoto 6eafac8a34 providers/aws: aws_vpc supports tags 2014-10-08 17:54:00 -07:00
Mitchell Hashimoto f43528957e helper/schema: support top-level TypeMap 2014-10-08 17:35:14 -07:00
Mitchell Hashimoto 0db9d98fff helper/schema: can set Partial(true) multiple times 2014-10-08 16:35:14 -07:00
Mitchell Hashimoto 9b2b3a963f ResourceDiff => InstanceDiff 2014-09-17 16:33:24 -07:00
Mitchell Hashimoto 81d9d70296 helper/schema: conforms to new API, tests pass 2014-09-16 17:07:13 -07:00
Mitchell Hashimoto faa4e75960 helper/schema: Get on computed diff should be empty 2014-08-27 15:45:52 -07:00
Mitchell Hashimoto 0c9b65f3f4 helper/schema: documentation 2014-08-26 21:52:09 -07:00
Mitchell Hashimoto 87a488092c helper/schema: support partial states 2014-08-26 20:19:44 -07:00
Mitchell Hashimoto 56cf1e6faa Fix go vet complaints 2014-08-24 21:50:35 -07:00
Mitchell Hashimoto 9ed601d541 helper/schema: store state with processed properly 2014-08-22 12:09:06 -07:00
Mitchell Hashimoto 50026a6d5c helper/schema: When having a StateFunc, make sure NewExtra contains
original
2014-08-22 08:57:44 -07:00
Mitchell Hashimoto d009ea800a helper/schema: add support for StateFunc 2014-08-22 08:45:54 -07:00