From 74302319773b19d72f94f47e29c3817763aacf28 Mon Sep 17 00:00:00 2001 From: Austin Kline Date: Wed, 17 Aug 2022 08:59:50 -0700 Subject: [PATCH 1/8] add 20220817-add-fields-to-existing-definitions.md --- ...0817-add-fields-to-existing-definitions.md | 80 +++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 flips/20220817-add-fields-to-existing-definitions.md diff --git a/flips/20220817-add-fields-to-existing-definitions.md b/flips/20220817-add-fields-to-existing-definitions.md new file mode 100644 index 00000000..7ab5fd7a --- /dev/null +++ b/flips/20220817-add-fields-to-existing-definitions.md @@ -0,0 +1,80 @@ +# Allow new fields in deployed Resources and Structs + +| Status | Proposed | +|---------------|:--------------------------------------------------------- | +| **FLIP #** | [NNN](https://github.com/onflow/flow/pull/NNN) | +| **Author(s)** | Deniz Mert Edincik (deniz@edincik.com) | +| | Austin Kline (austin@flowty.io) | +| **Sponsor** | | +| **Updated** | 2022-08-17 | + +## Objective +This proposed change will allow existing structs and resources to add new fields to them by +using optional and default assignment embedded in the object definition itself. + +## Motivation + +One major challenge in smart contract design in cadence is the inability to add new fields to already deployed +structs and resources. Not having the ability to do this means that contracts either get extended +with data that exists in another contract, entirely new contracts are made that are essentially reprints of the +original, except with the newly added fields, or some other workaround such as storing fields as dictionaries +and building them into at runtime. + +Each of these workarounds come with their downsides. +- New contracts lead to complex migrations for applications and sunsetting existing contracts. +- Hosting new data in another contract leads to harder-to-follow code and added complexity. +- Factory patterns increase compute since objects must be built at runtime and also lose the benefits of type-checking + since the underlying structure is not truly typed. + +## User Benefit + +- It will allow developers to add fields as they need them, making contract development more focused on the needs of the current version + as opposed to undue complexity to take future plans into account. +- It will allow more maintainable contract code with less splintered logic as existing contracts update themselves + +## Design Proposal + +We propose to add the ability to define new optional fields, and the assignment of default values to fields in +structs and resources. All fields added after the first deployment of a contract must be either optional or have +a default parameter assigned so that initialization of existing instances of these objects can be properly +initialized. + +```cadence +pub struct Message { + pub let content: String +} +``` + +We could now add a new field to this struct in a few ways + +```cadence +pub struct Message { + pub let content: String + + // new fields + // an optional new field. Existing instances of Message will have timestamp set to nil when accessed + // unless they are set at a later date + pub let timestamp: UInt64? + // a default initialized field, Existing instances of Message will take the default value. + pub let received: Bool = false +} +``` + +### Limitations + +- This will not allow existing fields to be altered. That is, you cannot take a field and alter its type. +- This will not allow new fields to be derived from existing ones. + +### Compatibility + +This should be backwards compatible + +### User Impact + +- Cadence developers will be able to modify their contract more to their needs instead of over-designing with +the first launch of their contract(s). +- If new unforeseen features or fixes require new fields, those additions can be kept in the original contract instead of being silo'd off into their own contract. + +## Prior Art + +N/A \ No newline at end of file From 136e07f298df69d58aa807ec7e552001ed912510 Mon Sep 17 00:00:00 2001 From: Austin Kline Date: Fri, 26 Aug 2022 18:38:12 -0700 Subject: [PATCH 2/8] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Daniel Sainati Co-authored-by: Bastian Müller --- flips/20220817-add-fields-to-existing-definitions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/flips/20220817-add-fields-to-existing-definitions.md b/flips/20220817-add-fields-to-existing-definitions.md index 7ab5fd7a..912fe7e8 100644 --- a/flips/20220817-add-fields-to-existing-definitions.md +++ b/flips/20220817-add-fields-to-existing-definitions.md @@ -2,7 +2,7 @@ | Status | Proposed | |---------------|:--------------------------------------------------------- | -| **FLIP #** | [NNN](https://github.com/onflow/flow/pull/NNN) | +| **FLIP #** | [1097](https://github.com/onflow/flow/pull/1097) | | **Author(s)** | Deniz Mert Edincik (deniz@edincik.com) | | | Austin Kline (austin@flowty.io) | | **Sponsor** | | @@ -36,7 +36,7 @@ Each of these workarounds come with their downsides. We propose to add the ability to define new optional fields, and the assignment of default values to fields in structs and resources. All fields added after the first deployment of a contract must be either optional or have -a default parameter assigned so that initialization of existing instances of these objects can be properly +a default value assigned so that initialization of existing instances of these objects can be properly initialized. ```cadence From df1ad5c2fc3b9d68d7cb49fe0c6d5e8a26ac9ceb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bastian=20M=C3=BCller?= Date: Tue, 22 Oct 2024 15:11:53 -0700 Subject: [PATCH 3/8] Update flips/20220817-add-fields-to-existing-definitions.md --- .../20220817-add-fields-to-existing-definitions.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/flips/20220817-add-fields-to-existing-definitions.md b/flips/20220817-add-fields-to-existing-definitions.md index 912fe7e8..2d6a4500 100644 --- a/flips/20220817-add-fields-to-existing-definitions.md +++ b/flips/20220817-add-fields-to-existing-definitions.md @@ -1,12 +1,12 @@ # Allow new fields in deployed Resources and Structs -| Status | Proposed | -|---------------|:--------------------------------------------------------- | -| **FLIP #** | [1097](https://github.com/onflow/flow/pull/1097) | -| **Author(s)** | Deniz Mert Edincik (deniz@edincik.com) | -| | Austin Kline (austin@flowty.io) | -| **Sponsor** | | -| **Updated** | 2022-08-17 | +--- +status: proposed +flip: TBD +authors: Deniz Mert Edincik (deniz@edincik.com), Austin Kline (austin@flowty.io) +sponsor: Bastian Mueller (bastian.mueller@flowfoundation.org) +updated: 2022-08-17 +--- ## Objective This proposed change will allow existing structs and resources to add new fields to them by From 3797b81ae1f5b3933dd5dc69fffeca6d9f38afad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bastian=20M=C3=BCller?= Date: Tue, 22 Oct 2024 15:48:09 -0700 Subject: [PATCH 4/8] move to correct directory --- {flips => cadence}/20220817-add-fields-to-existing-definitions.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename {flips => cadence}/20220817-add-fields-to-existing-definitions.md (100%) diff --git a/flips/20220817-add-fields-to-existing-definitions.md b/cadence/20220817-add-fields-to-existing-definitions.md similarity index 100% rename from flips/20220817-add-fields-to-existing-definitions.md rename to cadence/20220817-add-fields-to-existing-definitions.md From 9ab59b936920b9ac4cd3a9d70b042350f7a146e5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bastian=20M=C3=BCller?= Date: Tue, 22 Oct 2024 15:51:59 -0700 Subject: [PATCH 5/8] move title after front matter, remove trailing whitespace --- ...0817-add-fields-to-existing-definitions.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/cadence/20220817-add-fields-to-existing-definitions.md b/cadence/20220817-add-fields-to-existing-definitions.md index 2d6a4500..9ca904d2 100644 --- a/cadence/20220817-add-fields-to-existing-definitions.md +++ b/cadence/20220817-add-fields-to-existing-definitions.md @@ -1,28 +1,28 @@ -# Allow new fields in deployed Resources and Structs - --- -status: proposed +status: proposed flip: TBD -authors: Deniz Mert Edincik (deniz@edincik.com), Austin Kline (austin@flowty.io) +authors: Deniz Mert Edincik (deniz@edincik.com), Austin Kline (austin@flowty.io) sponsor: Bastian Mueller (bastian.mueller@flowfoundation.org) updated: 2022-08-17 --- +# FLIP: Allow adding new fields to existing types + ## Objective This proposed change will allow existing structs and resources to add new fields to them by using optional and default assignment embedded in the object definition itself. ## Motivation -One major challenge in smart contract design in cadence is the inability to add new fields to already deployed -structs and resources. Not having the ability to do this means that contracts either get extended +One major challenge in smart contract design in cadence is the inability to add new fields to already deployed +structs and resources. Not having the ability to do this means that contracts either get extended with data that exists in another contract, entirely new contracts are made that are essentially reprints of the original, except with the newly added fields, or some other workaround such as storing fields as dictionaries and building them into at runtime. -Each of these workarounds come with their downsides. -- New contracts lead to complex migrations for applications and sunsetting existing contracts. -- Hosting new data in another contract leads to harder-to-follow code and added complexity. +Each of these workarounds come with their downsides. +- New contracts lead to complex migrations for applications and sunsetting existing contracts. +- Hosting new data in another contract leads to harder-to-follow code and added complexity. - Factory patterns increase compute since objects must be built at runtime and also lose the benefits of type-checking since the underlying structure is not truly typed. @@ -50,13 +50,13 @@ We could now add a new field to this struct in a few ways ```cadence pub struct Message { pub let content: String - + // new fields // an optional new field. Existing instances of Message will have timestamp set to nil when accessed // unless they are set at a later date pub let timestamp: UInt64? // a default initialized field, Existing instances of Message will take the default value. - pub let received: Bool = false + pub let received: Bool = false } ``` @@ -72,7 +72,7 @@ This should be backwards compatible ### User Impact - Cadence developers will be able to modify their contract more to their needs instead of over-designing with -the first launch of their contract(s). +the first launch of their contract(s). - If new unforeseen features or fixes require new fields, those additions can be kept in the original contract instead of being silo'd off into their own contract. ## Prior Art From 97861b17145245e24a68c424db4116f26df21da2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bastian=20M=C3=BCller?= Date: Tue, 22 Oct 2024 16:20:21 -0700 Subject: [PATCH 6/8] update to Cadence 1.0, remove default value proposal --- ...0817-add-fields-to-existing-definitions.md | 82 ++++++++++++------- 1 file changed, 51 insertions(+), 31 deletions(-) diff --git a/cadence/20220817-add-fields-to-existing-definitions.md b/cadence/20220817-add-fields-to-existing-definitions.md index 9ca904d2..6ef95c4d 100644 --- a/cadence/20220817-add-fields-to-existing-definitions.md +++ b/cadence/20220817-add-fields-to-existing-definitions.md @@ -9,60 +9,78 @@ updated: 2022-08-17 # FLIP: Allow adding new fields to existing types ## Objective -This proposed change will allow existing structs and resources to add new fields to them by -using optional and default assignment embedded in the object definition itself. + +This proposed change will allow updating existing types (contracts) with new fields. ## Motivation -One major challenge in smart contract design in cadence is the inability to add new fields to already deployed -structs and resources. Not having the ability to do this means that contracts either get extended -with data that exists in another contract, entirely new contracts are made that are essentially reprints of the -original, except with the newly added fields, or some other workaround such as storing fields as dictionaries -and building them into at runtime. +One major challenge in smart contract design in Cadence is the inability to add new fields to already deployed types. +The lack of this functionality means that developers have to resort to brittle workarounds. +For example, developers may choose to deploy entirely new contracts, +that are essentially reprints of the original, except with the newly added fields. +Developers may also choose to +- Manually storing fields as dictionaries to accommodate future new fields +- Manually storing new fields in a new, different contract +- Manually storing new fields in storage Each of these workarounds come with their downsides. - New contracts lead to complex migrations for applications and sunsetting existing contracts. -- Hosting new data in another contract leads to harder-to-follow code and added complexity. -- Factory patterns increase compute since objects must be built at runtime and also lose the benefits of type-checking - since the underlying structure is not truly typed. +- Storing new data manually is brittle, and leads to harder-to-follow code and added complexity. +- Factory patterns increase compute since objects must be built at runtime + and also lose the benefits of type-checking, since the underlying structure is not truly typed. ## User Benefit -- It will allow developers to add fields as they need them, making contract development more focused on the needs of the current version - as opposed to undue complexity to take future plans into account. -- It will allow more maintainable contract code with less splintered logic as existing contracts update themselves +Allowing developers to add new fields to existing types as they need them +will make contract development more focused on the needs of the current version +as opposed to undue complexity to take future plans into account. +It will also allow more maintainable contract code with less complex logic when existing contracts get updated. ## Design Proposal -We propose to add the ability to define new optional fields, and the assignment of default values to fields in -structs and resources. All fields added after the first deployment of a contract must be either optional or have -a default value assigned so that initialization of existing instances of these objects can be properly -initialized. +The [contract updatability checking](https://cadence-lang.org/docs/language/contract-updatability) +is extended to allow new fields to be added to existing types, +such as contracts, structs, resources, and attachments. +These new fields must be optional. + +When the added field of an existing stored value is accessed, the result is `nil`. + +For example, assume the following struct type is currently deployed as part of a contract: ```cadence -pub struct Message { - pub let content: String +access(all) +struct Message { + access(all) + let content: String + + init(content: String) { + self.content = content + } } ``` -We could now add a new field to this struct in a few ways +We can now add a new field to this struct: ```cadence -pub struct Message { - pub let content: String +access(all) +struct Message { + access(all) + let content: String + + init(content: String) { + self.content = content + } - // new fields - // an optional new field. Existing instances of Message will have timestamp set to nil when accessed + // An optional new field. Existing instances of Message will have timestamp set to nil when accessed // unless they are set at a later date - pub let timestamp: UInt64? - // a default initialized field, Existing instances of Message will take the default value. - pub let received: Bool = false + access(all) + let timestamp: UInt64? } ``` ### Limitations -- This will not allow existing fields to be altered. That is, you cannot take a field and alter its type. +- This will not allow existing fields to be altered. That is, it is invalid to alter a field's type. - This will not allow new fields to be derived from existing ones. ### Compatibility @@ -71,9 +89,11 @@ This should be backwards compatible ### User Impact -- Cadence developers will be able to modify their contract more to their needs instead of over-designing with -the first launch of their contract(s). -- If new unforeseen features or fixes require new fields, those additions can be kept in the original contract instead of being silo'd off into their own contract. +- Cadence developers will be able to modify their contract more to their needs + instead of over-designing with the first launch of their contract(s). +- If new unforeseen features or fixes require new fields, + those additions can be kept in the original contract, + instead of being silo'd off into their own contract. ## Prior Art From 7012713b04bbcdc6fe3a859f53bc72f9d58f57a4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bastian=20M=C3=BCller?= Date: Tue, 22 Oct 2024 17:55:46 -0700 Subject: [PATCH 7/8] add open questions: field removal and default values --- ...0817-add-fields-to-existing-definitions.md | 31 ++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/cadence/20220817-add-fields-to-existing-definitions.md b/cadence/20220817-add-fields-to-existing-definitions.md index 6ef95c4d..07c7c983 100644 --- a/cadence/20220817-add-fields-to-existing-definitions.md +++ b/cadence/20220817-add-fields-to-existing-definitions.md @@ -97,4 +97,33 @@ This should be backwards compatible ## Prior Art -N/A \ No newline at end of file +N/A + +## Open Questions + +- Adding fields can only be allowed if it is not possible to re-add previously removed fields. + + One option could be to allow keep allowing to remove existing fields, + but require them to be "tomb-stoned", + similar to how [FLIP 275](https://github.com/onflow/flips/issues/275) proposes + to allow removing existing type definitions. + +- A previous iteration of the proposal also proposed to allow adding non-optional fields, + by allowing the definition of the added field to provide a *default* value. + + This could be useful for types for which an optional type unnecessarily complicates the type, + such as for simple types like `Bool`, `String`, `Int`, etc. + + For example, the field added in the example above could be written instead as: + + ```cadence + let timestamp: UInt64 = 0 + ``` + + Support for default values could be restricted to just certain types (e.g. `Bool`, `String`, numbers), + and default value expressions could be restricted to just literals (e.g. `true`, `""`, `0`). + Further, value expressions could be extended to allow access to `self`, + and access expressions on it could be allowed. + + The default value functionality would be similar to default values of + [resource destruction events](https://github.com/onflow/flips/blob/main/cadence/20230811-destructor-removal.md#destruction-events). From 6200b4ea15b132959e4818f45eb749ffb53cebd2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bastian=20M=C3=BCller?= Date: Wed, 23 Oct 2024 13:40:46 -0700 Subject: [PATCH 8/8] add open question regarding already removed fields --- cadence/20220817-add-fields-to-existing-definitions.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/cadence/20220817-add-fields-to-existing-definitions.md b/cadence/20220817-add-fields-to-existing-definitions.md index 07c7c983..972e4b50 100644 --- a/cadence/20220817-add-fields-to-existing-definitions.md +++ b/cadence/20220817-add-fields-to-existing-definitions.md @@ -108,6 +108,13 @@ N/A similar to how [FLIP 275](https://github.com/onflow/flips/issues/275) proposes to allow removing existing type definitions. + For example, field declarations would need to be replaced by e.g. `#removedField(name)`. + + However, how do we handle fields that were already previously removed, + but are still stored in the object? + We could clean up existing stored composite values and remove stored fields + that are not declared in the type. + - A previous iteration of the proposal also proposed to allow adding non-optional fields, by allowing the definition of the added field to provide a *default* value.