Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pulumi.Asset support to lambda function #182

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
## HEAD (Unreleased)

- Add lambda.Function support for pulumi Assets [#182](https://github.com/pulumi/pulumi-aws-native/pull/182)

## 0.2.0 (October 8, 2021)

- Deduplicate type names [#160](https://github.com/pulumi/pulumi-aws-native/issues/160)
Expand Down
6 changes: 1 addition & 5 deletions examples/aws-native-ts-stepfunctions/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,7 @@ const worldFunction = new awsnative.lambda.Function('worldFunction',
runtime: "nodejs14.x",
handler: "index.handler",
code: {
zipFile: `exports.handler = function(event, context, callback){
var response = event.response;
const updated = { "response": response + "World!" };
callback(null, updated);
};`,
zipFile: new pulumi.asset.FileAsset("world_function.js"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's why I'm not a fan of the zipFile property... There's no zip in here, right? Should we rename to code, as in the classic provider? The motivation is that we are changing the behavior of a property, so we may rename it as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm conflicted on this. I agree that the name is confusing, but I'm also hesitant to change the name at the native provider layer. I would be more open to adding an alias, perhaps. My main concern is that we should make it easy for integrators to build on top of the native layer, so I think we should avoid changing the API in incompatible ways.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that's reasonable. Let's proceed with the current name for now then.

Copy link
Contributor

@viveklak viveklak Nov 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consider updating the description field in the schema for this attribute to make the confusing semantics clearer?

},
}, {dependsOn: lambdaRolePolicy});

Expand Down
5 changes: 5 additions & 0 deletions examples/aws-native-ts-stepfunctions/world_function.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
exports.handler = function(event, context, callback){
var response = event.response;
const updated = { "response": response + "World!" };
callback(null, updated);
};
16 changes: 15 additions & 1 deletion examples/simple-ts/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,18 @@
// Copyright 2016-2021, Pulumi Corporation.
/*
* Copyright 2016-2021, Pulumi Corporation.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import * as aws from "@pulumi/aws-native";
import * as random from "@pulumi/random";
Expand Down
2 changes: 1 addition & 1 deletion provider/cmd/cf2pulumi/schema-full.json
Original file line number Diff line number Diff line change
Expand Up @@ -44953,7 +44953,7 @@
"description": "For versioned objects, the version of the deployment package object to use."
},
"zipFile": {
"type": "string",
"$ref": "pulumi.json#/Asset",
"description": "The source code of your Lambda function. If you include your function source inline with this parameter, AWS CloudFormation places it in a file named index and zips it to create a deployment package.."
}
},
Expand Down
2 changes: 1 addition & 1 deletion provider/cmd/pulumi-resource-aws-native/metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -45159,7 +45159,7 @@
"description": "For versioned objects, the version of the deployment package object to use."
},
"zipFile": {
"type": "string",
"$ref": "pulumi.json#/Asset",
"description": "The source code of your Lambda function. If you include your function source inline with this parameter, AWS CloudFormation places it in a file named index and zips it to create a deployment package.."
}
}
Expand Down
4 changes: 2 additions & 2 deletions provider/cmd/pulumi-resource-aws-native/schema.json

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion provider/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -485,8 +485,9 @@ github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxv
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.4/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.8 h1:AkaSdXYQOWeaO3neb8EM634ahkXXe3jYbVh/F9lq+GI=
Expand Down
17 changes: 10 additions & 7 deletions provider/pkg/provider/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ func (p *cfnProvider) DiffConfig(ctx context.Context, req *pulumirpc.DiffRequest
news, err := plugin.UnmarshalProperties(req.GetNews(), plugin.MarshalOptions{
Label: fmt.Sprintf("%s.news", label),
KeepUnknowns: true,
RejectAssets: true,
RejectAssets: false,
})
if err != nil {
return nil, errors.Wrapf(err, "diffConfig failed because of malformed resource inputs")
Expand Down Expand Up @@ -511,7 +511,7 @@ func (p *cfnProvider) Check(ctx context.Context, req *pulumirpc.CheckRequest) (*
newInputs, err := plugin.UnmarshalProperties(req.GetNews(), plugin.MarshalOptions{
Label: fmt.Sprintf("%s.properties", label),
KeepUnknowns: true,
RejectAssets: true,
RejectAssets: false,
KeepSecrets: true,
})
if err != nil {
Expand Down Expand Up @@ -579,7 +579,7 @@ func (p *cfnProvider) Create(ctx context.Context, req *pulumirpc.CreateRequest)
inputs, err := plugin.UnmarshalProperties(req.GetProperties(), plugin.MarshalOptions{
Label: fmt.Sprintf("%s.properties", label),
KeepUnknowns: true,
RejectAssets: true,
RejectAssets: false,
KeepSecrets: true,
})
if err != nil {
Expand Down Expand Up @@ -611,7 +611,10 @@ func (p *cfnProvider) Create(ctx context.Context, req *pulumirpc.CreateRequest)
cfType = spec.CfType

// Convert SDK inputs to CFN payload.
payload = schema.SdkToCfn(&spec, p.resourceMap.Types, inputs.MapRepl(nil, mapReplStripSecrets))
payload, err = schema.SdkToCfn(&spec, p.resourceMap.Types, inputs.MapRepl(nil, mapReplStripSecrets))
if err != nil {
return nil, fmt.Errorf("failed to convert SDK inputs to CFN: %w", err)
lblackstone marked this conversation as resolved.
Show resolved Hide resolved
}
}

// Serialize inputs as a desired state JSON.
Expand Down Expand Up @@ -814,7 +817,7 @@ func (p *cfnProvider) Update(ctx context.Context, req *pulumirpc.UpdateRequest)
newInputs, err := plugin.UnmarshalProperties(req.GetNews(), plugin.MarshalOptions{
Label: fmt.Sprintf("%s.newInputs", label),
KeepUnknowns: true,
RejectAssets: true,
RejectAssets: false,
KeepSecrets: true,
})
if err != nil {
Expand Down Expand Up @@ -1001,7 +1004,7 @@ func (p *cfnProvider) diffState(olds *pbstruct.Struct, news *pbstruct.Struct, la
oldState, err := plugin.UnmarshalProperties(olds, plugin.MarshalOptions{
Label: fmt.Sprintf("%s.oldState", label),
KeepUnknowns: true,
RejectAssets: true,
RejectAssets: false,
KeepSecrets: true,
})
if err != nil {
Expand All @@ -1014,7 +1017,7 @@ func (p *cfnProvider) diffState(olds *pbstruct.Struct, news *pbstruct.Struct, la
newInputs, err := plugin.UnmarshalProperties(news, plugin.MarshalOptions{
Label: fmt.Sprintf("%s.newInputs", label),
KeepUnknowns: true,
RejectAssets: true,
RejectAssets: false,
KeepSecrets: true,
})
if err != nil {
Expand Down
103 changes: 76 additions & 27 deletions provider/pkg/schema/convert.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,16 @@
// Copyright 2016-2021, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package schema

Expand All @@ -15,7 +27,7 @@ import (

// SdkToCfn converts Pulumi-SDK-shaped state to CloudFormation-shaped payload. In particular, SDK properties
// are lowerCamelCase, while CloudFormation is usually (but not always) PascalCase.
func SdkToCfn(res *CloudAPIResource, types map[string]CloudAPIType, properties map[string]interface{}) map[string]interface{} {
func SdkToCfn(res *CloudAPIResource, types map[string]CloudAPIType, properties map[string]interface{}) (map[string]interface{}, error) {
converter := sdkToCfnConverter{res, types}
return converter.sdkToCfn(properties)
}
Expand All @@ -24,98 +36,132 @@ func SdkToCfn(res *CloudAPIResource, types map[string]CloudAPIType, properties m
// mapped to corresponding patch terms, and SDK properties are translated to respective CFN names.
func DiffToPatch(res *CloudAPIResource, types map[string]CloudAPIType, diff *resource.ObjectDiff) ([]jsonpatch.JsonPatchOperation, error) {
converter := sdkToCfnConverter{res, types}
return converter.diffToPatch(diff), nil
return converter.diffToPatch(diff)
}

type sdkToCfnConverter struct {
spec *CloudAPIResource
types map[string]CloudAPIType
}

func (c *sdkToCfnConverter) sdkToCfn(properties map[string]interface{}) map[string]interface{} {
func (c *sdkToCfnConverter) sdkToCfn(properties map[string]interface{}) (map[string]interface{}, error) {
result := map[string]interface{}{}
var err error
for k, prop := range c.spec.Inputs {
if v, ok := properties[k]; ok {
result[ToCfnName(k)] = c.sdkTypedValueToCfn(&prop.TypeSpec, v)
result[ToCfnName(k)], err = c.sdkTypedValueToCfn(&prop.TypeSpec, v)
if err != nil {
return nil, err
}
}
}
for k, attr := range c.spec.Outputs {
if v, ok := properties[k]; ok {
result[ToCfnName(k)] = c.sdkTypedValueToCfn(&attr.TypeSpec, v)
result[ToCfnName(k)], err = c.sdkTypedValueToCfn(&attr.TypeSpec, v)
if err != nil {
return nil, err
}
}
}
return result
return result, nil
}

func (c *sdkToCfnConverter) sdkTypedValueToCfn(spec *pschema.TypeSpec, v interface{}) interface{} {
func (c *sdkToCfnConverter) sdkTypedValueToCfn(spec *pschema.TypeSpec, v interface{}) (interface{}, error) {
if spec.Ref != "" {
if spec.Ref == "pulumi.json#/Any" {
return v
switch spec.Ref {
case "pulumi.json#/Any":
return v, nil
case "pulumi.json#/Asset", "pulumi.json#/Archive":
lblackstone marked this conversation as resolved.
Show resolved Hide resolved
switch t := v.(type) {
case *resource.Asset:
b, err := t.Bytes()
if err != nil {
return nil, err
}
return string(b), nil
}
}

typName := strings.TrimPrefix(spec.Ref, "#/types/")
return c.sdkObjectValueToCfn(typName, v)
}

var err error
switch spec.Type {
case "array":
array := v.([]interface{})
vs := make([]interface{}, len(array))
for i, item := range array {
vs[i] = c.sdkTypedValueToCfn(spec.Items, item)
vs[i], err = c.sdkTypedValueToCfn(spec.Items, item)
if err != nil {
return nil, err
}
}
return vs
return vs, nil
case "object":
sourceMap := v.(map[string]interface{})
vs := map[string]interface{}{}
for n, item := range sourceMap {
vs[n] = c.sdkTypedValueToCfn(spec.AdditionalProperties, item)
vs[n], err = c.sdkTypedValueToCfn(spec.AdditionalProperties, item)
if err != nil {
return nil, err
}
}
return vs
return vs, nil
default:
return v
return v, nil
}
}

func (c *sdkToCfnConverter) sdkObjectValueToCfn(typeName string, value interface{}) interface{} {
func (c *sdkToCfnConverter) sdkObjectValueToCfn(typeName string, value interface{}) (interface{}, error) {
properties, ok := value.(map[string]interface{})
if !ok {
return value
return value, nil
}

spec := c.types[typeName]
result := map[string]interface{}{}
var err error
for k, prop := range spec.Properties {
if v, ok := properties[k]; ok {
result[ToCfnName(k)] = c.sdkTypedValueToCfn(&prop.TypeSpec, v)
result[ToCfnName(k)], err = c.sdkTypedValueToCfn(&prop.TypeSpec, v)
if err != nil {
return nil, err
}
}
}
return result
return result, nil
}

func (c *sdkToCfnConverter) diffToPatch(diff *resource.ObjectDiff) []jsonpatch.JsonPatchOperation {
func (c *sdkToCfnConverter) diffToPatch(diff *resource.ObjectDiff) ([]jsonpatch.JsonPatchOperation, error) {
var ops []jsonpatch.JsonPatchOperation
for sdkName, prop := range c.spec.Inputs {
cfnName := ToCfnName(sdkName)
key := resource.PropertyKey(sdkName)
if v, ok := diff.Updates[key]; ok {
op := c.valueToPatch("replace", cfnName, prop, v.New)
op, err := c.valueToPatch("replace", cfnName, prop, v.New)
if err != nil {
return nil, err
}
ops = append(ops, op)
}
if v, ok := diff.Adds[key]; ok {
op := c.valueToPatch("add", cfnName, prop, v)
op, err := c.valueToPatch("add", cfnName, prop, v)
if err != nil {
return nil, err
}
ops = append(ops, op)
}
if _, ok := diff.Deletes[key]; ok {
op := jsonpatch.NewPatch("remove", "/" + cfnName, nil)
op := jsonpatch.NewPatch("remove", "/"+cfnName, nil)
ops = append(ops, op)
}
}
return ops
return ops, nil
}

func (c *sdkToCfnConverter) valueToPatch(opName, propName string, prop pschema.PropertySpec, value resource.PropertyValue) jsonpatch.JsonPatchOperation {
op := jsonpatch.NewPatch(opName, "/" + propName, nil)
func (c *sdkToCfnConverter) valueToPatch(opName, propName string, prop pschema.PropertySpec, value resource.PropertyValue) (jsonpatch.JsonPatchOperation, error) {
op := jsonpatch.NewPatch(opName, "/"+propName, nil)
switch {
case value.IsNumber() && prop.Type == "integer":
i := int32(value.NumberValue())
Expand All @@ -128,12 +174,15 @@ func (c *sdkToCfnConverter) valueToPatch(opName, propName string, prop pschema.P
op.Value = value.StringValue()
default:
sdkObj := value.MapRepl(nil, nil)
cfnObj := c.sdkTypedValueToCfn(&prop.TypeSpec, sdkObj)
cfnObj, err := c.sdkTypedValueToCfn(&prop.TypeSpec, sdkObj)
if err != nil {
return jsonpatch.JsonPatchOperation{}, err
}
jsonBytes, err := json.Marshal(cfnObj)
contract.AssertNoError(err)
op.Value = string(jsonBytes)
}
return op
return op, nil
}

// CfnToSdk converts CloudFormation-shaped payload to Pulumi-SDK-shaped state. In particular, SDK properties
Expand Down
15 changes: 14 additions & 1 deletion provider/pkg/schema/convert_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,16 @@
// Copyright 2016-2021, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package schema

Expand All @@ -19,7 +31,8 @@ func TestCfnToSdk(t *testing.T) {

func TestSdkToCfn(t *testing.T) {
res := sampleSchema.Resources["aws-native:ecs:Service"]
actual := SdkToCfn(&res, sampleSchema.Types, sdkState)
actual, err := SdkToCfn(&res, sampleSchema.Types, sdkState)
assert.NoError(t, err)
assert.Equal(t, cfnPayload, actual)
}

Expand Down
Loading