Bucket import and creation inconsistency & delete_before_replace option not working properly #1065

cristiciortea · 2023-04-21T11:49:32Z

What happened?

I encountered an issue while trying to manage Google Cloud Storage buckets using Pulumi with the pulumi-gcp provider. My goal is to import an existing bucket, and then create a bucket with the same parameters (in my usecase, this happens when I import the bucket and then when I update my infrastructure). However, I'm facing two main problems:

After importing the existing bucket and attempting to create a new one with identical parameters, Pulumi still wants to replace the existing bucket. This suggests that there is a non-deterministic behavior in the way buckets are created and imported.

Furthermore the delete_before_replace=True option does not seem to function as expected. Pulumi attempts to replace the existing bucket but fails to delete it first, resulting in an error:
Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict

Stack trace

Updating (test_stack):

    pulumi:pulumi:Stack test-project-bucket-test_stack running
 ++ gcp:storage:Bucket nuc-test-bucket-0001 creating replacement [diff: +__defaults,forceDestroy~cors]
 ++ gcp:storage:Bucket nuc-test-bucket-0001 creating replacement [diff: +__defaults,forceDestroy~cors]; error: 1 error occurred:
 ++ gcp:storage:Bucket nuc-test-bucket-0001 **creating failed** [diff: +__defaults,forceDestroy~cors]; error: 1 error occurred:
    pulumi:pulumi:Stack test-project-bucket-test_stack running error: update failed
    pulumi:pulumi:Stack test-project-bucket-test_stack running Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict
    pulumi:pulumi:Stack test-project-bucket-test_stack **failed** 1 error; 1 message

Diagnostics:
  pulumi:pulumi:Stack (test-project-bucket-test_stack):
    Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict

    error: update failed

  gcp:storage:Bucket (nuc-test-bucket-0001):
    error: 1 error occurred:
    	* googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict

Resources:
    1 unchanged

Duration: 2s

error: Resource monitor has terminated, shutting down

CommandError                              Traceback (most recent call last)
Cell In[3], line 1
----> 1 auto.select_stack("test_stack", program=program_create, project_name="test-project-bucket").up(on_output=print, diff=False)

File /usr/local/lib/python3.9/site-packages/pulumi/automation/_stack.py:290, in Stack.up(self, parallel, message, target, policy_packs, policy_pack_configs, expect_no_changes, diff, target_dependents, replace, color, on_output, on_event, program, plan, show_secrets)
    287     log_watcher_thread.start()
    289 try:
--> 290     up_result = self._run_pulumi_cmd_sync(args, on_output)
    291     outputs = self.outputs()
    292     summary = self.info(show_secrets)

File /usr/local/lib/python3.9/site-packages/pulumi/automation/_stack.py:683, in Stack._run_pulumi_cmd_sync(self, args, on_output)
    681 args.extend(additional_args)
    682 args.extend(["--stack", self.name])
--> 683 result = _run_pulumi_cmd(args, self.workspace.work_dir, envs, on_output)
    684 self.workspace.post_command_callback(self.name)
    685 return result

File /usr/local/lib/python3.9/site-packages/pulumi/automation/_cmd.py:78, in _run_pulumi_cmd(args, cwd, additional_env, on_output)
     74 result = CommandResult(
     75     stderr=stderr_contents, stdout="\n".join(stdout_chunks), code=code
     76 )
     77 if code != 0:
---> 78     raise create_command_error(result)
     80 return result

CommandError: 
 code: 255
 stdout: Updating (test_stack):

    pulumi:pulumi:Stack test-project-bucket-test_stack running
 ++ gcp:storage:Bucket nuc-test-bucket-0001 creating replacement [diff: +__defaults,forceDestroy~cors]
 ++ gcp:storage:Bucket nuc-test-bucket-0001 creating replacement [diff: +__defaults,forceDestroy~cors]; error: 1 error occurred:
 ++ gcp:storage:Bucket nuc-test-bucket-0001 **creating failed** [diff: +__defaults,forceDestroy~cors]; error: 1 error occurred:
    pulumi:pulumi:Stack test-project-bucket-test_stack running error: update failed
    pulumi:pulumi:Stack test-project-bucket-test_stack running Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict
    pulumi:pulumi:Stack test-project-bucket-test_stack **failed** 1 error; 1 message

Diagnostics:
  pulumi:pulumi:Stack (test-project-bucket-test_stack):
    Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict

    error: update failed

  gcp:storage:Bucket (nuc-test-bucket-0001):
    error: 1 error occurred:
    	* googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict

Resources:
    1 unchanged

Duration: 2s

 stderr: warning: A new version of Pulumi is available. To upgrade from version '3.30.0' to '3.64.0', visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.

Expected Behavior

The expected behavior for the given scenario should have been as follows:

Upon importing the existing bucket with the same parameters, Pulumi should recognize that the new bucket has the same configuration as the existing one and avoid any replacement. Essentially, it should detect no differences between the two configurations, treating them as identical.

If there was any difference between the configurations and Pulumi decided to replace the existing bucket, the delete_before_replace=True option should have instructed Pulumi to first delete the existing bucket and then create a new one with the updated configuration. This would have avoided the conflict error (Error 409) caused by attempting to create a bucket with the same name as an existing one.

Steps to reproduce

In linux:
mkdir -p /pulumi-work-dir-testing/workspace_test
cd /pulumi-work-dir-testing/workspace_test
echo "description: Test infrastracture\n name: test-project-bucket\n runtime: python" >> Pulumi.yaml
pulumi login file:///pulumi-work-dir-testing/workspace_test
pulumi stack init test_stack

In ipython interractive shell:

from pulumi import automation as auto
import pulumi_gcp
import os
import pulumi

def program_get():
    bucket_name = 'nuc-test-bucket-0001'  # this already exists in google cloud platform
    buckets_location = 'europe-west4'
    storage_class = 'Standard'

    # Import the existent bucket
    pulumi_gcp.storage.Bucket.get(
                              resource_name=bucket_name,
                              id=bucket_name,
                              cors=[pulumi_gcp.storage.BucketCorArgs(
                                  max_age_seconds=3600,
                                  methods=[
                                      "GET",
                                      "HEAD",
                                      "PUT",
                                      "POST",
                                      "DELETE",
                                  ],
                              )],
                              location=buckets_location,
                              name=bucket_name,
                              project=os.environ["GOOGLE_PROJECT"],
                              storage_class=storage_class,
                              uniform_bucket_level_access=True,
                              opts=pulumi.ResourceOptions(delete_before_replace=True),
    )

def program_create():
    bucket_name = 'nuc-test-bucket-0001'  # this already exists in google cloud platform
    buckets_location = 'europe-west4'
    storage_class = 'Standard'
    pulumi_gcp.storage.Bucket(
        resource_name=bucket_name,
        cors=[
            pulumi_gcp.storage.BucketCorArgs(
                max_age_seconds=3600,
                methods=[
                    "GET",
                    "HEAD",
                    "PUT",
                    "POST",
                    "DELETE",
                ],
            )
        ],
        name=bucket_name,
        force_destroy=True,
        location=buckets_location,
        project=os.environ["GOOGLE_PROJECT"],
        storage_class=storage_class,
        uniform_bucket_level_access=True,
        opts=pulumi.ResourceOptions(delete_before_replace=True),
    )

auto.select_stack("test_stack", program=program_get, project_name="test-project-bucket").up(on_output=print, diff=False)
auto.select_stack("test_stack", program=program_create, project_name="test-project-bucket").up(on_output=print, diff=False)

Output of `pulumi about`

In [4]: !pulumi about
CLI
Version 3.30.0
Go Version go1.17.9
Go Compiler gc

Plugins
NAME VERSION
gcp 6.53.0
kubernetes 3.24.2
python unknown

Host
OS debian
Version 11.1
Arch x86_64

This project is written in python (/usr/local/bin/python3 v3.9.7)

Current Stack: test_stack

TYPE URN
pulumi:pulumi:Stack urn:pulumi:test_stack::test-project-bucket::pulumi:pulumi:Stack::test-project-bucket-test_stack
pulumi:providers:gcp urn:pulumi:test_stack::test-project-bucket::pulumi:providers:gcp::default_6_53_0
gcp:storage/bucket:Bucket urn:pulumi:test_stack::test-project-bucket::gcp:storage/bucket:Bucket::nuc-test-bucket-0001

Found no pending operations associated with test_stack

Backend
Name backend-54777f96d-z69pz
URL file:///pulumi-work-dir-testing/workspace_test
User root
Organizations

NAME VERSION
central-backend 0.1.0
ipython 8.12.0
pip 21.2.4
pre-commit 2.21.0
pydevd-pycharm 222.4554.11
pytest-aioresponses 0.2.0
pytest-dotenv 0.5.2
ruff 0.0.254
types-toml 0.10.8.6
wheel 0.37.0

Pulumi locates its logs in /tmp by default
warning: A new version of Pulumi is available. To upgrade from version '3.30.0' to '3.64.0', visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.

Additional context

I found a workaround, if import the bucket with pulumi import instead of pulumi_gcp.storage.Bucket.get, this bug/s do not reproduce.

Additionally, if I attempt to import the bucket using the following approach, the issue still persists:
pulumi_gcp.storage.Bucket( resource_name=bucket_name, cors=[pulumi_gcp.storage.BucketCorArgs( max_age_seconds=3600, methods=[ "GET", "HEAD", "PUT", "POST", "DELETE", ], )], location=buckets_location, name=bucket_name, project=os.environ["GOOGLE_PROJECT"], storage_class=storage_class, uniform_bucket_level_access=True, opts=pulumi.ResourceOptions(delete_before_replace=True, id=bucket_name, import_=bucket_name), )

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

The text was updated successfully, but these errors were encountered:

rquitales · 2023-04-24T19:13:09Z

@cristiciortea Apologies for this issue that you're running into. It appears that you may have gotten closer with a likely solution in the additional context that you have provided.

When importing a GCP storage bucket, and having it managed by Pulumi, you will need to just use pulumi_gcp.storage.Bucket, whereas the pulumi_gcp.storage.Bucket.get method is simply just a GET API request that does not import information into state.

Would you be able to provide the code you're using when you just did:

pulumi_gcp.storage.Bucket(..., opts=pulumi.ResourceOptions(..., import_=bucket_name), )

In particular, it'd be helpful for me to see how you're importing your resources and attempting to update it using this approach.

I've attempted to reproduce what you're trying to attempt, and was able to successfully import and create the bucket without issues. Here is a minimal code for reference:

from pulumi import automation as auto
import pulumi_gcp
import os
import pulumi

def program_get() -> pulumi_gcp.storage.Bucket:
    bucket_name = 'my-bucket'  # this already exists in google cloud platform
    buckets_location = 'us-central1'

    # Import the existent bucket.
    pulumi_gcp.storage.Bucket(
                              resource_name=bucket_name,
                              location=buckets_location,
                              name=bucket_name,
                              project=os.environ["GOOGLE_PROJECT"],
                              opts=pulumi.ResourceOptions(import_=bucket_name), # Note the import_ option here.
    )

def program_create():
    bucket_name = 'my-bucket'  # this already exists in google cloud platform
    buckets_location = 'us-central1'
    pulumi_gcp.storage.Bucket(
                              resource_name=bucket_name,
                              location=buckets_location,
                              name=bucket_name,
                              force_destroy=True,
                              project=os.environ["GOOGLE_PROJECT"],
                              opts=pulumi.ResourceOptions(delete_before_replace=True),
    )

auto.create_or_select_stack("test_stack", program=program_get, project_name="my-proj").up(on_output=print, diff=False)
auto.create_or_select_stack("test_stack", program=program_create, project_name="my-proj").up(on_output=print, diff=False)

The error that is returned by the Google API in your example: Your previous request to create the named bucket succeeded and you already own it., conflict indicates that the import you're trying to do was not done correctly, and its state is not stored for Pulumi to use. As such, when you attempt to update the bucket, Pulumi will attempt to create the bucket from scratch, rather than attempting to update the existing pre-created bucket, hence resulting in the conflict error.

cristiciortea · 2023-04-25T17:01:40Z

Thank you for taking the time to look into my issue @rquitales .

First of all, if I try to use only the import_ parameter in the opts=pulumi.ResourceOptions, I will get this error gcp:storage:Bucket nuc-test-bucket-0001 importing error: inputs to import do not match the existing resource.
Full snippet from ipython shell:

In [37]: from pulumi import automation as auto
    ...: import pulumi_gcp
    ...: import os
    ...: import pulumi
    ...: 
    ...: 
    ...: def program_get():
    ...:     bucket_name = 'nuc-test-bucket-0001'  # this already exists in google cloud platform
    ...:     buckets_location = 'europe-west4'
    ...:     storage_class = 'Standard'
    ...: 
    ...:     # Import the existent bucket
    ...:     pulumi_gcp.storage.Bucket(
    ...:         resource_name=bucket_name,
    ...:         cors=[pulumi_gcp.storage.BucketCorArgs(
    ...:             max_age_seconds=3600,
    ...:             methods=[
    ...:                 "GET",
    ...:                 "HEAD",
    ...:                 "PUT",
    ...:                 "POST",
    ...:                 "DELETE",
    ...:             ],
    ...:         )],
    ...:         location=buckets_location,
    ...:         name=bucket_name,
    ...:         project=os.environ["GOOGLE_PROJECT"],
    ...:         storage_class=storage_class,
    ...:         uniform_bucket_level_access=True,
    ...:         opts=pulumi.ResourceOptions(delete_before_replace=True,
    ...:                                     import_=bucket_name),
    ...:     )

In [38]: auto.select_stack("test_stack", program=program_get, project_name="test-project-bucket").up(on_output=print, diff=False)
    ...: 
Updating (test_stack):

 +  pulumi:pulumi:Stack test-project-bucket-test_stack creating
 =  gcp:storage:Bucket nuc-test-bucket-0001 importing
 =  gcp:storage:Bucket nuc-test-bucket-0001 importing error: inputs to import do not match the existing resource
 =  gcp:storage:Bucket nuc-test-bucket-0001 **importing failed** error: inputs to import do not match the existing resource
 +  pulumi:pulumi:Stack test-project-bucket-test_stack creating error: update failed
 +  pulumi:pulumi:Stack test-project-bucket-test_stack **creating failed** 1 error

Diagnostics:
  pulumi:pulumi:Stack (test-project-bucket-test_stack):
    error: update failed

  gcp:storage:Bucket (nuc-test-bucket-0001):
    error: inputs to import do not match the existing resource

Resources:
    + 1 created

But if I add, the id parameter on pulumi.ResourceOptions, I have no problem importing the Bucket.
ipython snippet:

In [39]: def program_get():
    ...:     bucket_name = 'nuc-test-bucket-0001'  # this already exists in google cloud platform
    ...:     buckets_location = 'europe-west4'
    ...:     storage_class = 'Standard'
    ...: 
    ...:     # Import the existent bucket
    ...:     pulumi_gcp.storage.Bucket(
    ...:         resource_name=bucket_name,
    ...:         cors=[pulumi_gcp.storage.BucketCorArgs(
    ...:             max_age_seconds=3600,
    ...:             methods=[
    ...:                 "GET",
    ...:                 "HEAD",
    ...:                 "PUT",
    ...:                 "POST",
    ...:                 "DELETE",
    ...:             ],
    ...:         )],
    ...:         location=buckets_location,
    ...:         name=bucket_name,
    ...:         project=os.environ["GOOGLE_PROJECT"],
    ...:         storage_class=storage_class,
    ...:         uniform_bucket_level_access=True,
    ...:         opts=pulumi.ResourceOptions(delete_before_replace=True,
    ...:                                     id=bucket_name,  # added the id parameter
    ...:                                     import_=bucket_name),
    ...:     )

In [40]: 

In [40]: auto.select_stack("test_stack", program=program_get, project_name="test-project-bucket").up(on_output=print, diff=False)
Updating (test_stack):

    pulumi:pulumi:Stack test-project-bucket-test_stack running
    pulumi:pulumi:Stack test-project-bucket-test_stack running read gcp:storage:Bucket nuc-test-bucket-0001
    pulumi:pulumi:Stack test-project-bucket-test_stack running read gcp:storage:Bucket nuc-test-bucket-0001
    pulumi:pulumi:Stack test-project-bucket-test_stack

Resources:
    1 unchanged

Duration: 1s

The issue now, is that I am unable to create the bucket even after importing it using the import_ and id parameters with pulumi.ResourceOptions; I will get this error as a result when I try to create it pulumi:pulumi:Stack test-project-bucket-test_stack running Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict
Full snippet:

In [41]: def program_create():
    ...:     bucket_name = 'nuc-test-bucket-0001'  # this already exists in google cloud platform
    ...:     buckets_location = 'europe-west4'
    ...:     storage_class = 'Standard'
    ...:     pulumi_gcp.storage.Bucket(
    ...:         resource_name=bucket_name,
    ...:         cors=[
    ...:             pulumi_gcp.storage.BucketCorArgs(
    ...:                 max_age_seconds=3600,
    ...:                 methods=[
    ...:                     "GET",
    ...:                     "HEAD",
    ...:                     "PUT",
    ...:                     "POST",
    ...:                     "DELETE",
    ...:                 ],
    ...:             )
    ...:         ],
    ...:         name=bucket_name,
    ...:         force_destroy=True,
    ...:         location=buckets_location,
    ...:         project=os.environ["GOOGLE_PROJECT"],
    ...:         storage_class=storage_class,
    ...:         uniform_bucket_level_access=True,
    ...:         opts=pulumi.ResourceOptions(delete_before_replace=True),
    ...:     )
    ...: 

In [42]: 

In [42]: auto.select_stack("test_stack", program=program_create, project_name="test-project-bucket").up(on_output=print, diff=False)
    ...: 
Updating (test_stack):

    pulumi:pulumi:Stack test-project-bucket-test_stack running
 ++ gcp:storage:Bucket nuc-test-bucket-0001 creating replacement [diff: +__defaults,cors,forceDestroy,location,name,project,storageClass,uniformBucketLevelAccess]
 ++ gcp:storage:Bucket nuc-test-bucket-0001 creating replacement [diff: +__defaults,cors,forceDestroy,location,name,project,storageClass,uniformBucketLevelAccess]; error: 1 error occurred:
 ++ gcp:storage:Bucket nuc-test-bucket-0001 **creating failed** [diff: +__defaults,cors,forceDestroy,location,name,project,storageClass,uniformBucketLevelAccess]; error: 1 error occurred:
    pulumi:pulumi:Stack test-project-bucket-test_stack running error: update failed
    pulumi:pulumi:Stack test-project-bucket-test_stack running Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict
    pulumi:pulumi:Stack test-project-bucket-test_stack **failed** 1 error; 1 message

Diagnostics:
  pulumi:pulumi:Stack (test-project-bucket-test_stack):
    error: update failed

    Error creating bucket nuc-test-bucket-0001: googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict

  gcp:storage:Bucket (nuc-test-bucket-0001):
    error: 1 error occurred:
    	* googleapi: Error 409: Your previous request to create the named bucket succeeded and you already own it., conflict

Resources:
    1 unchanged

Duration: 2s

I also want to highlight this line from the logs, which clearly shows that pulumi is trying to create a replacement of the currently existing bucket:
++ gcp:storage:Bucket nuc-test-bucket-0001 creating replacement [diff: +__defaults,cors,forceDestroy,location,name,project,storageClass,uniformBucketLevelAccess] ++ gcp:storage:Bucket nuc-test-bucket-0001 **creating failed** [diff: +__defaults,cors,forceDestroy,location,name,project,storageClass,uniformBucketLevelAccess]; error: 1 error occurred:

mjeffryes · 2024-09-25T22:27:43Z

Unfortunately, it looks like this issue hasn't seen any updates in a while. If you're still encountering this problem, could you leave a quick comment to let us know so we can prioritize it?

cristiciortea added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Apr 21, 2023

rquitales added awaiting-feedback area/import An issue related to `pulumi import` or the import resource option. and removed needs-triage Needs attention from the triage team labels Apr 24, 2023

mikhailshilkov removed the awaiting-feedback label Jul 28, 2023

mjeffryes added the awaiting-feedback Blocked on input from the author label Sep 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bucket import and creation inconsistency & delete_before_replace option not working properly #1065

Bucket import and creation inconsistency & delete_before_replace option not working properly #1065

cristiciortea commented Apr 21, 2023 •

edited

Loading

rquitales commented Apr 24, 2023

cristiciortea commented Apr 25, 2023 •

edited

Loading

mjeffryes commented Sep 25, 2024

Bucket import and creation inconsistency & delete_before_replace option not working properly #1065

Bucket import and creation inconsistency & delete_before_replace option not working properly #1065

Comments

cristiciortea commented Apr 21, 2023 • edited Loading

What happened?

Expected Behavior

Steps to reproduce

Output of pulumi about

Additional context

Contributing

rquitales commented Apr 24, 2023

cristiciortea commented Apr 25, 2023 • edited Loading

mjeffryes commented Sep 25, 2024

cristiciortea commented Apr 21, 2023 •

edited

Loading

Output of `pulumi about`

cristiciortea commented Apr 25, 2023 •

edited

Loading