Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Timeline offloading persistence #9444

Merged
merged 33 commits into from
Oct 22, 2024
Merged

Timeline offloading persistence #9444

merged 33 commits into from
Oct 22, 2024

Commits on Oct 17, 2024

  1. util functions

    @arpad-m
    arpad-m committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    407e644 View commit details
    Browse the repository at this point in the history

  2. wip

    @arpad-m
    arpad-m committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    82202ac View commit details
    Browse the repository at this point in the history

  3. Implement the remainder of offloading persistence

    @arpad-m
    arpad-m committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    fa3e490 View commit details
    Browse the repository at this point in the history

  4. comments

    @arpad-m
    arpad-m committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    116efe2 View commit details
    Browse the repository at this point in the history

  5. comment

    @arpad-m
    arpad-m committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    b9d33fb View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2024

  1. Add test for persistence of offloaded state 

    The test has some smaller modifications from test_timeline_offloading:

* the method by which we determine the offloaded state
* us doing manual offloading all the time
* only one offloaded branch
* a test in the end that deletion of a tenant works (so timeline deletion
  deletes the manifest as well)
    @arpad-m
    arpad-m committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    fe0a1ea View commit details
    Browse the repository at this point in the history

  2. Add archived_at field for future flattening logic

    @arpad-m
    arpad-m committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    0bf6b0a View commit details
    Browse the repository at this point in the history

  3. clippy

    @arpad-m
    arpad-m committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    149f394 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. comment

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    b78a79b View commit details
    Browse the repository at this point in the history

  2. Make tests pass

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    115e717 View commit details
    Browse the repository at this point in the history

  3. Ensure the timeline is archived upon offloading it

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    bf2bdb3 View commit details
    Browse the repository at this point in the history

  4. fmt and doc fix

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    9623a7d View commit details
    Browse the repository at this point in the history

  5. Address some review comments

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    8476799 View commit details
    Browse the repository at this point in the history

  6. Use TENANT_MANIFEST instead of Generation::none

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    542642c View commit details
    Browse the repository at this point in the history

  7. Merge remote-tracking branch 'origin/main' into arpad/offload_persist… 

    …ence
    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    64efdfd View commit details
    Browse the repository at this point in the history

  8. Add "archived_at" field

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    a99b865 View commit details
    Browse the repository at this point in the history

  9. Add back paragraph

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    88fc3da View commit details
    Browse the repository at this point in the history

  10. Put imports to top level

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    958b356 View commit details
    Browse the repository at this point in the history

  11. Comments

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    63e0cbf View commit details
    Browse the repository at this point in the history

  12. Add comment

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    4063340 View commit details
    Browse the repository at this point in the history

  13. Move retry_forever_download_to_vec into GenericRemoteStorage

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    d4fec5f View commit details
    Browse the repository at this point in the history

  14. Make RemoteTimelineClient be lazily constructed

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    b458926 View commit details
    Browse the repository at this point in the history

  15. Fix

    @arpad-m
    arpad-m committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    0e03442 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. Fix and rename manifest path

    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    ba92ed2 View commit details
    Browse the repository at this point in the history

  2. fmt

    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    99451bf View commit details
    Browse the repository at this point in the history

  3. Default for deletion progress

    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    3206740 View commit details
    Browse the repository at this point in the history

  4. Revert "Move retry_forever_download_to_vec into GenericRemoteStorage" 

    This reverts commit d4fec5f.
    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    795f05a View commit details
    Browse the repository at this point in the history

  5. Add UploadQueueNotReadyError and do upload_tenant_manifest in shard s… 

    …plitting
    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    ea949e2 View commit details
    Browse the repository at this point in the history

  6. Delete offloaded timelines from the manifest 

    Also, handle dangling references in the tenant loading code
    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    0949d3e View commit details
    Browse the repository at this point in the history

  7. fmt

    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    5b29d28 View commit details
    Browse the repository at this point in the history

  8. Adjustments

    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    7ac1c43 View commit details
    Browse the repository at this point in the history

  9. fix and improve logging

    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    08159f2 View commit details
    Browse the repository at this point in the history

  10. Fixes

    @arpad-m
    arpad-m committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    5ad1198 View commit details
    Browse the repository at this point in the history