Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: rename control plane client & chunk validation requests #8997

Merged
merged 5 commits into from
Sep 26, 2024

Conversation

jcsp
Copy link
Collaborator

@jcsp jcsp commented Sep 13, 2024

Problem

  • In storage controller: read from database in validate API #8784, the validate controller API is modified to check generations directly in the database. It batches tenants into separate queries to avoid generating a huge statement, but
  • While updating this, I realized that "control_plane_client" is a kind of confusing name for the client code now that it primarily talks to the storage controller (the case of talking to the control plane will go away in a few months).

Summary of changes

  • Big rename to "ControllerUpcallClient" -- this reflects the storage controller's api naming, where the paths used by the pageserver are in /upcall/
  • When sending validate requests, break them up into chunks so that we avoid possible edge cases of generating any HTTP requests that require database I/O across many thousands of tenants.

This PR mixes a functional change with a refactor, but the commits are cleanly separated -- only the last commit is a functional change.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

Copy link

@orca-security-us orca-security-us bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

Copy link

github-actions bot commented Sep 13, 2024

5013 tests run: 4849 passed, 0 failed, 164 skipped (full report)


Flaky tests (4)

Postgres 16

Postgres 15

Code coverage* (full report)

  • functions: 32.0% (7491 of 23395 functions)
  • lines: 50.0% (60478 of 120871 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
0733a2e at 2024-09-26T09:22:04.082Z :recycle:

@jcsp jcsp added c/storage/pageserver Component: storage: pageserver a/tech_debt Area: related to tech debt labels Sep 19, 2024
@jcsp jcsp changed the title Jcsp/upcall cleanup pageserver: rename control plane client & batch validation requests Sep 19, 2024
@jcsp jcsp changed the title pageserver: rename control plane client & batch validation requests pageserver: rename control plane client & chunk validation requests Sep 19, 2024
@jcsp jcsp marked this pull request as ready for review September 19, 2024 16:04
@jcsp jcsp requested a review from a team as a code owner September 19, 2024 16:04
@jcsp jcsp requested a review from problame September 19, 2024 16:04
@jcsp jcsp enabled auto-merge (squash) September 24, 2024 12:47
@jcsp jcsp merged commit 3d255d6 into main Sep 26, 2024
83 checks passed
@jcsp jcsp deleted the jcsp/upcall-cleanup branch September 26, 2024 15:06
bayandin pushed a commit that referenced this pull request Sep 26, 2024
…8997)

## Problem

- In #8784, the validate
controller API is modified to check generations directly in the
database. It batches tenants into separate queries to avoid generating a
huge statement, but
- While updating this, I realized that "control_plane_client" is a kind
of confusing name for the client code now that it primarily talks to the
storage controller (the case of talking to the control plane will go
away in a few months).

## Summary of changes

- Big rename to "ControllerUpcallClient" -- this reflects the storage
controller's api naming, where the paths used by the pageserver are in
`/upcall/`
- When sending validate requests, break them up into chunks so that we
avoid possible edge cases of generating any HTTP requests that require
database I/O across many thousands of tenants.

This PR mixes a functional change with a refactor, but the commits are
cleanly separated -- only the last commit is a functional change.

---------

Co-authored-by: Christian Schwarz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/tech_debt Area: related to tech debt c/storage/pageserver Component: storage: pageserver
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants