Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experimental support for pg_anon #9706

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

luist18
Copy link

@luist18 luist18 commented Nov 10, 2024

Problem

Summary of changes

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@luist18 luist18 self-assigned this Nov 10, 2024
@luist18 luist18 force-pushed the luist18/postgresql_anonymizer-experiments branch from 75b83d2 to f54250b Compare November 10, 2024 15:30
@luist18 luist18 added the approved-for-ci-run Changes are safe to trigger CI for the PR label Nov 10, 2024
@github-actions github-actions bot removed the approved-for-ci-run Changes are safe to trigger CI for the PR label Nov 10, 2024
@vipvap vipvap mentioned this pull request Nov 10, 2024
Copy link

github-actions bot commented Nov 10, 2024

5589 tests run: 5367 passed, 0 failed, 222 skipped (full report)


Flaky tests (1)

Postgres 17

Code coverage* (full report)

  • functions: 31.7% (7871 of 24806 functions)
  • lines: 49.4% (62281 of 126036 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
d202885 at 2024-11-11T13:55:05.996Z :recycle:

@luist18 luist18 force-pushed the luist18/postgresql_anonymizer-experiments branch from f54250b to d9da270 Compare November 10, 2024 16:25
@bayandin
Copy link
Member

bayandin commented Nov 10, 2024

@luist18 approved-for-ci-run label is to handle PRs from external contributors, see https://github.com/neondatabase/neon/blob/main/CONTRIBUTING.md#how-to-run-a-ci-pipeline-on-pull-requests-from-external-contributors

You don't need to use it, CI works for your's PRs straight away

@luist18 luist18 force-pushed the luist18/postgresql_anonymizer-experiments branch from 9446c2e to 43a4b9e Compare November 10, 2024 20:50
@luist18 luist18 force-pushed the luist18/postgresql_anonymizer-experiments branch 3 times, most recently from fc14d32 to f76582f Compare November 10, 2024 21:54
@luist18 luist18 force-pushed the luist18/postgresql_anonymizer-experiments branch from f76582f to 4845f4f Compare November 10, 2024 22:49
@luist18 luist18 force-pushed the luist18/postgresql_anonymizer-experiments branch from e91a3fb to d202885 Compare November 11, 2024 11:42
Copy link
Member

@bayandin bayandin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for reviewing in draft PR, I've just spotted a couple of things, and wanted to let you know ❤️

# This is an experimental extension, never got to real production.
# !Do not remove! It can be present in shared_preload_libraries and compute will fail to start if library is not found.
ENV PATH="/usr/local/pgsql/bin/:$PATH"
RUN wget https://github.com/luist18/postgresql_anonymizer/archive/refs/heads/add-guc-hook.tar.gz -O pg_anon.tar.gz && \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note, that this should be moved to neondatabase-labs/neondatabase before merging (and preferably tagged / used from a commit hash)

Comment on lines +1074 to +1078
echo 'trusted = true' >> /usr/local/pgsql/share/extension/anon.control && \
find /usr/local/pgsql -type f | sed 's|^/usr/local/pgsql/||' > /after.txt &&\
mkdir -p /extensions/anon && cp /usr/local/pgsql/share/extension/anon.control /extensions/anon && \
sort -o /before.txt /before.txt && sort -o /after.txt /after.txt && \
comm -13 /before.txt /after.txt | tar --directory=/usr/local/pgsql --zstd -cf /extensions/anon.tar.zst -T -
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 4 lines are not necessary (they were added to make anon a "custom" extension, which lives in a separate repo; here, it's a no-op); we can remove them.

Suggested change
echo 'trusted = true' >> /usr/local/pgsql/share/extension/anon.control && \
find /usr/local/pgsql -type f | sed 's|^/usr/local/pgsql/||' > /after.txt &&\
mkdir -p /extensions/anon && cp /usr/local/pgsql/share/extension/anon.control /extensions/anon && \
sort -o /before.txt /before.txt && sort -o /after.txt /after.txt && \
comm -13 /before.txt /after.txt | tar --directory=/usr/local/pgsql --zstd -cf /extensions/anon.tar.zst -T -
echo 'trusted = true' >> /usr/local/pgsql/share/extension/anon.control

RUN case "${PG_VERSION}" in "v17") \
echo "v17 extensions are not supported yet. Quit" && exit 0;; \
esac && \
patch -p1 </ext-src/pg_anon.patch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably can delete the patch as well

@luist18
Copy link
Author

luist18 commented Nov 11, 2024

@bayandin thank you for the review. I want to get the image tag to create a small internal POC. This is not supposed to be merged any time soon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants