Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Match v6 lockfile behaviour in v9 around injected deps #1924

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Silic0nS0ldier
Copy link

This PR makes a few tweaks to how package references are acquired and processed such that handling of injected dependencies (unsupported, but non-fatal with v6 lockfile) works as expected with the new v9 lockfile.

Additionally;

  • Adds .git to .bazelignore (editors see .git when inspecting test logs otherwise)
  • Fixed typo file -> find in npm/private/npm_translate_lock_generate.bzl
  • Documented params for _new_package_info in npm/private/pnpm.bzl
  • Documented and tested _convert_pnpm_v6_v9_version_peer_dep in npm/private/pnpm.bzl
  • .strip over .(r|l)strip in npm/private/pnpm.bzl
  • Added tests for v6 and v9 lockfiles covering current behaviour around injected deps (unsupported feature that is non-fatal when used in v6 and now v9)
  • Remove inlined JSON in favour of Starlark + json.encode in npm/private/test/parse_pnpm_lock_tests.bzl
  • Added test suite target for //npm/private/test:test_parse_pnpm_lock

Changes are visible to end-users: yes(-ish)

  • Searched for relevant documentation and updated as needed: yes
  • Breaking change (forces users to change their own code or config):no
  • Suggested release notes appear below: no

Test plan

  • Covered by existing test cases
  • New test cases added

- Adds `.git` to `.bazelignore` (editors see `.git` when inspecting test logs otherwise)
- Fixed typo `file` -> `find` in `npm/private/npm_translate_lock_generate.bzl`
- Documented params for `_new_package_info` in `npm/private/pnpm.bzl`
- Documented and tested `_convert_pnpm_v6_v9_version_peer_dep` in `npm/private/pnpm.bzl`
- `.strip` over `.(r|l)strip` in `npm/private/pnpm.bzl`
- Added tests for v6 and v9 lockfiles covering current behaviour around injected deps (unsupported feature that is non-fatal when used in v6 and now v9)
- Remove inlined JSON in favour of Starlark + `json.encode` in `npm/private/test/parse_pnpm_lock_tests.bzl`
- Added test suite target for `//npm/private/test:test_parse_pnpm_lock`
Copy link

aspect-workflows bot commented Sep 26, 2024

Test

All tests were cache hits

200 tests (100.0%) were fully cached saving 38s.


Test

e2e/bzlmod

All tests were cache hits

5 tests (100.0%) were fully cached saving 1s.


Test

e2e/gyp_no_install_script

All tests were cache hits

2 tests (100.0%) were fully cached saving 236ms.


Test

e2e/js_image_oci

All tests were cache hits

1 test (100.0%) was fully cached saving 7s.


Test

e2e/npm_link_package

All tests were cache hits

3 tests (100.0%) were fully cached saving 839ms.


Test

e2e/npm_link_package-esm

All tests were cache hits

3 tests (100.0%) were fully cached saving 745ms.


Test

e2e/npm_translate_lock

All tests were cache hits

1 test (100.0%) was fully cached saving 27ms.


Test

e2e/npm_translate_lock_empty

All tests were cache hits

1 test (100.0%) was fully cached saving 27ms.


Test

e2e/npm_translate_lock_multi

All tests were cache hits

2 tests (100.0%) were fully cached saving 296ms.


Test

e2e/npm_translate_lock_partial_clone

All tests were cache hits

1 test (100.0%) was fully cached saving 112ms.


Test

e2e/npm_translate_lock_replace_packages

All tests were cache hits

3 tests (100.0%) were fully cached saving 716ms.


Test

e2e/npm_translate_lock_subdir_patch

All tests were cache hits

1 test (100.0%) was fully cached saving 192ms.


Test

e2e/npm_translate_package_lock

All tests were cache hits

1 test (100.0%) was fully cached saving 81ms.


Test

e2e/npm_translate_yarn_lock

All tests were cache hits

1 test (100.0%) was fully cached saving 81ms.


Test

e2e/package_json_module

All tests were cache hits

1 test (100.0%) was fully cached saving 713ms.


Test

e2e/pnpm_lockfiles

All tests were cache hits

40 tests (100.0%) were fully cached saving 3s.


Test

e2e/pnpm_workspace

All tests were cache hits

10 tests (100.0%) were fully cached saving 2s.


Test

e2e/pnpm_workspace_rerooted

All tests were cache hits

10 tests (100.0%) were fully cached saving 2s.


Test

e2e/repo_mapping

All tests were cache hits

2 tests (100.0%) were fully cached saving 233ms.


Test

e2e/rules_foo

All tests were cache hits

2 tests (100.0%) were fully cached saving 141ms.


Test

e2e/vendored_node

All tests were cache hits

1 test (100.0%) was fully cached saving 171ms.


Buildifier      Format

@@ -485,8 +508,20 @@ def _convert_v9_packages(packages, snapshots):
# package_data can have the resolved "version" for things like https:// deps
friendly_version = package_data["version"] if "version" in package_data else static_key[version_index + 1:]

package_id = package_snapshot.get("id", None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any tests that show the side-effects of this change? Can we see that in a diff?

@@ -499,6 +534,11 @@ def _convert_v9_packages(packages, snapshots):
resolution = package_data.get("resolution"),
)

# Match v6 lockfile behaviour with local dependency keys
if "file:" in package_key:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This as well. I'd like to see this change along with a test demonstrating the side-effects...

@Silic0nS0ldier
Copy link
Author

Quick update as I've not been able to return to this as soon as I would have liked, and likely won't be able to until the internal project this change was made for is complete.

As for the questions, I don't believe any of the existing tests demonstrate the issue. New tests cases in this PR do (partially, they don't show the final label but they do show the malformed data that leads generation of the bad label), however since this PR also aligns v9 with v6 that isn't obvious.

Splitting the test changes into a separate PR and merging that first (with the erroneous outputs) would it a lot more obvious exactly what this PR changes. If this is acceptable, I'll be happy to implement the split once I'm back on this.

@jbedard
Copy link
Member

jbedard commented Oct 19, 2024

Splitting the PR into multiple would be great and make this a lot easier to merge one thing at a time.

The minor cleanup/refactoring/comments can probably be done in a few isolated PRs?
I think some of the tests can be added in their own PRs?

Then the actual fixes + tests (which failed before the fix) in their own PRs.

Also checkout #1989, does that overlap with your file: changes in here at all?

@Silic0nS0ldier
Copy link
Author

Tests (and other trivial changes) split to #2004

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants