Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Knob extractor spark props #470

Merged
merged 3 commits into from
Nov 12, 2024
Merged

Knob extractor spark props #470

merged 3 commits into from
Nov 12, 2024

Conversation

JoschD
Copy link
Member

@JoschD JoschD commented Nov 11, 2024

This fixes

15:43:43 - omc3.knob_extractor STDERR: Traceback (most recent call last):
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_acc_py/base/2021.12/lib/python3.9/runpy.py", line 197, in _run_module_as_main
15:43:43 - omc3.knob_extractor STDERR:     return _run_code(code, main_globals, None,
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_acc_py/base/2021.12/lib/python3.9/runpy.py", line 87, in _run_code
15:43:43 - omc3.knob_extractor STDERR:     exec(code, run_globals)
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_python3/lib/python3.9/site-packages/omc3/knob_extractor.py", line 633, in <module>
15:43:43 - omc3.knob_extractor STDERR:     main()
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_python3/lib/python3.9/site-packages/generic_parser/entrypoint_parser.py", line 451, in wrapper
15:43:43 - omc3.knob_extractor STDERR:     return func(self.parse(*args, **kwargs))
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_python3/lib/python3.9/site-packages/omc3/knob_extractor.py", line 286, in main
15:43:43 - omc3.knob_extractor STDERR:     ldb = pytimber.LoggingDB(source="nxcals", loglevel=logging.ERROR, 
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_python3/lib/python3.9/site-packages/pytimber/pytimber.py", line 1017, in __init__
15:43:43 - omc3.knob_extractor STDERR:     super().__init__(
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_python3/lib/python3.9/site-packages/pytimber/pytimber.py", line 108, in __init__
15:43:43 - omc3.knob_extractor STDERR:     self._spark = spark_session or _get_spark_session(
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_python3/lib/python3.9/site-packages/pytimber/spark_session_utils.py", line 34, in _get_spark_session
15:43:43 - omc3.knob_extractor STDERR:     maybe_spark_session = try_to_get_or_create(*args, **kwargs)
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_python3/lib/python3.9/site-packages/pytimber/spark_session_utils.py", line 30, in try_to_get_or_create
15:43:43 - omc3.knob_extractor STDERR:     return _get_or_create(flavor=flavor, *args, **kwargs)
15:43:43 - omc3.knob_extractor STDERR:   File "/afs/cern.ch/eng/sl/lintrack/omc_python3/lib/python3.9/site-packages/nxcals/spark_session_builder/__init__.py", line 189, in get_or_create
15:43:43 - omc3.knob_extractor STDERR:     for key, value in conf.items():
15:43:43 - omc3.knob_extractor STDERR: AttributeError: 'set' object has no attribute 'items'

@JoschD JoschD added Type: Bug Something isn't working as it should. Estimate: Easy Good first issue for newcomers. Straightforward fixes. Type: Release Issue preparing for a release. Status: Review Needed Work currently stopped, untils someone else reviews it. Priority: Critical Work on this!! labels Nov 11, 2024
@JoschD JoschD self-assigned this Nov 11, 2024
@JoschD JoschD requested a review from a team as a code owner November 11, 2024 14:54
Copy link

codeclimate bot commented Nov 11, 2024

Code Climate has analyzed commit 0eec467 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 86.0% (0.0% change).

View more on Code Climate.

Copy link
Member

@fsoubelet fsoubelet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, I had not caught this either in the previous PR.

@JoschD JoschD merged commit 2f56871 into master Nov 12, 2024
37 checks passed
@JoschD JoschD deleted the knob_extractor_spark_props branch November 12, 2024 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Estimate: Easy Good first issue for newcomers. Straightforward fixes. Priority: Critical Work on this!! Status: Review Needed Work currently stopped, untils someone else reviews it. Type: Bug Something isn't working as it should. Type: Release Issue preparing for a release.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants