Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arc corr and glob check fix #437

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
68f6d8b
Added arc-by-arc corrections, and new .json lists
fscarlier Mar 6, 2024
284544e
fixed passing of opt for include_ips
fscarlier Mar 6, 2024
49354ae
minor import removal
fscarlier Mar 8, 2024
2cf3400
small fixes, renaming of parameters, +fake arcbyarc test data
fscarlier Mar 8, 2024
0eead2b
version bump and flag as store_true
JoschD Mar 8, 2024
d908b34
get bpms per plane
JoschD Mar 8, 2024
0b34c5f
fixed check corrections
JoschD Mar 8, 2024
2488022
lines in correction test style
JoschD Mar 8, 2024
e861a08
plot test marker style
JoschD Mar 8, 2024
9406ea8
knob extractor hack
JoschD Mar 9, 2024
f977613
remove kq5.l2b1 from MQM_INJ_2024
JoschD Mar 13, 2024
a7cf7e9
removed kq4.r6b2
JoschD Mar 13, 2024
8c762a1
removed kq4.r6b2 also from TOP
JoschD Mar 13, 2024
5e0257a
changed optics measurements style
JoschD Mar 13, 2024
0904fae
Print exciter BPM in error message
JoschD Mar 18, 2024
b56bd02
Merge branch 'master' into arc_corr_and_glob_check_fix
JoschD Mar 18, 2024
aecbb3c
fix lhc
JoschD Mar 18, 2024
23e41b3
added NaN check
JoschD May 6, 2024
3020b80
made warning
JoschD May 6, 2024
5cfe24c
added generic accelerator
JoschD May 6, 2024
099c42d
skip coupling for generic and sps
JoschD May 6, 2024
7d4c1ad
Merge branch 'master' into arc_corr_and_glob_check_fix
JoschD Jun 5, 2024
51c2400
fixed changelog bugs
JoschD Jun 5, 2024
67570e0
Merge branch 'master' into arc_corr_and_glob_check_fix
JoschD Oct 28, 2024
5ac386f
nan in output
JoschD Oct 28, 2024
ee71a70
CHANGELOG
JoschD Oct 28, 2024
4d18da9
bad bpm message
JoschD Oct 28, 2024
c97a9ff
fix no data in plane
JoschD Oct 28, 2024
bba82c7
changelog
JoschD Oct 28, 2024
33b67ce
Merge 'main' into 'arc_corr_and_glob_check_fix'
jgray-19 Nov 12, 2024
a34f920
Merege 'master' into 'arc_corr_and_glob_check_fix'
jgray-19 Nov 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 62 additions & 44 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,25 @@
# OMC3 Changelog

#### IN PROGRESS - v0.15.0 - _jdilly_, _fscarlier_, _fesoubel_

- Fixed:
- Plot Check Corrections: Fixed Crash when normalized dispersion is in measurements
- Plot Optics Measurements: Added extra mpl style for clearer plots
- Knob extractor: fixing path hack. To be removed in the future when java is fixed on CCC machines
- LHC exciter BPM not found: Tells you which BPMs were searched for

- Added:
- Global Correction: Total phase correction arc-by-arc
- Global Correction: New MQM-knob categories `MQM_INJ_2024` and `MQM_TOP_2024` without Q4 and Q4-6
- Script to copy KMod results into optics directory

#### 2024-06-05 - v0.14.1 - _jdilly_

- Fixed:
- LHC Knobs: Fixed typo "MQSOR" to "MQSORG" in LHC Beam 2 coupling knobs.

- CI: Dropped python 3.8
- CI:
- Dropped python 3.8

#### 2024-03-18 - v0.14.0 - _jdilly_

Expand All @@ -24,26 +38,27 @@
#### 2023-12-07 - v0.13.0 - _awegsche_

- Added:
- complete overhaul of model creation, uses now `acc-models` for LHC, PS and PSB and prints
useful information about available model parameters. Can load from either a user defined path
(`--path <PATH>`) or from the afs copy of acc-models (`--afs`)
- complete overhaul of model creation, uses now `acc-models` for LHC, PS and PSB and prints
useful information about available model parameters. Can load from either a user defined path
(`--path <PATH>`) or from the afs copy of acc-models (`--afs`)

#### 2023-11-29 - v0.12.1 - _jdilly_

- Fixed:
- `tbt_converter` now also passes given output format to writer when running without noise.
- `tbt_converter` now also passes given output format to writer when running without noise.

#### 2023-11-29 - v0.12.0 - _jdilly_

- Added to harmonic analysis:
- `suffix` input parameter: adds suffix to output files, which e.g. allows running the same file
- `suffix` input parameter: adds suffix to output files, which e.g. allows running the same file
with different parameters without overwriting it.
- `bunch_ids` input parameter: in case of multibunch-files only analyse these bunches.
- `bunch_ids` input parameter: in case of multibunch-files only analyse these bunches.
If not given, all bunches will be analysed, as before.

#### 2023-09-20 - v0.11.4 - _fscarlier_, _awegsche_

- Fixed:
- bug in beta from phase (3BPM method) that calculated too high errors for first and last BPMs
- bug in beta from phase (3BPM method) that calculated too high errors for first and last BPMs

#### 2023-09-20 - v0.11.3 - _jdilly_, _awegsche_

Expand All @@ -58,25 +73,25 @@
- Plot Optics: making normalized dispersion plot a special case.

- Added:
- Plot Optics: optional input "--labels" to manually set the legend-labels.
- Plot Optics: optional input "--labels" to manually set the legend-labels.

#### 2023-06-16 - v0.11.1 - _jdilly_

- Fixed:
- OptionalString: 'None' as input is converted to None.
- Missing Kerberos config added to MANIFEST for packaging.
- Plot Optics plots now correct error-column, e.g. for beta-beating.
- Added warnings/errors for too few bpms in N-BPM/3-BPM methods.
- Added navbar to sphinx documentation.
- OptionalString: 'None' as input is converted to None.
- Missing Kerberos config added to MANIFEST for packaging.
- Plot Optics plots now correct error-column, e.g. for beta-beating.
- Added warnings/errors for too few bpms in N-BPM/3-BPM methods.
- Added navbar to sphinx documentation.

- Tests:
- Added test for the classes in omc3.plotting.utils.windows
- Added test for the classes in omc3.plotting.utils.windows

#### 2023-06-05 - v0.11.0 - _jdilly_

- Added:
- `omc3.plotting.utils.windows`: Qt-based windows and widgets for matplotlib-figure organization.
- Using the new windows in `omc3.plotting.plot_checked_corrections` and `omc3.plotting.plot_tfs`
- `omc3.plotting.utils.windows`: Qt-based windows and widgets for matplotlib-figure organization.
- Using the new windows in `omc3.plotting.plot_checked_corrections` and `omc3.plotting.plot_tfs`

#### 2023-05-15 - v0.10.0 - _jdilly_

Expand All @@ -99,21 +114,21 @@

#### 2023-04-20 - v0.8.0 - _jdilly_

- Fix:
- Fixed:
- Changed all `pandas`/`tfs-pandas` `append()` and `join()` to `concat()`
- Moved `InputFiles` into separate module

#### 2023-03-16 - v0.7.2 - _jdilly_

- Fix:
- Fixed:
- Added missing columns to coupling in BBS-OMC3 converter

#### 2023-01-20 - v0.7.1 - _jdilly_

- Added:
- Amplitude Detuning plots: Switch to plot only with/without BBQ correction

- Fix:
- Fixed:
- Second Order Amplitude Detuning fit now working
- Correct print/calculation of second order direct terms for forced
kicks in plot-labels.
Expand All @@ -140,19 +155,22 @@

#### 2022-10-12 - v0.6.4

- Fixed the phase filtering for coupling calculation to not forget columns.
- Fixed:
- phase filtering for coupling calculation will not forget columns

#### 2022-09-27 - v0.6.3

- Pandafied `knob_extractor` internally and python output.
- Added:
- pandafied `knob_extractor` internally and python output

#### 2022-09-22 - v0.6.2

- Cleaned logging in `knob_extractor`
- Added:
- cleaned logging in `knob_extractor`

#### 2022-09-21 - v0.6.1

- Added:
- Added:
- tbt output datatype for converter.

#### 2022-09-20 - v0.6.0
Expand All @@ -171,7 +189,8 @@

#### 2022-09-12 - v0.5.1

- Updated to turn_by_turn v0.4.0: Includes SPS reader
- Added:
- updated to turn_by_turn v0.4.0: Includes SPS reader

#### 2022-07-25 - v0.5.0 - _Mael-Le-Garrec_

Expand All @@ -188,7 +207,7 @@

#### 2022-05-30 - v0.4.0 - _jdilly_

- Added:
- Added:
- 2D amplitude detuning analysis and 3D plotting of the results
- Converter for amp.det. analysis from bbs to omc3
- general typehinting/doc/unification of entrypoint parameters/saving
Expand Down Expand Up @@ -253,12 +272,11 @@
- introducing pathlib.Path in some places
- output-paths in model job-files are relative

- Fixes:
- Fixed:
- Matplotlib warnings for `set_window_title`
- excluded Windows and MacOS py3.9 from normal testing, due to installation issues of pyTables
- model creation accepts relative and absolute paths


#### 2020-09-30

- Added:
Expand All @@ -275,31 +293,31 @@
#### 2020-03-04

- Added:
- lin-file natural tune updater
- lin-file natural tune updater

#### 2020-02-24

- Added:
- amplitude detuning analysis
- amplitude detuning and bbq plotting
- time tools
- plotting helpers
- amplitude detuning analysis
- amplitude detuning and bbq plotting
- time tools
- plotting helpers
- Distinction between `BasicTests` and `Extended Tests`

#### Before 2020-02

- Updated and moved main functionalities from python 2.7
- Madx wrapper
- Frequency Analysis of turn by turn
- Optics measurement analysis scripts
- Accelerator class and Model Creator
- K-mod
- Spectrum Plotting
- Turn-by-Turn Converter
- Madx wrapper
- Frequency Analysis of turn by turn
- Optics measurement analysis scripts
- Accelerator class and Model Creator
- K-mod
- Spectrum Plotting
- Turn-by-Turn Converter

- `setup.py` and packaging functionality
- Automated CI
- Multiple versions of python
- Accuracy tests
- Unit tests
- Release automation
- Multiple versions of python
- Accuracy tests
- Unit tests
- Release automation
2 changes: 1 addition & 1 deletion omc3/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
__title__ = "omc3"
__description__ = "An accelerator physics tools package for the OMC team at CERN."
__url__ = "https://github.com/pylhc/omc3"
__version__ = "0.14.1"
__version__ = "0.15.0"
__author__ = "pylhc"
__author_email__ = "[email protected]"
__license__ = "MIT"
Expand Down
15 changes: 11 additions & 4 deletions omc3/check_corrections.py
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@
from omc3.correction.response_twiss import PLANES
from omc3.definitions.optics import (
OpticsMeasurement, ColumnsAndLabels,
FILE_COLUMN_MAPPING, RDT_COLUMN_MAPPING, TUNE_COLUMN
FILE_COLUMN_MAPPING, RDT_COLUMN_MAPPING, TUNE_COLUMN, TOTAL_PHASE_NAME, MU_COLUMN
)
from omc3.global_correction import _get_default_values, CORRECTION_DEFAULTS, OPTICS_PARAMS_CHOICES
from omc3.model import manager
Expand Down Expand Up @@ -475,8 +475,10 @@ def _create_model_and_write_diff_to_measurements(
diff_columns = (
list(OPTICS_PARAMS_CHOICES[:-4]) +
[col for col in corr_model_elements.columns if col.startswith("F1")] +
list(PLANES)
list(PLANES) +
['MUX', 'MUY']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also probably be from constants

)

diff_models = diff_twiss_parameters(corr_model_elements, accel_inst.model, parameters=diff_columns)
LOG.debug(f"Differences to nominal model calculated.")

Expand Down Expand Up @@ -514,12 +516,17 @@ def _create_model_and_write_diff_to_measurements(
LOG.debug(f"Checking correction for {attribute}")
plane = filename[-1].upper()
cols = colmap.set_plane(plane)
if filename[:-1] == TOTAL_PHASE_NAME:
model_cols = MU_COLUMN.set_plane(plane)
else:
model_cols = cols

_create_check_columns(
measurement=measurement,
output_measurement=output_measurement,
diff_models=diff_models,
colmap_meas=cols,
colmap_model=cols,
colmap_model=model_cols,
attribute=attribute,
rms_mask=rms_mask,
)
Expand Down Expand Up @@ -571,7 +578,7 @@ def _create_check_columns(measurement: OpticsMeasurement, output_measurement: Op
diff = diff_models.loc[df.index, colmap_model.delta_column]

df[colmap_meas.diff_correction_column] = diff
if colmap_meas.column == PHASE:
if colmap_meas.column[:-1] == PHASE:
df[colmap_meas.expected_column] = pd.to_numeric(ang_diff(df[colmap_meas.delta_column], diff)) # assumes period 1
df.headers[colmap_meas.delta_rms_header] = circular_rms(df[colmap_meas.delta_column], period=1)
df.headers[colmap_meas.expected_rms_header] = circular_rms(df[colmap_meas.expected_column], period=1)
Expand Down
98 changes: 98 additions & 0 deletions omc3/correction/arc_by_arc.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
import numpy as np
import pandas as pd


def identify_closest_arc_bpm_to_ip(ip, side, beam, bpms):
indices = range(1,15)
for ii in indices:
bpm = f'BPM.{ii}{side}{ip}.B{beam}'
if bpm in bpms:
return bpm


def get_left_right_pair(arc, beam, bpms):
left_of_arc = identify_closest_arc_bpm_to_ip(int(arc[0]), 'R', beam, bpms)
right_of_arc = identify_closest_arc_bpm_to_ip(int(arc[1]), 'L', beam, bpms)
return [left_of_arc, right_of_arc]


def get_arc_by_arc_bpm_pairs(meas_dict, opt, plane):
bpms = meas_dict[f'PHASE{plane}'].index
beam = bpms[0][-1]
bpm_pairs = {}
bpm_pairs_with_ips = {}

arcs_to_cycle = ['81', '12', '23', '34', '45', '56', '67', '78']

for lhc_arc in arcs_to_cycle:
bpm_pairs[lhc_arc] = get_left_right_pair(lhc_arc, beam, bpms)

if opt.include_ips_in_arc_by_arc == 'left':
bpm_pairs_with_ips['81'] = [bpm_pairs['78'][1], bpm_pairs['81'][1]]
bpm_pairs_with_ips['12'] = [bpm_pairs['81'][1], bpm_pairs['12'][1]]
bpm_pairs_with_ips['23'] = [bpm_pairs['12'][1], bpm_pairs['23'][1]]
bpm_pairs_with_ips['34'] = [bpm_pairs['23'][1], bpm_pairs['34'][1]]
bpm_pairs_with_ips['45'] = [bpm_pairs['34'][1], bpm_pairs['45'][1]]
bpm_pairs_with_ips['56'] = [bpm_pairs['45'][1], bpm_pairs['56'][1]]
bpm_pairs_with_ips['67'] = [bpm_pairs['56'][1], bpm_pairs['67'][1]]
bpm_pairs_with_ips['78'] = [bpm_pairs['67'][1], bpm_pairs['78'][1]]
bpm_pairs = bpm_pairs_with_ips
elif opt.include_ips_in_arc_by_arc == 'right':
bpm_pairs_with_ips['81'] = [bpm_pairs['78'][0], bpm_pairs['81'][0]]
bpm_pairs_with_ips['12'] = [bpm_pairs['81'][0], bpm_pairs['12'][0]]
bpm_pairs_with_ips['23'] = [bpm_pairs['12'][0], bpm_pairs['23'][0]]
bpm_pairs_with_ips['34'] = [bpm_pairs['23'][0], bpm_pairs['34'][0]]
bpm_pairs_with_ips['45'] = [bpm_pairs['34'][0], bpm_pairs['45'][0]]
bpm_pairs_with_ips['56'] = [bpm_pairs['45'][0], bpm_pairs['56'][0]]
bpm_pairs_with_ips['67'] = [bpm_pairs['56'][0], bpm_pairs['67'][0]]
bpm_pairs_with_ips['78'] = [bpm_pairs['67'][0], bpm_pairs['78'][0]]
bpm_pairs = bpm_pairs_with_ips

return bpm_pairs


def circular_sum_phase(phase_df, tune, bpm_pair, key):
idx_0 = phase_df[key].index.get_loc(bpm_pair[0])
idx_1 = phase_df[key].index.get_loc(bpm_pair[1])
if idx_0 > idx_1:
inverted_result = sum(phase_df[key][bpm_pair[1]:bpm_pair[0]])
result = tune - inverted_result
else:
result = sum(phase_df[key][bpm_pair[0]:bpm_pair[1]])
return result


def circular_sum_phase_error(phase_df, bpm_pair):
idx_0 = phase_df['ERROR'].index.get_loc(bpm_pair[0])
idx_1 = phase_df['ERROR'].index.get_loc(bpm_pair[1])
if idx_0 > idx_1:
selection = pd.concat([phase_df['ERROR'].loc[:bpm_pair[1]], phase_df['ERROR'].loc[bpm_pair[0]:]])
result = np.sqrt(np.sum(selection**2))
else:
result = np.sqrt(np.sum(phase_df['ERROR'][bpm_pair[0]:bpm_pair[1]]**2))
return result

def get_arc_phases(bpm_pairs, meas_dict, tune, plane):
arc_meas = []
for arc, bpm_pair in bpm_pairs.items():
results = {}
results['NAME'] = bpm_pair[0]
results['NAME2'] = bpm_pair[1]
results['WEIGHT'] = meas_dict[f'PHASE{plane}'].loc[bpm_pair[0], 'WEIGHT']
results['VALUE'] = circular_sum_phase(meas_dict[f'PHASE{plane}'], tune, bpm_pair, 'VALUE')
results['MODEL'] = circular_sum_phase(meas_dict[f'PHASE{plane}'], tune, bpm_pair, 'MODEL')
results['ERROR'] = circular_sum_phase_error(meas_dict[f'PHASE{plane}'], bpm_pair)
results['DIFF'] = results['VALUE'] - results['MODEL']
arc_meas.append(results)

meas_dict[f'PHASE{plane}'] = pd.DataFrame(arc_meas).set_index('NAME')

return meas_dict


def reduce_to_arc_extremities(meas_dict, nominal_model, opt):
bpm_pairs_x = get_arc_by_arc_bpm_pairs(meas_dict, opt, "X")
bpm_pairs_y = get_arc_by_arc_bpm_pairs(meas_dict, opt, "Y")
meas_dict = get_arc_phases(bpm_pairs_x, meas_dict, nominal_model.headers['Q1'], 'X')
meas_dict = get_arc_phases(bpm_pairs_y, meas_dict, nominal_model.headers['Q2'], 'Y')
return meas_dict
Loading
Loading