Skip to content

Commit

Permalink
winsorizing, finally without the extra D
Browse files Browse the repository at this point in the history
  • Loading branch information
ThomasLecocq committed Feb 7, 2024
1 parent 8354cc7 commit fd649bb
Show file tree
Hide file tree
Showing 7 changed files with 15 additions and 17 deletions.
2 changes: 0 additions & 2 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,6 @@ citing MSNoise:
for Monitoring Seismic Velocity Changes Using Ambient Seismic Noise,
*Seismological Research Letters*, 85(3), 715‑726, doi:10.1785/0220130073.

This documentation is also available in PDF format on the MSNoise Website
(PDF_).

Installation
============
Expand Down
2 changes: 1 addition & 1 deletion doc/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Note that MSNoise is always tested against the latest release versions of the ma

To run MSNoise, you need:

* A recent version of Python (3.x recommended). We suggest using Miniconda_
* A recent version of Python (3.11 recommended). We suggest using Miniconda_
and creating a fresh environment for running msnoise.
MSNoise is tested "continuously" on GitHub for the last 3 most recent python version, and the three OS.

Expand Down
2 changes: 1 addition & 1 deletion doc/releasenotes/msnoise-1.3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ Some improvements to the maths have been done for MSNoise 1.3:
* ``whiten``: the symmetric hermitian was not properly defined and could lead to
a 1 sample shift in the negative frequencies.
* ``compute_cc``: it is now possible to define an overlap of the windows
* ``compute_cc``: setting ``windsorizing`` to ``-1`` now computes the 1-bit
* ``compute_cc``: setting ``winsorizing`` to ``-1`` now computes the 1-bit
normalization of the trace. Reminder: ``0``: no normalization, ``N``: N*rms
clipping.
* ``mwcs``: the tapering of the windowed CCF has been improved in order to
Expand Down
2 changes: 1 addition & 1 deletion doc/releasenotes/msnoise-1.5.rst
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ Improvements in terms of performances have also been done for MSNoise 1.5:
* Added fftpack optimized nfft (scipy's next_fast_len). This could lead to some
small differences in the final result of the MWCS procedure, because of the
number of points used for smoothing the (cross-)spectra.
* Replaced binarization (sign) and windsorizing (clip) by standard numpy
* Replaced binarization (sign) and winsorizing (clip) by standard numpy
functions operating directly inplace on the arrays, avoiding unecessary
copies.
* The preprocessing only reads files that should contain the right component.
Expand Down
2 changes: 1 addition & 1 deletion msnoise/default.csv
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ response_prefilt,"(0.005, 0.006, 30.0, 35.0)","Remove instrument correction pre-
maxlag,120,Maximum lag (in seconds),float,,,
corr_duration,1800,Data windows to correlate (in seconds),float,,,
overlap,0,Amount of overlap between data windows [0:1[,float,,,
windsorizing,3,"Windsorizing at N time RMS , 0 disables windsorizing, -1 enables 1-bit normalization",float,,,
winsorizing,3,"Winsorizing at N time RMS , 0 disables winsorizing, -1 enables 1-bit normalization",float,,,
whitening,A,"Whiten Traces before cross-correlation: [A]ll (except for autocorr), [N]one, or only if [C]omponents are different",str,A/N/C,,
whitening_type,B,"Type of spectral whitening function to use: [B]rutal (amplitude to 1.0), divide spectrum by its [PSD] or by band-passing the white spectrum with a hanning-window [HANN]. WARNING: only works for compute_cc, not compute_cc_rot, where it will always be B",str,B/PSD/HANN,,
clip_after_whiten,N,"Do the clipping (winsorizing) after whitening?",bool,Y/N,,
Expand Down
10 changes: 5 additions & 5 deletions msnoise/s03compute_cc.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* |overlap|
* |maxlag|
* |corr_duration|
* |windsorizing|
* |winsorizing|
* |resampling_method|
* |remove_response|
* |response_format|
Expand Down Expand Up @@ -71,7 +71,7 @@
*station* table. The rotation is computed for Radial and Transverse components.
Then, for each ``corr_duration`` window in the signal, and for each filter
configured in the database, the traces are clipped to ``windsorizing`` times
configured in the database, the traces are clipped to ``winsorizing`` times
the RMS (or 1-bit converted) and then whitened in the frequency domain
(see :ref:`whiten`) between the frequency bounds. The whitening procedure can be
skipped by setting the ``whitening`` configuration to `None`. The two other
Expand Down Expand Up @@ -340,10 +340,10 @@ def main(loglevel="INFO"):
tmp.detrend("demean")

for tr in tmp:
if params.windsorizing == -1:
if params.winsorizing == -1:
np.sign(tr.data, tr.data) # inplace
elif params.windsorizing != 0:
rms = tr.data.std() * params.windsorizing
elif params.winsorizing != 0:
rms = tr.data.std() * params.winsorizing
np.clip(tr.data, -rms, rms, tr.data) # inplace

# TODO should not hardcode 4 percent!
Expand Down
12 changes: 6 additions & 6 deletions msnoise/s03compute_no_rotation.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
* |overlap|
* |maxlag|
* |corr_duration|
* |windsorizing|
* |winsorizing|
* |resampling_method|
* |remove_response|
* |response_format|
Expand Down Expand Up @@ -102,7 +102,7 @@
*station* table. The rotation is computed for Radial and Transverse components.
Then, for each ``corr_duration`` window in the signal, and for each filter
configured in the database, the traces are clipped to ``windsorizing`` times
configured in the database, the traces are clipped to ``winsorizing`` times
the RMS (or 1-bit converted) and then whitened in the frequency domain
(see :ref:`whiten`) between the frequency bounds. The whitening procedure can be
skipped by setting the ``whitening`` configuration to `None`. The two other
Expand Down Expand Up @@ -227,15 +227,15 @@ def winsorizing(data, params, input="timeseries", nfft=0):
data = sf.ifftn(data, [nfft, ], axes=[1, ]).astype(float)

for i in range(data.shape[0]):
if params.windsorizing == -1:
if params.winsorizing == -1:
np.sign(data[i], data[i]) # inplace
elif params.windsorizing != 0:
elif params.winsorizing != 0:

# imin, imax = scoreatpercentile(data[i], [0.1, 99.9])
# not_outliers = np.where((data[i] >= imin) &
# (data[i] <= imax))[0]
# rms = data[i][not_outliers].std() * params.windsorizing
rms = data[i].std() * params.windsorizing
# rms = data[i][not_outliers].std() * params.winsorizing
rms = data[i].std() * params.winsorizing
np.clip(data[i], -rms, rms, data[i]) # inplace
if input == "fft":
data = sf.fftn(data, [nfft, ], axes=[1, ])
Expand Down

0 comments on commit fd649bb

Please sign in to comment.