Strange anomaly scores of DWT_MLEAD with some basic time series

[Noskario]

Have a look at the following code:

import numpy as np
from aeon.anomaly_detection import DWT_MLEAD
import plotly.express as px
N = 100
X = np.concatenate((np.full(N,0),np.linspace(0,1,N),np.full(N,1)), dtype=np.float_)
detector = DWT_MLEAD(
   #start_level=1, quantile_boundary_type='percentile', quantile_epsilon=0.01
)
px.scatter(y=X, color=detector.fit_predict(X))

You get outlier-scores that are unintuitive:

Why are the points at the right outliers? Why not the ones around 100 and 200? Does it have to do with padding?

[baraline]

Hey, @CodeLionX might have some element of answer on this point, unfortunately I'm not that familiar with the method to interpret these results. (He may be on holiday for now until the 8th).
Do you get similar results with other methods ?

[Noskario]

I looked at the Haar-wavelet-decomposition of this function and it looks almost exactly like the original signal in each step (approximating coefficients) or like a washed out step function (detail coefficients), so at least I would expect the anomaly score to be symmetric.
Here you have some R-code:

library(wavelets)
x <- c(
  rep(0, 100),
  (0:99)/100,
  rep(1,100)
)
ht <- dwt(x, filter = "haar")
ht@V$V4 |> plot()
ht@W$W4 |> plot()

If you mean "other anomaly detection" methods, then no, I do not get similar results. E.g.

import numpy as np
from aeon.anomaly_detection import DWT_MLEAD, KMeansAD
import plotly.express as px
N = 100
X = np.concatenate((np.full(N,0),np.linspace(0,1,N),np.full(N,1)), dtype=np.float_)
detector_dwt_mlead = DWT_MLEAD()
detector_kmeansad = KMeansAD()
px.scatter(y=X, color=detector_kmeansad.fit_predict(X))

Thank you for your quit response! Let's wait for CodeLionX then.

[SebastianSchmidl]

Hi @Noskario,

Thank you for your patience.

You are right. Your strange anomaly scores are an artifact of the padding to the next power of 2. This plot shows the padded series in row two:

We can see that the last part of the series just occurs once and thus has the highest anomaly score. Unfortunately, DWT-MLEAD does not work well on non-periodic time series or time series with no reoccurring patterns, especially if the time series is very short.

Script to reproduce figure:

import matplotlib.pyplot as plt
import numpy as np
from aeon.anomaly_detection import DWT_MLEAD
from aeon.anomaly_detection._dwt_mlead import _pad_series

N = 100


def main():
    X = np.concatenate(
        (np.full(N, 0), np.linspace(0, 1, N), np.full(N, 1)), dtype=np.float_
    )
    fig, axs = plt.subplots(3, sharex="all", layout="tight")
    axs[0].set_title("Original")
    axs[0].scatter(x=np.arange(X.shape[0]), y=X)

    padded_x, _, _ = _pad_series(X)
    axs[1].set_title("Padded")
    axs[1].plot(padded_x)

    detector = DWT_MLEAD()
    scores = detector.fit_predict(X)
    axs[2].set_title("Scores")
    axs[-1].plot(scores)
    plt.show()


if __name__ == "__main__":
    main()