Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grafana getting incident plugin settings 404 #5253

Closed
Ennakin opened this issue Nov 13, 2024 · 0 comments
Closed

Grafana getting incident plugin settings 404 #5253

Ennakin opened this issue Nov 13, 2024 · 0 comments
Labels
bug Something isn't working part:helm/kubernetes/docker

Comments

@Ennakin
Copy link

Ennakin commented Nov 13, 2024

What went wrong?

What happened:

  • Alerts aren't sent through Grafana Oncall anymore.In "Contact points' I see 'last delivery attempt... minutes ago'. I'm sure that notification policy and oncall routes are configured correctly, cause it worked. There is nothing useful in logs, but some wierd errors:
msg="getting incident plugin settings" error="request did not return 200: 404"  
msg="getting labels plugin settings" error="request did not return 200: 404"

I'm running grafana in enterprise (from v1.11.0 to main-ubuntu - we tried every version), self hosted mode. Incident or lables plugins seem to be а part of Grafana Cloud infrastucture. Something went wrong here. Can it be associated with undelivered alerts? In OnCall logs we also get
status=404 method=GET path=/api/internal/v1/msteams/channels

  • something from the Cloud instance mode.
    What did you expect to happen:
  • receive alerts through grafana oncall. Or at least have relevant errors in log.

How do we reproduce it?

  1. Configure an alert rule in folder "incident". In Grafana Oncall Integration add route with
    {{ payload.commonLabels.grafana_folder == 'incident' }} and telegram channel selected.
  2. Get the alert firing.
  3. Nothing going to happend.
    "Send demo alert" is working though.

Grafana OnCall Version

OnCall version is 1.11.0, plugin version is 1.9.26.

Product Area

Helm/Kubernetes/Docker

Grafana OnCall Platform?

Docker

User's Browser?

No response

Anything else to add?

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working part:helm/kubernetes/docker
Projects
None yet
Development

No branches or pull requests

1 participant