Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

query_ingesters_within didn't work as expected #14984

Open
yukinami opened this issue Nov 18, 2024 · 0 comments
Open

query_ingesters_within didn't work as expected #14984

yukinami opened this issue Nov 18, 2024 · 0 comments

Comments

@yukinami
Copy link

yukinami commented Nov 18, 2024

Describe the bug
query_ingesters_within didn't work as expected

To Reproduce
Steps to reproduce the behavior:

  1. set reject_old_samples: false, query_ingesters_within=168h', max_chunk_age=2h`
  2. Started Loki 3.2.1
  3. Started Promtail 3.1.2 to tail logs with settings
server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: ./positions.yaml

clients:
  - url: http://10.160.2.104:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: mes
      __path__: D:\Develop\Opt\promtail-windows-amd64\logs\mes\*log
      stream: stdout
  - targets:
      - localhost
    labels:
      job: training
      __path__: D:\Develop\Opt\promtail-windows-amd64\logs\training\*log
      stream: stdout
  pipeline_stages:
  - match:
      selector: '{job="training"}'
      stages:
        - regex:
            expression: '^(?s)(?P<timestamp>\S+?\s+\S+?\s+\S+?) \[(?P<level>\S+?)\] (?P<content>.*)$'
        - timestamp:
            format: "2006-01-02 15:04:05.000 -07:00"
            source: timestamp
        - labels:
            level:

  1. Query: {job="training"} term

I imported some logs which were logged 4 days ago, and I hoped to query the logs immediately after I imported. I read this blog
According to my understanding, the logs of 4 days ago I justed imported has not yet be flushed, and with the default 'query_ingesters_within=2h' settings, the query searching logs for thoese logs will directly go to the store, which would match nothing.
SoI set query_ingesters_within=168h and the query (searching logs of 4 days ago)should be executed on the ingester (where the logs data exist currently). But still I return nothing. Only If I set both max_chunks_age= 168h and query_ingesters_within= 168h, It would return data.

Expected behavior
With setting query_ingesters_within= 168h, loki should return historical data immediately.

Environment:

  • Infrastructure: bare-metal
  • Deployment tool: mannully

Screenshots, Promtail config, or terminal output

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant