7.4.1 - log.level not showing for all, while available

[vincent-fuchs]

Hi,

I am ingesting logs from Filebeat through Logstash and visualizing them using Logtrail 7.4.1-0.1.31 .

despite having standard naming for the fields, I have one application for which the log.level doesn't show in logtrail, while present in the json doc. For other applications, it works, so I am a bit confused..

here's my config : I am expecting to display the log.level along with the message :

  "fields" : {
    "mapping" : {
        "timestamp" : "@timestamp",
        "hostname" : "host.name",
        "program": "service.name",
        "message": "message"
    },
    "message_format": "{{{log.level}}} - {{{message}}}",
    "keyword_suffix" : "keyword"
  },
  "color_mapping" : {
    "field": "log.level",
    "mapping": {
      "ERROR": "#FF0000",
      "WARN": "#FFEF96",
      "DEBUG": "#B5E7A0",
      "TRACE": "#CFE0E8"
    }
  }

here's what I see in Logtrail :

But if I look into the json document, log.level is there :

For another application, it works as expected, ie log.level shows up :

while the json document is very similar to the one that doesn't work :

Do you have any idea of what could be happening here ?

Thanks !

[sivasamyk]

@vincent-fuchs This seems strange. Did you try adding some other field in message format?

[vincent-fuchs]

yes, I tried few things and still very strange, as it works :

• I added type field, and I could see "filebeat" as value
• I added another nested field, log.logger, and it works also..

only log.level doesn't seem to show for this program ... maybe there are other program s for which it doesn't work, but for now, it's the only one I have noticed..

[vincent-fuchs]

OK, I found the issue !

It came from my logstash pipeline, that for this application was first groking the value into a log_level field, that was then renamed into log.level .

Other applications were grokked directly into [log][level]. The confusion is that both will show the same way in the document "table view" in Kibana. However, the Json view show a clear difference :

• one has a proper nested field
• the other has one field named "log.level" - and that doesn't work.

I hope that can help others !! Do you want me to propose a small "caveats" section somewhere in the documentation (please tell me where) ?

[sivasamyk]

Happy to hear you found the root cause. This appears to be confusion due to the way kibana displays nested fields. We can add note here at the end, where we talk about nested fields.. https://github.com/sivasamyk/logtrail/blob/master/docs/add_fields.md